Parameter optimization apparatus, object tracking system, parameter optimization method, and computer program

Abstract

To provide technique for easily optimizing a parameter in a parameter optimization system.SOLUTION: A parameter optimization apparatus includes: an object time series acquisition unit which acquires an object time series including information on positions of an object at each time; a ground-truth time series acquisition unit which acquires a ground-truth time series including information pertaining to a movement start time, a movement end time and an identification number of an object measured in parallel with acquisition of the object time series; a tracking time series acquisition unit which acquires a tracking time series including information pertaining to the result of tracking the object, output from an object tracker by inputting a parameter and the object time series to the object tracker; a storage unit which stores a time-series set obtained by associating the object time series, the ground-truth time series, and the tracking time series; an evaluation unit which evaluates the parameter set to the object tracker using the time-series set; a search unit which searches for a new parameter using the result evaluated by the evaluation unit; and an output unit which outputs the parameter searched by the search unit.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a parameter optimization device, an object tracking system, a parameter optimization method, and a computer program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, a parameter optimization device that optimizes parameters of an object tracker that tracks a moving object has been known (Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Zhaozhong Chen, Harel Biggie, Nisar Ahmed, Simon Julier, Christoffer Heckman, "Kalman Filter Auto-tuning through Enforcing Chi-Squared Normalized Error Distributions with Bayesian Optimization" 34 pages, 9 figures, submitted to IEEE Transactions on Aerospace and Electronic Systems, [Retrieved July 19, 2023], Internet<https: / / arxiv.org / abs / 2306.07225> Summary of the Invention [Problem to be solved by the invention]

[0004] However, even with the prior art such as that disclosed in Non-Patent Document 1, there is still room for improvement in terms of the technique for easily optimizing parameters in a parameter optimization device.

[0005] The present invention has been made to solve the above-mentioned problems, and has an object to provide a technique that enables parameters to be easily optimized in a parameter optimization device. [Means for solving the problem]

[0006] The present invention has been made to solve at least part of the above-mentioned problems, and can be realized in the following forms.

[0007] (1) According to one aspect of the present invention, there is provided a parameter optimization device that optimizes parameters set in an object tracker that tracks an object. The parameter optimization device includes an object time series acquisition unit that acquires an object time series including information on the position of the object at each time, a correct time series acquisition unit that acquires a correct time series including information on a movement start time, a movement end time, and an identification number of an object for which the object time series is acquired, measured in parallel with the acquisition of the object time series, a tracking time series acquisition unit that acquires a tracking time series output from the object tracker by inputting the object time series to the object tracker, the tracking time series including information on a tracking result of the object calculated using the parameters and the object time series, a storage unit that stores a time series set in which the object time series, the correct time series, and the tracking time series are linked, an evaluation unit that evaluates parameters set in the object tracker using the time series set stored in the storage unit, a search unit that searches for new parameters to be set in the object tracker using the evaluation result by the evaluation unit, and an output unit that outputs the parameters searched by the search unit.

[0008] According to this configuration, the ground truth time series including information on the object's movement start time, movement end time, and identification number is measured in parallel with the acquisition of the object time series including information on the object's position at each time. This makes it easy to acquire the ground truth time series, and therefore makes it easy to compare the tracking time series including information on the object tracking result acquired by inputting the parameters and the object time series to the object tracker with the ground truth time series. Therefore, the parameters can be easily optimized.

[0009] (2) The parameter optimization device of the above embodiment may further include a trade-off generating unit that processes the time series set so that a trade-off occurs between an increase and a decrease in parameter value for at least two parameters among the multiple parameters set in the object tracker. According to this configuration, the trade-off generating unit generates a trade-off between an increase and a decrease in parameter value for the multiple parameters set in the object tracker. This makes it easier to optimize the parameters.

[0010] (3) In the parameter optimization device of the above embodiment, the correct time series acquisition unit may include a signal transmitter provided on the object, the signal transmitter emitting a signal including information about the identification number at the time when the object starts moving and the time when the object stops moving, and a signal receiver capable of receiving the signal emitted by the signal transmitter, and the correct time series may be acquired using the signal received by the signal receiver. According to this configuration, the correct time series is measured based on the signal emitted by the signal transmitter provided on the object. This allows the correct time series acquisition unit to easily and reliably acquire the correct time series in parallel with the acquisition of the object time series, making it easier to compare the tracking time series with the correct time series. Therefore, the parameters can be further easily optimized.

[0011] (4) In the parameter optimization device of the above embodiment, the object time series may include information on the position of the object at the movement start time and information on the position of the object at the movement end time. According to this configuration, the object time series includes information on the position of the object at each of the movement start time and the movement end time. This makes it possible to reliably compare the tracking time series output from the object tracker by inputting the object time series with the correct time series at each of the movement start and movement end, thereby making it possible to reliably search for parameters.

[0012] (5) In the parameter optimization device of the above embodiment, the correct time series acquisition unit may acquire the correct time series by taking the time when the object starts moving after stopping for a predetermined time as the movement start time, and taking the time when the moving object stops and continues to stop for a predetermined time as the movement end time. According to this configuration, the movement start time and the movement end time are set before and after the object stops for the predetermined time. This makes it clear when the object starts moving and when it stops moving in the object time series, so that it is possible to reliably compare the tracking time series output by inputting the object time series with the correct time series. Therefore, it is possible to reliably search for parameters.

[0013] (6) In the parameter optimization device of the above embodiment, the object time series acquisition unit acquires a plurality of object time series corresponding to a plurality of different movement patterns of the object, the correct time series acquisition unit acquires the correct time series corresponding to each of the plurality of object time series, the tracking time series acquisition unit acquires the tracking time series corresponding to each of the plurality of object time series, the storage unit stores a time series set in which one object time series of the plurality of object time series is linked to a correct time series corresponding to the one object time series and a tracking time series corresponding to the one object time series, and the evaluation unit may use each of the plurality of time series sets to obtain a plurality of evaluation index values ​​for a parameter and evaluate the parameter set in the object tracker using the plurality of evaluation index values. According to this configuration, a plurality of time series sets are created corresponding to a plurality of different movement patterns of the object. The evaluation unit evaluates the parameter using a plurality of evaluation index values ​​for the parameter obtained using each of the plurality of time series sets. This allows the parameters to be evaluated taking into account each of a plurality of different movement patterns of the object, thereby improving the degree of optimization of the parameters.

[0014] (7) According to another aspect of the present invention, there is provided an object tracking system. This object tracking system includes the parameter optimization device described above, and the object tracker to which the parameters output by the parameter optimization device are input. According to this configuration, the parameters output by the parameter optimization device are set in the object tracker, thereby improving the object tracking accuracy of the object tracker. This makes it possible to prepare an object tracker with relatively high object tracking accuracy.

[0015] (8) According to yet another aspect of the present invention, there is provided a parameter optimization method for optimizing parameters set in an object tracker that tracks an object. The parameter optimization method includes a first step of acquiring an object time series including information on the position of the object at each time, a second step of acquiring a ground truth time series including information on a movement start time, a movement end time, and an identification number of an object from which the object time series is acquired, measured in parallel with the acquisition of the object time series, a third step of acquiring a tracking time series output from the object tracker by inputting the object time series to the object tracker, the tracking time series including information on a tracking result of the object calculated using the parameters and the object time series, a storage step of storing a time series set in which the object time series, the ground truth time series, and the tracking time series are linked, an evaluation step of evaluating parameters set in the object tracker using the time series set, a search step of searching for new parameters to be set in the object tracker using the evaluation result in the evaluation step, and an output step of outputting the parameters searched in the search step. According to this configuration, the ground truth time series acquired in the second step can be easily acquired because it is measured in parallel with the acquisition of the object time series including information on the object's position at each time. This makes it easy to compare the tracking time series with the ground truth time series in the evaluation step, and therefore makes it easy to optimize parameters in the search step.

[0016] (9) According to yet another aspect of the present invention, there is provided a computer program for causing a computer to execute optimization of parameters set in an object tracker that tracks an object. The computer program causes a computer to execute a first function of acquiring an object time series including information on the position of the object at each time, a second function of acquiring a ground truth time series including information on a movement start time, a movement end time, and an identification number of an object from which the object time series is acquired, measured in parallel with acquisition of the object time series, a third function of acquiring a tracking time series output from the object tracker by inputting the object time series to the object tracker, the tracking time series including information on a tracking result of the object calculated using the parameters and the object time series, a storage function of storing a time series set in which the object time series, the ground truth time series, and the tracking time series are linked, an evaluation function of evaluating parameters set in the object tracker using the time series set, a search function of searching for new parameters to be set in the object tracker using the evaluation result by the evaluation function, and an output function of outputting the parameters searched by the search function. According to this configuration, the ground truth time series acquired by the second function can be easily acquired because it is measured in parallel with the acquisition of the object time series including information on the object's position at each time. This makes it easy for the evaluation function to compare the tracking time series with the ground truth time series, so that the search function can easily optimize the parameters.

[0017] The present invention can be realized in various forms, for example, in the form of a system including a parameter optimization device, a control method for these devices and systems, a computer program for causing these devices and systems to perform parameter optimization, a server device for distributing the computer program, a non-transitory storage medium on which the computer program is stored, etc. [Brief description of the drawings]

[0018] [Figure 1] FIG. 1 is a schematic diagram illustrating an overview of an object tracking system according to a first embodiment. [Diagram 2] 1 is a diagram illustrating a schematic configuration of a parameter optimization device according to a first embodiment. [Diagram 3] FIG. 2 is a diagram illustrating a change in a captured image over time. [Figure 4] 2 is a flowchart of an object tracking method according to the first embodiment. [Diagram 5] FIG. 13 is a diagram illustrating an example of a tracking time series of a comparative example. [Figure 6] FIG. 4 is a diagram illustrating an example of a tracking time series in the first embodiment. [Figure 7] FIG. 11 is a diagram illustrating a schematic configuration of a parameter optimization device according to a second embodiment. [Figure 8] 13 is a flowchart of a parameter optimization method according to a second embodiment. [Figure 9] FIG. 11 is a schematic diagram for explaining an overview of a modified example of the object tracking system according to the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] First Embodiment FIG. 1 is a schematic diagram for explaining an overview of an object tracking system according to a first embodiment. The object tracking system 100 according to this embodiment is a system for tracking a moving object, and is used, for example, to collect information on the gender and age of people gathering at a specific location, and on the number and type of cars traveling on a road. As shown in FIG. 1, the object tracking system 100 according to this embodiment includes a parameter optimization device 1 and an object tracker 5. Note that FIG. 1 shows a state at a stage before the operation of the object tracker 5, at a stage where a plurality of parameters set in the object tracker 5 are optimized by the parameter optimization device 1.

[0020] First, the object tracker 5, whose parameters are optimized by the parameter optimization device 1 of this embodiment, will be described. The object tracker 5 inputs information such as feature values ​​such as the position at each time and the type of object for one or more objects, and outputs information that adds an identification number for identifying the same object while adding features derived from the input such as extrapolation when the input is missing, reduction of minute fluctuations in the input, and the time-dependent rate of change of features such as speed. Known object trackers include SORT (Simple Online and Realtime Tracking), DeepSORT, ByteTrack, etc., FairMOT (Fair Multiple Object Tracking) and TransMOT, which simultaneously detect and track objects. The parameters set in the object tracker 5 affect the quality of the output, such as the assignment of an identification number and extrapolation when the input is missing.

[0021] The parameter optimization device 1 optimizes parameters to be set in the object tracker 5 so as to obtain an appropriate output from the object tracker 5. In this embodiment, the parameter optimization device 1 includes a test camera 21, a mobile terminal 22, and a personal computer (Personal Computer: PC) 30.

[0022] The test camera 21 is prepared separately from the object tracker 5, and captures an image for adjusting parameters set in the object tracker 5. The test camera 21 is electrically connected to a personal computer 30, and transmits the captured image to the personal computer 30. In this embodiment, the test camera 21 captures an image A1 including two people H1 and H2, for example, as shown in FIG. 1. In FIG. 1, the direction D1 in which the person H1 moves is different from the direction D2 in which the person H2 moves. As a result, if the image A1 continues to be captured, it is expected that the people H1 and H2 will cross each other.

[0023] The two people H1 and H2 in the image A1 each carry a mobile terminal 22. The mobile terminal 22 can transmit various signals by being operated by each of the people H1 and H2. The signals transmitted by the mobile terminal 22 are received by the personal computer 30.

[0024] The personal computer 30 searches for optimal values ​​of parameters set in the object tracker 5 using the image captured by the test camera 21, the signal transmitted by the mobile terminal 22, and the information output by the object tracker 5 as a test, and outputs the searched parameters. The personal computer 30 includes an input / output terminal 31, a storage medium 32, and a CPU (Central Processing Unit) 33. The input / output terminal 31 is an interface with an external recording device such as a keyboard, a mouse, a display, a USB memory, or a flash memory, and a transceiver capable of transmitting and receiving signals, and receives various information and outputs information related to the searched parameters to the object tracker 5. The storage medium 32 is a general term for memory devices, and various types of storage devices such as a ROM (Read Only Memory), a RAM (Random Access Memory), a solid state drive (SSD), a hard disk (HDD), and a flash memory card can be used. The storage medium 32 stores information input by the input / output terminal 31, and contains a computer program for optimizing the parameters of the object tracker 5. The function of the storage medium 32 will be described in detail later. The CPU 33 executes the functions of various programs by expanding the programs stored in the ROM of the storage medium 32 into the RAM. Details of the functions executed by the CPU 33 will be described later. Note that, although the parameter optimization device 1 includes one personal computer 30 in this embodiment, it may be configured with a plurality of arithmetic devices.

[0025] Fig. 2 is a diagram showing a schematic configuration of the parameter optimization device 1 of the first embodiment. Fig. 2 is a diagram showing the relationship between functions of the parameter optimization device 1. The parameter optimization device 1 functions as an object time series acquisition unit 11, a correct answer time series acquisition unit 12, a tracking time series acquisition unit 13, a storage unit 14, an evaluation unit 15, a parameter search unit 16, and an output unit 17.

[0026] The object time series acquisition unit 11 includes the test camera 21 and the object detection unit 33a. As described above, the test camera 21 continuously captures images including moving people H1 and H2.

[0027] The object detection unit 33a obtains the type of each object, the confidence level that the object is of that type, and the circumscribing rectangle on the image of the object at each time point using the imaging results of the test camera 21. As a result, the object detection unit 33a creates an object time series including information on the position of a moving person at each time. The object detection unit 33a of this embodiment corresponds to the object detection function of the CPU 33. In this embodiment, since the imaging results of the test camera 21 are used in creating the object time series, the object detection unit 11a can use known means such as YOLO (You Only Look Once) using a neural network, SSD (Single Shot multibox Detector), a combination of features such as HOG (Histogram of Oriented Gradients), and a classification means such as SVM (Support Vector Machine). In this embodiment, the object detection unit 33a limits the type of object to be detected to only people in order to suppress erroneous detection of people. In addition, the object detection unit 33a may discard the detection result when the confidence level that the detected object is a "person" is low.

[0028] FIG. 3 is a diagram for explaining the change of a captured image over time. FIG. 3 shows images captured by the test camera 21 at three different times t1, t2, and t3. The three images in FIG. 3 are captured in the order of time t1, time t2, and time t3, and show a person H2 moving from the left side Ls to the right side Rs of the image behind a person H1 moving from the right side Rs to the left side Ls of the image. For these two people H1 and H2, circumscribing rectangles R1 and R2 are obtained in the image at time t1 and the image at time t3. However, in the image at time t2, the person H1 and the person H2 overlap, and it can be seen that no circumscribing rectangle is obtained for each of the people H1 and H2. For this reason, the object tracker 5 may not be able to track the moving person H2.

[0029] The correct time series acquisition unit 12 acquires a correct time series including information on the movement start time, movement end time, and identification number of the object whose object time series is acquired, which are measured in parallel with the acquisition of the object time series. The correct time series acquisition unit 12 of this embodiment has a mobile terminal 22 carried by each of the persons H1 and H2. Each of the persons H1 and H2 operates the mobile terminal 22 when they start moving and when they end moving, thereby notifying the personal computer 30 of the movement start time, which is the time when they started moving, and the movement end time, which is the time when they ended moving. These operations by the persons H1 and H2 are performed simultaneously with the imaging by the test camera 21. The function of the correct time series acquisition unit 12 will be described in detail later.

[0030] The tracking time series acquisition unit 13 inputs the object time series to the object tracker 5 to acquire the tracking time series output from the object tracker 5. The tracking time series acquisition unit 13 in this embodiment corresponds to a control function of the CPU 33 that controls the driving of the object tracker 5. The object tracker 5 is started by setting the initial values ​​of parameters or inputting the parameters searched by the parameter search unit 16 described later. When the object tracker 5 is started, the tracking time series acquisition unit 13 inputs the object time series acquired by the object time series acquisition unit 11 to the object tracker 5. As a result, the object tracker 5 outputs a tracking time series including information on the tracking results of the people H1 and H2 calculated using the parameters and the object time series. The tracking time series acquisition unit 13 acquires the tracking time series output by the object tracker 5. In this embodiment, the tracking time series acquisition unit 13 calculates a derived feature amount corresponding to the speed of the object, and makes it possible to evaluate the start and end of the movement of the person in the correct time series in association with the tracking time series.

[0031] The storage unit 14 stores a time series set in which the object time series acquired by the object time series acquisition unit 11, the correct time series acquired by the correct time series acquisition unit 12, and the tracking time series acquired by the tracking time series acquisition unit 13 are linked together. The storage unit 14 in this embodiment corresponds to a storage medium 32 provided in the personal computer 30. In this embodiment, one time series set corresponds to one pattern (movement pattern) related to the way a person moves. Therefore, in this embodiment, in order to optimize the parameters of the object tracker 5 for each of the multiple movement patterns, multiple time series sets are stored separately. Therefore, one correct time series and one tracking time series are linked together and stored in correspondence with the object time series of one movement pattern. As a method for storing such a time series set, in addition to a hard disk, a file system that can group individual electronic files by folders or directories, and a combination of file formats that can express a time series can be used. In this case, the time series set can be configured by, for example, storing files corresponding to the object time series, the ground truth time series, and the tracking time series in a group. Also, a relational database system or the like may be used instead of the file system and electronic files. The file system can be a known NTFS or the like, the file format can be JSON Lines, Apache Parquet, or the like, and the relational database system can be SQLite, or the like.

[0032] The evaluation unit 15 evaluates the parameters set in the object tracker 5 by using the time series set stored in the storage unit 14. The evaluation unit 15 in this embodiment corresponds to the evaluation function of the CPU 33. In this embodiment, the evaluation unit 15 compares the tracking time series with the correct time series for each of the multiple time series sets stored in the storage medium 32, and evaluates the parameters set in the object tracker 5. The function of the evaluation unit 15 will be described in detail later.

[0033] The parameter search unit 16 uses the evaluation result by the evaluation unit 15 to search for optimal values ​​of parameters set in the object tracker 5. The parameter search unit 16 in this embodiment corresponds to a parameter search function of the CPU 33. The parameter search unit 16 can use a computer program that performs known black-box optimization such as Bayesian optimization and genetic algorithm.

[0034] The output unit 17 outputs the parameters searched for by the parameter search unit 16 to the object tracker 5. The output unit 17 in this embodiment corresponds to an input / output terminal 31 that electrically connects the personal computer 30 and the object tracker 5.

[0035] Fig. 4 is a flowchart of the object tracking method of this embodiment. Next, the object tracking method by the object tracking system 100 will be described. Here, a method for optimizing parameters set in the object tracker 5 by the parameter optimization device 1 before actually operating the object tracker 5 will be described in detail. When executing the object tracking method shown in Fig. 4, it is assumed that any parameter, for example, an initial value of the parameter, has already been set in the object tracker 5.

[0036] In the object tracking method of this embodiment, first, as a preparation step for parameter optimization, an object time series and a correct answer time series are input (step S11). In step S11, the object time series acquisition unit 11 uses the test camera 21 to capture an image including moving people H1 and H2. The captured image is output to the object detection unit 33a. The object detection unit 33a uses the image input from the test camera 21 to create an object time series including information on the positions of the moving people H1 and H2 at each time. The object time series created by the object detection unit 33a is stored in the storage medium 32.

[0037] In step S11, in parallel with the image capture by the test camera 21, the movement start time and movement end time of each of the two people H1 and H2 included in the image are recorded using the mobile terminal 22 carried by each of the two people H1 and H2. Specifically, each of the two people H1 and H2 operates the mobile terminal 22 by voice, button, etc., and when starting to move, transmits a signal that allows the movement start time and their own identification number to be recognized, and when ending the movement, transmits a signal that allows the movement end time and their own identification number to be recognized. When the personal computer 30 receives the signal transmitted from the mobile terminal 22, it links it to the object time series stored in the storage medium 32. That is, in step S11, the object time series and the correct answer time series are input to the personal computer 30 at the same time. In this embodiment, the storage medium 32 stores a plurality of combinations of object time series and correct answer time series that are linked to each other.

[0038] In the input of the correct answer time series in step S11, in order to distinguish between the start and end of the movement of the two people H1 and H2, it is desirable that at least the positions where the two people H1 and H2 start moving and the positions where the two people H1 and H2 end moving are within a range that can be captured by the test camera 21. That is, it is desirable that the object time series includes information on the positions of the two people H1 and H2 at the start time of the movement and information on the positions of the two people H1 and H2 at the end time of the movement. In addition, the start time and end time of the movement of the two people H1 and H2 are not overlapped, and each of the two people H1 and H2 is stationary for a predetermined time or more before the start of the movement and after the end of the movement. By imposing such a control condition, it is possible to define a correct answer that does not depend on the positions of the two people H1 and H2 in the correct answer time series and to enable evaluation by the evaluation unit 15. In this case, it is acceptable for the person to pause between the start of the movement and the end of the movement.

[0039] If the mobile terminal 22 has an acceleration measuring function, the correct time series may be created by taking the time when the acceleration is maximum around the time when the operation to start the movement is performed as the movement start time and taking the time when the acceleration is minimum around the time when the operation to end the movement is performed as the movement end time. This makes it possible to suppress a decrease in accuracy due to a mismatch between the operation by a person and the actual movement of a person.

[0040] When the purpose of using the object tracker 5 is clear, information for optimization according to the purpose can be added to the correct time series. For example, when determining whether a person approaches a specific area, the time rate of change of the distance between the person and the specific area and the time rate of change of the size of a circumscribing rectangle in an image captured by a test camera installed in the specific area are obtained. In this case, the parameters for converting the distance and size of the circumscribing rectangle can be optimized by adding the time period from the start of approach to the specific area to the end of approach between the start of movement and the end of movement. As a method of adding information for optimization according to the purpose to the correct time series, it is possible to instruct the program running on the mobile terminal 22 to start and end the approach by voice, button operation, or the like.

[0041] Next, the object tracker 5 is driven to acquire a tracking time series (step S12). In step S12, the tracking time series acquisition unit 13 inputs the object time series stored in the storage medium 32 to the object tracker 5 activated by the parameter search unit 16. As a result, the object tracker 5 outputs the tracking results of the two people H1 and H2 to the tracking time series acquisition unit 13 based on the parameters already set. The tracking results of the two people H1 and H2 output to the tracking time series acquisition unit 13 are further linked as tracking time series to the combination of the object time series and the correct time series stored in the storage medium 32, and a time series set corresponding to one operation pattern is created. In this embodiment, the storage medium 32 stores a plurality of combinations of the object time series and the correct time series, and in step S12, each of the plurality of object time series is input to the object tracker 5, which outputs a tracking time series separately. Therefore, the storage medium 32 stores a plurality of time series sets in which the object time series and the tracking time series of the correct answer time series are linked together.

[0042] Next, the time series sets are evaluated (step S13). In step S13, the evaluation unit 15 obtains an evaluation index value for each of the multiple time series sets. The evaluation unit 15 evaluates the parameters set in the object tracker 5 by aggregating the obtained multiple evaluation index values.

[0043] In this embodiment, the evaluation unit 15 calculates the discrepancy d between the number of people included in the tracking time series and the number of people included in the correct answer time series. normal The evaluation index value d can be defined as set The parameters are evaluated by calculating the deviation d normal is the deviation for each person d obj Find the deviation d obj That is, it can be expressed by the following formula (1).

number

[0044] The deviation d shown on the left side of equation (1) normalThe larger the value, the lower the evaluation, and it directly corresponds to the quality of the essential parameters of the object tracker 5. It also indirectly corresponds to parameters related to input extrapolation by a Kalman filter or the like, reduction of minute fluctuations, and calculation of derived features.

[0045] deviation d obj is the identification number (start number: i begin ) and the similar identification number of the derived feature at the point after the movement end time and closest to the movement end time (end number: i end ) can be searched, it is approximated by the following equation (2).

number

[0046] The value of formula (2) is always 0 if the same person is being tracked, and if the person is tracked as a different person halfway through, the identification number is switched to a new number that has not been used since the object tracker 5 started operating, so the value is always greater than 0. That is, since the relationship in formula (3) below is generally satisfied, if the left side of formula (3) is 0, it can be said that the number of objects is not deviated.

number

[0047] Next, the starting number i begin and the end number i end Here is an example of a search with the following: The correct time series is the movement start time 2, the movement end time 5, and the identification number 4 (when multiple time series sets are input) for one person. In JSON Lines, which is one of the file formats that can express time series, the start number i begin is expressed by equation (4), and the end number i endis expressed by the formula (5). For convenience, the time is shown here without units.

number

[0048] On the other hand, if the parameters are not set appropriately, the tracking time series acquired by the tracking time series acquisition unit 13 may have switching of the identification number or splitting of the object. For this reason, if the moving speed of the object is expressed as a scalar by the tag "v" of JSON Lines, the tracking time series from time 0 to time 6 is expressed by, for example, the following formulas (6) to (12). Note that here, for convenience, the moving speed is shown without a unit.

number

[0049] In the formulas (6) to (12), an example is shown in which the identification number changes from 0 to 1 from time 3 to time 4, and object 1 is split into object 1 and object 2 from time 4 to time 5. In the example shown in the formulas (6) to (12), the starting number i begin The row containing i corresponds to equation (6), so the starting number i begin is 0. Also, the end number i end The row containing is equation (12), so the end number i end is 2, which corresponds to the smallest absolute value of the tag "v", 0.1. Therefore, the deviation d obj can be approximated by 2 using equation (2).

[0050] Start number i begin and the end number i end If either or both of the above cannot be found, it is possible that the person was out of the detection range at the relevant time, or was blocked by another person or object and could not be detected. In this case, it can be said that the control condition described in step S11 has been violated, so the relevant time series set will be excluded from the evaluation target. Note that if the identification numbers are assigned in ascending order due to circumstances that make it impossible to exclude them, the starting number i beginInstead of i, the smallest identification number may be used, or the final number i end Alternatively, the maximum identification number may be used instead of the number 1. Also, when searching for the closest time point, if it cannot be found within a time limit, the search may not be possible.

[0051] Start number i begin When searching for the identification number with the longest continuous minimum value of the norm when going back in time including the time closest to the start of movement, instead of the time closest to the start of movement, it is also possible to search for the identification number with the smallest representative value of the norm for a predetermined stop time in the correct time series acquisition unit 12 closest to the start of movement.

[0052] End number i end When searching for i, the starting number is the same as i, except that you search forward in time instead of backward. begin The process is similar to searching for the norm. Considering that the distribution is unpredictable, the median may be used as the representative value of the norm, or the maximum value representing the worst case, or the third quartile representing a case that is not as extreme as the worst case, may be used. In addition, if one is in a position where the influence of outliers is positively accepted, the average value, or the sum of the average value closer to the worst case and the standard deviation while adopting the concept of the average value, may be used.

[0053] Next, the evaluation index value d set An example of a method for aggregating the evaluation index value d set When aggregating, the time length of the object time series and the influence of the number of people may be reflected as they are. In addition, from the viewpoint of obtaining effective parameters for the complex movements of people, the evaluation index value d set The evaluation index value d may be normalized by dividing it by the time, the maximum number of people appearing at the same time, or by both the time and the maximum number of people appearing at the same time, before aggregation. setConsidering that the distribution is unpredictable, the representative value used when aggregating may be the median, the maximum value representing the worst case, or the third quartile representing cases that are not as extreme as the worst case. In addition, if one is in a position where the influence of outliers is positively accepted, the average value, or the sum of the average value closer to the worst case and the standard deviation while adopting the concept of the average value, may be used.

[0054] Evaluation index value o according to the purpose of use of the object tracker 5 set As an example of the above, a case will be described in which a time series of circumscribing rectangles of a person is input to the object tracker 5, and the rate of change over time of distance, size, etc., which indicates the tendency of a person to approach a specific area, is calculated. In this case, the evaluation index value o set is the degree of differentiation of the rate of change between when approaching (approach) and when not approaching (non-approach) for each person. obj The representative value of the rate of change during approach can be defined as v coming The representative value of the rate of change when not approaching is the representative value v other The degree of differentiation is expressed as o obj is calculated as the norm of the difference between the two, using the following equation (13).

number

[0055] In the tracking time series, under the assumption that the order of occurrence of events for a particular person is movement start, approach start, approach end, and movement end, object tracking tends to be more stable at the end of movement than at the start of movement. end By selecting the person with the same number as i, we can obtain the rate of change for each person when approaching and when not approaching. endIf the search cannot be performed, it can be said that the control condition described in step S11 has been violated. Therefore, the corresponding time series set is excluded from the evaluation, and the index value for the corresponding person may be set to 0 since it is considered that the person cannot be differentiated. end Substitute the maximum identification number and calculate the deviation d obj Similarly, for the person with the highest identification number, the degree of discrimination o obj You may ask for:

[0056] The rate of change at each time point is the representative value v coming and the representative value v other When aggregating, the median may be used as the representative value, taking into consideration that the distribution is unpredictable if the rate of change is a scalar. Also, if you are willing to accept the influence of outliers, the average may be used. obj From the evaluation index value o set When aggregating to , and the evaluation index value o set The representative value used when aggregating may be the median, taking into account that the distribution is unpredictable, or the minimum value representing the worst, or the first quartile as a representative of cases that are not as extreme as the worst. In addition, if one is in a position to actively accept the influence of outliers, it may be possible to use the average value, or the difference between the average value closer to the worst and the standard deviation while adopting the concept of the average value. Furthermore, the evaluation index value o set Prior to the aggregation of the evaluation index value o set A normalization may be performed by dividing by time.

[0057] Next, it is determined whether or not the termination condition is satisfied (step S14). In step S14, the evaluation unit 15 determines whether or not the evaluation result in step S13 satisfies a predefined termination condition. Here, the termination condition may be that the time required to optimize the parameters exceeds a predefined time limit, that a desired evaluation index value is obtained, or that either of these two conditions is satisfied. In step S14, if the evaluation unit 15 determines that the evaluation result satisfies the predefined termination condition (step S14: Yes), the process proceeds to step S15. In step S14, if the evaluation unit 15 determines that the evaluation result does not satisfy the predefined termination condition (step S14: No), the process proceeds to step S16.

[0058] If it is determined in step S14 that the evaluation result in step S13 satisfies a predetermined termination condition, the operation of the object tracker 5 is started (step S15). In step S15, the object tracker 5 performs object tracking using the parameters input in the previous step S13.

[0059] In step S14, if it is determined that the evaluation result in step S13 does not satisfy the predetermined termination condition, new parameters are generated (step S16). In step S16, the parameter search unit 16 searches for and generates new parameters as a black-box optimization problem such as Bayesian optimization or a genetic algorithm. For example, when Optuna, a program capable of multi-objective optimization, is used, a Pareto-optimal parameter candidate can be obtained in a case where comprehensive evaluation is difficult due to different meanings of evaluation index values, such as differentiation between deviation and rate of change in the evaluation unit 15, and thus unnecessary trial and error is not required. In addition, parameter candidates may be enumerated based on the evaluation index value obtained by having the evaluation unit 15 perform evaluation using a part of the time-series set, and the evaluation unit 15 may perform evaluation using the parameter candidates and the remainder of the time-series set. In this case, candidates whose evaluation index values ​​calculated using the parameter candidates and the remainder of the time-series set are worse than the evaluation index values ​​when the parameter candidates were enumerated, or candidates whose evaluation can be said to be poor in comparison with a threshold value may be excluded. For example, for machine learning, a set ratio such as 75% may be used to split the data into training and testing data, as in the well-known sklearn.model_selection.train_test_split. This makes it possible to narrow down parameter candidates and prevent overfitting (overlearning in machine learning).

[0060] After step S16, the process returns to step S12, and the object tracker 5 is driven using the newly generated parameters and the object time series stored in the storage medium 32 to obtain a new tracking time series. In the parameter optimization method of this embodiment, if the termination condition in step S14 is not satisfied in this manner, the optimization process consisting of steps S12, S13, S14, and S16 is repeated to optimize the parameters.

[0061] FIG. 5 is a diagram showing a tracking time series of a comparative example. FIG. 5 shows, as a comparative example, the tracking results of two people output by an object tracker whose parameters are not optimized. In the tracking results shown in FIG. 5, the horizontal axis shows time, and the vertical axis shows the horizontal position (the position between the right end and the left end in the image) in an image such as that shown in FIG. 1 captured by a test camera. The tracking results in FIG. 5 show that two people H1 and H2 move so as to cross each other, as described in FIG. 1. Here, the tracking time series of person H1 is shown by a circle with identification number 1, and the tracking time series of person H2 is shown by a triangle with identification number 2. It can be seen that the object tracker of the comparative example is unable to track person H2 after person H1 and person H2 cross at time t00, and loses sight of person H2. Furthermore, even if the object tracker of the comparative example becomes able to track person H2 again at time t01, identification number 3 is given as the tracking time series of another person (time series shown by x marks), as shown in FIG. 5. In this way, if the parameters are not optimized, for example, the tracking result may be that two people are tracked when only one person is supposed to be tracked, and the accuracy of object tracking will be low.

[0062] FIG. 6 is a diagram showing a tracking time series of this embodiment. FIG. 6 shows the tracking result of two people output by the object tracker 5 whose parameters are optimized by the parameter optimization device 1 of this embodiment. In the tracking result shown in FIG. 6, the horizontal axis shows time, and the vertical axis shows the horizontal position in the image captured by the test camera, as in FIG. 5. In the tracking result of FIG. 6, after the person H1 and the person H2 cross each other at time t00, the time series of the person H2 cannot be tracked by the object tracker 5, but is formed by extrapolation (see the inside of the region E1). After that, when the object tracker 5 becomes able to track the person H2 at time t01, the tracking time series is again given the identification number 2 of the person H2. Therefore, it can be seen that the object tracker 5 whose parameters are optimized has high object tracking accuracy.

[0063] According to the parameter optimization device 1 of the present embodiment described above, the correct time series including information on the movement start time, movement end time, and identification number of a person is measured in parallel with the acquisition of the object time series including information on the position of the person at each time. This makes it possible to easily acquire the correct time series, and therefore makes it easy to compare the tracking time series including information on the tracking result of a person acquired by inputting the parameters and the object time series to the object tracker 5 with the correct time series. Therefore, the parameters can be easily optimized.

[0064] Furthermore, according to the parameter optimization device 1 of the present embodiment, the parameters of the object tracker 5 can be easily optimized, so that the effort required to improve the person tracking accuracy can be reduced. Since the effort required to improve the tracking accuracy can be reduced, the person tracking accuracy can be easily improved.

[0065] Moreover, according to the parameter optimization device 1 of this embodiment, the correct time series is measured based on a signal transmitted by the portable terminal 22 carried by the person. This allows the personal computer 30 to easily and reliably acquire the correct time series in parallel with the acquisition of the object time series, making it easier to compare the tracking time series with the correct time series. Therefore, the parameters can be further easily optimized.

[0066] Moreover, according to the parameter optimization device 1 of this embodiment, the positions where the two people H1 and H2 start moving and the positions where the two people H1 and H2 end moving are within a range that can be captured by the test camera 21. That is, the object time series includes information on the positions of the people at the movement start time and the movement end time. This allows the tracking time series output from the object tracker 5 by inputting the object time series to be reliably compared with the correct time series at each of the movement start and movement end, so that parameters can be reliably searched for.

[0067] In addition, according to the parameter optimization device 1 of this embodiment, the movement start time and the movement end time are set before and after the person stops for a predetermined time. This makes the movement start and end of the person clear in the object time series, so that the tracking time series output by inputting the object time series can be reliably compared with the correct answer time series. Therefore, the parameters can be reliably searched.

[0068] Moreover, according to the parameter optimization device 1 of this embodiment, a plurality of time series sets are created corresponding to a plurality of different movement patterns in which two people cross. The evaluation unit 15 evaluates the parameter using a plurality of evaluation index values ​​for the parameter obtained using each of the plurality of time series sets. This allows the parameter to be evaluated in consideration of each of a plurality of different movement patterns in which two people cross, thereby improving the degree of optimization of the parameter.

[0069] Furthermore, according to the object tracking system 100 of the present embodiment, the parameters output by the parameter optimization device 1 are set in the object tracker 5, thereby improving the person tracking accuracy of the object tracker 5. This makes it possible to prepare an object tracker 5 with a relatively high person tracking accuracy.

[0070] Furthermore, according to the object tracking method of this embodiment, the ground truth time series input in step S11 can be easily acquired because it is measured in parallel with the acquisition of the object time series including information on the person's position at each time. This makes it easy to compare the tracking time series with the ground truth time series in step S13, and therefore makes it easy to optimize the parameters in step S16.

[0071] Furthermore, according to the computer program of this embodiment, the correct time series acquired by the correct time series acquisition unit 12 can be easily acquired because it is measured in parallel with the acquisition of the object time series including information on the position of the person at each time. This makes it easy for the evaluation unit 15 to compare the tracking time series with the correct time series, so that the parameter search unit 16 can easily optimize the parameters.

[0072] <Second embodiment> Fig. 7 is a diagram showing a schematic configuration of a parameter optimization device according to the second embodiment. The parameter optimization device 2 according to the second embodiment is different from the parameter optimization device 1 according to the first embodiment (Fig. 2) in that it includes a trade-off generating unit.

[0073] The object tracking system of the second embodiment includes a parameter optimization device 2 and an object tracker 5. The parameter optimization device 2 is composed of a test camera 21, a mobile terminal 22, and a personal computer 40, and functions as an object time series acquisition unit 11, a correct answer time series acquisition unit 12, a tracking time series acquisition unit 13, a storage unit 14, a trade-off occurrence unit 28, an evaluation unit 15, a parameter search unit 16, and an output unit 17.

[0074] The trade-off generating unit 28 corresponds to a trade-off generating function of the CPU 33. The trade-off generating unit 28 processes the time series set stored in the storage unit 14 so that a trade-off occurs between an increase and a decrease in parameter value for at least two parameters among the multiple parameters set in the object tracker 5. The function of the trade-off generating unit 28 will be described in detail later.

[0075] 8 is a flowchart of the object tracking method of this embodiment. Next, the object tracking method by the object tracking system of this embodiment will be described. Here, the method for optimizing the parameters set in the object tracker 5 by the parameter optimization device 2 will be described, focusing on the differences from the object tracking method of the first embodiment. In the object tracking method of this embodiment, first, as a preparation step for parameter optimization, an object time series and a correct answer time series are input (step S11) in the same manner as in the object tracking method of the first embodiment.

[0076] After step S11, it is determined whether or not a trade-off occurs in the parameters (step S22). In step S22, the trade-off occurrence unit 28 determines whether or not a trade-off occurs in the increase and decrease of the parameter values ​​of the object tracker 5 due to a movement pattern included in each of the multiple object time series acquired by the object time series acquisition unit 11. If the trade-off occurrence unit 28 determines that a trade-off occurs (step S22: Yes), the process proceeds to step S12. If the trade-off occurrence unit 28 determines that a trade-off does not occur (step S22: No), the process proceeds to step S23.

[0077] When it is determined in step S22 that no trade-off occurs, the trade-off generating unit 28 processes the object time series linked with the correct answer time series (step S23). In step S23, the trade-off generating unit 28 generates a trade-off for the object time series of the movement pattern in which no trade-off occurs between the increase and decrease of the parameter value of the object tracker 5 among the movement patterns corresponding to the object time series for each of the multiple object time series acquired by the object time series acquisition unit 11.

[0078] Here, a concrete example will be used to explain the parameter trade-off by the trade-off generating unit 28. As examples of parameters to be traded off, a parameter α related to the association between the person being tracked by the object time series acquiring unit 11 and the person by the test camera 21 when the number of people is one, and a parameter β related to the end of tracking when each person is always moving within a detectable range in the object time series acquiring unit 11 will be taken up.

[0079] If the parameter α is made difficult to respond, there is a risk that the test camera 21 may start tracking the person being tracked as a new person who has appeared, and in this case, the person will be tracked as if it were split. On the other hand, if the parameter α is made too easy to respond, different people will be tracked as one person. For this reason, a trade-off needs to be made to set the parameter α appropriately. However, for the object time series acquired by the object time series acquisition unit 11, if the number of people is one, the evaluation result in the evaluation unit 15 is unlikely to deteriorate even if the parameter α is made too easy to respond. In such a case, in order to make a trade-off, if the number of people is one for the object time series acquired by the object time series acquisition unit 11, for example, the object time series linked with the correct answer time series is processed as if a person chasing the person before processing was measured. Specifically, the object time series delayed by a certain time is merged with the object time series, and the correct answer time series delayed by the same time and the identification number changed so as not to overlap with the original correct answer time series is merged with the correct answer time series. In addition to the time delay, an object time series whose features have been processed may be merged, for example, by translating the features related to the positions of the object time series. Note that the time delay is essential to maintain the control conditions in the correct time series acquisition unit 12.

[0080] With regard to the parameter β for the end of tracking, if the end of tracking is made earlier, the same person will be tracked as a different person as a new person appears after the tracking is once ended. On the other hand, if the end of tracking is made more difficult, the person will be considered to continue to exist, and the position, speed, etc. will continue to be extrapolated. For this reason, a trade-off must be made in order to set the parameter β appropriately. However, in the object time series acquisition unit 11, if each person is always moving within a detectable range, the evaluation result in the evaluation unit 15 is unlikely to deteriorate even if the end of tracking is made more difficult. In such a case, in order to make a trade-off, the middle of the object time series is processed as if the information of the person was interrupted and not input for a time period that sufficiently exceeds the temporary loss, for example, 2 seconds, and the object time series including the object time series before processing is added to the correct answer time series linked together as a section interrupted object time series distinguished from the normal object time series. Specifically, information about the person is deleted for the corresponding time. In order to make the effect of such deletion more noticeable, the time period of deletion may be overlapped with the time period in which the number of people appearing simultaneously in the correct answer time series is the largest. When the interval disruption object time series is included in the time series set by the trade-off generating unit 28, the interval disruption object time series is also input and started, and the output is stored as an interval disruption tracking time series in the time series set including the input.

[0081] In step S22, if the trade-off generating unit 28 determines that a trade-off occurs, and in step S23, if the object time series and the correct answer time series linked together are processed, then, similarly to the object tracking method of the first embodiment, the object tracker 5 is driven to obtain a tracking time series (step S12). After obtaining the tracking time series, the time series set is evaluated (step S13).

[0082] In step S13 of the object tracking method of this embodiment, the evaluation unit 15 also defines an evaluation index value for the time series set including the interval-disrupted tracking time series described above. An interval-disrupted object time series is a time series that is processed so that the middle of the object time series is interrupted and no human information is input for a period of time that sufficiently exceeds the temporary loss. Due to such an interruption, it is desirable that the identification number be changed at the point where object tracking is terminated once and input is resumed. However, if the parameter related to the end of tracking has a value that makes it difficult to terminate, the identification number is not changed, and the evaluation index value d when the interval-disrupted tracking time series is evaluated with the correct answer time series is d masked is the deviation d when evaluating the normal object time series with the correct answer time series. normal The value is almost the same as that of the number of objects affected by the disruption, n masked Considering this as the deviation, the deviation d normal It seems reasonable to add to

[0083] Therefore, assuming that there will be people whose identification numbers change due to the interruption and people whose identification numbers do not change, the evaluation index value d set Request.

number

[0084] The second term on the right side of equation (14) is the maximum number of objects n masked The identification number changes due to the interruption, and the evaluation index value d masked In addition, in the second term on the right side of equation (14), the evaluation index value d masked But the deviation d normal , and apply the maximum function max so that the second term does not become less than 0.

[0085] Next, as in the object tracking method of the first embodiment, it is determined whether or not the termination condition is satisfied (step S14). In step S14, if the evaluation unit 15 determines that the calculated evaluation index value does not satisfy the predetermined termination condition (step S14: No), the process proceeds to step S16, where new parameters are generated. In the object tracking method of this embodiment, if Optuna, a program capable of multi-objective optimization, is used in generating new parameters in step S16, Pareto-optimal parameter candidates can be obtained when a trade-off between multiple evaluation index values ​​is intentionally considered, eliminating the need for wasteful trial and error.

[0086] After step S16, the process returns to step S12, and the object tracker 5 is driven using the newly generated parameters and the object time series stored in the storage unit 14 to obtain a new tracking time series. In the parameter optimization method of this embodiment, similarly to the object tracking method of the first embodiment, if the termination condition in step S14 is not satisfied, the optimization process consisting of steps S12, S13, S14, and S16 is repeated to optimize the parameters.

[0087] According to the parameter optimization device 2 of the present embodiment described above, in addition to the effects of the parameter optimization device 1 of the first embodiment, the trade-off generating unit 28 generates a trade-off between an increase and a decrease in parameter value for a plurality of parameters set in the object tracker 5. This makes it possible to further easily optimize parameters.

[0088] <Modifications of this embodiment> The present invention is not limited to the above-described embodiment, and can be embodied in various forms without departing from the spirit and scope of the invention. For example, the following modifications are also possible.

[0089] [Variation 1] In the above embodiment, the object time series acquisition unit 11 captures an image using the test camera 21, and the object detection unit 33a creates an object time series using the captured image. However, the configuration of the object time series acquisition unit 11 is not limited to this. Instead of the test camera 21, an object tracker 5 in which parameters output by the parameter optimization device 1 are set may be used, and it does not have to be a single test camera, but may be one using multiple sensors. In addition, when optimizing parameters for an object tracker that can simultaneously perform object detection and tracking, the object detection unit may not be required.

[0090] [Variation 2] In the above embodiment, the correct time series acquisition unit 12 is a mobile terminal 22 owned by a person. However, the configuration of the correct time series acquisition unit 12 is not limited to this. When a specific signal associated with a time is issued, a correct time series may be obtained by associating an identification number with a movement start or movement end in control order each time.

[0091] FIG. 9 is a schematic diagram for explaining an outline of a modified example of the object tracking system 100 of the first embodiment. The parameter optimization device 1 included in the object tracking system 100 shown in FIG. 9 includes a test camera 21 and a personal computer 30. In the parameter optimization device 1 of FIG. 9, the correct time series acquisition unit 12 acquires a correct time series by passing the headlights L1 of the automobile Am1 included in the image captured by the test camera 21. Specifically, in the image A1 captured by the test camera 21, the automobile Am1 moves from the back side to the front side along the direction D3. In this case, when the automobile Am1 starts moving, the automobile Am1 can record the movement start time of the automobile Am1 in the image A1 by passing the headlights L1 (see automobile Am10 in FIG. 9). In addition, when the automobile Am1 ends moving, the automobile Am1 can record the movement end time of the automobile Am1 in the image A1 by passing the headlights L1 (see automobile Am11 in FIG. 9). In this way, the parameter optimization device 1 can acquire a ground truth time series measured in parallel with acquisition of an object time series.

[0092] In addition, when acquiring a correct time series, an order may be set for the timing of starting movement and the timing of ending movement for each of the two persons H1 and H2. For example, an order may be set such that the person H2 starts moving after the person H1 starts moving, the person H2 ends moving after the person H2 starts moving, and the person H1 ends moving after the person H2 ends moving.

[0093] [Variation 3] In the above embodiment, the tracking time series acquisition unit 13 acquires a tracking time series by inputting the object time series and parameters to the actual object tracker 5. When actual object tracking is performed by an electronic circuit or a computer for edge computing, the object tracker may be an actual device, or may be simulated on the same computer as the evaluation unit 15 and the parameter search unit 16 or on another computer. If the object tracker is realized as a computer program and the parameters are not precisely optimized, the computer program may be normally operated on the same computer as the evaluation unit 15 and the parameter search unit 16 or on another computer.

[0094] [Variation 4] In the above embodiment, the object time series includes information about the object's position at each of the movement start time and the movement end time. However, it is sufficient that the object time series includes either the information about the object's position at each of the movement start time and the movement end time, and it is not necessary that the object time series includes neither the information about the object's position at each of the movement start time and the movement end time.

[0095] [Variation 5] In the above embodiment, the movement start time and the movement end time are set before and after the object stops for a predetermined time. However, the timing for setting the movement start time and the movement end time is not limited to before and after the object stops for a predetermined time. The correct time series acquisition unit 12 may simply set the time when a person stops for a predetermined time and then starts moving as the movement start time, or may simply set the time when a moving person stops when the person stops and continues to stop for a predetermined time as the movement end time.

[0096] [Variation 6] In the above embodiment, a plurality of time series sets are created corresponding to a plurality of different movement patterns of an object, and the evaluation unit evaluates the parameter using a plurality of evaluation index values ​​for the parameter obtained using each of the plurality of time series sets. However, the parameter evaluation method is not limited to this.

[0097] [Variation 7] In the above embodiment, the object tracked by the object tracker 5 is a person, but the object tracked by the object tracker 5 is not limited to a person. It may be any moving object, such as a bicycle, a car, a train, or an airplane.

[0098] Although the present aspect has been described above based on the embodiment and modified examples, the above-mentioned embodiment of the aspect is intended to facilitate understanding of the present aspect and does not limit the present aspect. The present aspect may be modified or improved without departing from the spirit and scope of the claims, and equivalents are included in the present aspect. Furthermore, if a technical feature is not described as essential in this specification, it may be deleted as appropriate.

[0099] <Application example 1> A parameter optimization device for optimizing parameters set in an object tracker that tracks an object, comprising: an object time series acquisition unit that acquires an object time series including information about the position of the object at each time; a correct time series acquisition unit that acquires a correct time series for an object whose object time series is to be acquired, the correct time series including information on a movement start time, a movement end time, and an identification number, which are measured in parallel with the acquisition of the object time series; a tracking time series acquisition unit that acquires a tracking time series output from the object tracker by inputting the object time series to the object tracker, the tracking time series including information about a tracking result of the object calculated using the parameters and the object time series; a storage unit that stores a time series set in which the object time series, the correct answer time series, and the tracking time series are linked together; an evaluation unit that evaluates parameters set in the object tracker by using the time series set stored in the storage unit; a search unit that searches for new parameters to be set in the object tracker using the evaluation result by the evaluation unit; An output unit that outputs the parameters searched by the search unit. Parameter optimizer. <Application example 2> The parameter optimization device according to Application Example 1 further comprises: a trade-off generating unit that processes the time series set so that a trade-off occurs between an increase and a decrease in parameter value for at least two parameters among a plurality of parameters set in the object tracker; Parameter optimizer. <Application example 3> The parameter optimization device according to Application Example 1 or 2, The correct time series acquisition unit is a signal transmitter provided on the object, the signal transmitter transmitting a signal including information regarding the identification number at a time when the object starts moving and at a time when the object stops moving; A signal receiver capable of receiving a signal transmitted by the signal transmitter, Obtaining the correct time series using a signal received by the signal receiver. Parameter optimizer. <Application Example 4> The parameter optimization device according to any one of Application Examples 1 to 3, The object time series includes information regarding a position of the object at the movement start time and information regarding a position of the object at the movement end time. Parameter optimizer. <Application example 5> The parameter optimization device according to any one of Application Examples 1 to 4, The correct time series acquisition unit is The time when the object starts moving after stopping for a predetermined time is defined as the movement start time, When the moving object stops and continues to be stopped for a predetermined period of time, the time when the object stops is set as the movement end time and the correct time series is acquired. Parameter optimizer. <Application Example 6> The parameter optimization device according to any one of Application Examples 1 to 5, the object time series acquisition unit acquires a plurality of object time series corresponding to a plurality of different movement patterns of the object, the correct time series acquisition unit acquires the correct time series corresponding to each of the plurality of object time series; the tracking time series acquisition unit acquires the tracking time series corresponding to each of the plurality of object time series; the storage unit stores a time series set in which one object time series among the plurality of object time series, a correct answer time series corresponding to the one object time series, and a tracking time series corresponding to the one object time series are linked together; The evaluation unit is determining a plurality of evaluation index values ​​for the parameters using each of the plurality of time series sets; Evaluating parameters set in the object tracker using a plurality of evaluation index values; Parameter optimizer. <Application Example 7> 1. An object tracking system, comprising: A parameter optimization device according to any one of Application Examples 1 to 6, The object tracker to which the parameters output by the parameter optimization device are input. Object tracking system. <Application Example 8> A parameter optimization method for optimizing parameters set in an object tracker that tracks an object, comprising the steps of: a first step of acquiring an object time series including information about the object's location over time; a second step of acquiring a ground truth time series for the object for which the object time series is acquired, the ground truth time series including information on a movement start time, a movement end time, and an identification number, measured in parallel with the acquisition of the object time series; a third step of acquiring a tracking time series output from the object tracker by inputting the object time series to the object tracker, the tracking time series including information about a tracking result of the object calculated using the parameters and the object time series; a storage step of storing a time series set in which the object time series, the ground truth time series, and the tracking time series are linked together; an evaluation step of evaluating parameters set in the object tracker using the set of time series; a searching step of searching for new parameters to be set in the object tracker using the evaluation result in the evaluation step; and an output step of outputting the parameters searched for in the search step. Parameter optimization methods. <Application Example 9> A computer program for causing a computer to optimize parameters set in an object tracker that tracks an object, comprising: A first function for obtaining an object time series including information regarding the object's location over time; A second function of acquiring a ground truth time series for an object for which the object time series is acquired, the ground truth time series including information on a movement start time, a movement end time, and an identification number, the information being measured in parallel with the acquisition of the object time series; a third function of acquiring a tracking time series output from the object tracker by inputting the object time series to the object tracker, the tracking time series including information about a tracking result of the object calculated using the parameters and the object time series; a storage function for storing a time series set in which the object time series, the correct answer time series, and the tracking time series are linked together; an evaluation function that evaluates parameters set in the object tracker using the time series set; A search function that searches for new parameters to be set in the object tracker using the evaluation result by the evaluation function; and an output function for outputting the parameters searched for by the search function. Computer program. [Explanation of symbols]

[0100] 1,2...Parameter optimization device H1,H2…person (object) 5. Object tracker 11...Object time series acquisition section 12... Correct time series acquisition section 13…Tracking time series acquisition section 14...Storage section 15…Evaluation section 16…Parameter search section 17...Output section 28…Trade-off occurrence area 21…Test camera 22. Mobile terminal 31…Input / output terminal 32…Storage medium 33a...Object detection unit 100…Object tracking system

Claims

1. A parameter optimization device for optimizing parameters set in an object tracker that tracks an object, comprising: an object time series acquisition unit that acquires an object time series including information about the position of the object at each time; a correct time series acquisition unit that acquires a correct time series for an object whose object time series is to be acquired, the correct time series including information on a movement start time, a movement end time, and an identification number, which are measured in parallel with the acquisition of the object time series; a tracking time series acquisition unit that acquires a tracking time series output from the object tracker by inputting the object time series to the object tracker, the tracking time series including information about a tracking result of the object calculated using the parameters and the object time series; a storage unit that stores a time series set in which the object time series, the correct answer time series, and the tracking time series are linked together; an evaluation unit that evaluates parameters set in the object tracker by using the time series set stored in the storage unit; a search unit that searches for new parameters to be set in the object tracker using the evaluation result by the evaluation unit; An output unit that outputs the parameters searched by the search unit. Parameter optimizer.

2. The parameter optimization device according to claim 1 further comprises: a trade-off generating unit that processes the time series set so that a trade-off occurs between an increase and a decrease in parameter value for at least two parameters among a plurality of parameters set in the object tracker; Parameter optimizer.

3. The parameter optimization device according to claim 1 or 2, The correct time series acquisition unit is a signal transmitter provided on the object, the signal transmitter transmitting a signal including information regarding the identification number at a time when the object starts moving and a time when the object stops moving; A signal receiver capable of receiving a signal transmitted by the signal transmitter, Obtaining the correct time series using a signal received by the signal receiver. Parameter optimizer.

4. The parameter optimization device according to claim 1 or 2, The object time series includes information regarding a position of the object at the movement start time and information regarding a position of the object at the movement end time. Parameter optimizer.

5. The parameter optimization device according to claim 1 or 2, The correct time series acquisition unit is The time when the object starts moving after stopping for a predetermined time is defined as the movement start time, When the moving object stops and continues to be stopped for a predetermined period of time, the time when the object stops is set as the movement end time and the correct time series is acquired. Parameter optimizer.

6. The parameter optimization device according to claim 1 or 2, the object time series acquisition unit acquires a plurality of object time series corresponding to a plurality of different movement patterns of the object, the correct time series acquisition unit acquires the correct time series corresponding to each of the plurality of object time series; the tracking time series acquisition unit acquires the tracking time series corresponding to each of the plurality of object time series; the storage unit stores a time series set in which one object time series among the plurality of object time series, a correct answer time series corresponding to the one object time series, and a tracking time series corresponding to the one object time series are linked together; The evaluation unit is determining a plurality of evaluation index values ​​for the parameters using each of the plurality of time series sets; Evaluating parameters set in the object tracker using a plurality of evaluation index values; Parameter optimizer.

7. 1. An object tracking system, comprising: A parameter optimization device according to claim 1 or 2; The object tracker to which the parameters output by the parameter optimization device are input. Object tracking system.

8. A parameter optimization method for optimizing parameters set in an object tracker that tracks an object, comprising the steps of: a first step of obtaining an object time series comprising information about the object's location over time; a second step of acquiring a ground truth time series for the object for which the object time series is acquired, the ground truth time series including information on a movement start time, a movement end time, and an identification number, measured in parallel with the acquisition of the object time series; a third step of acquiring a tracking time series output from the object tracker by inputting the object time series to the object tracker, the tracking time series including information about a tracking result of the object calculated using the parameters and the object time series; a storage step of storing a time series set in which the object time series, the ground truth time series, and the tracking time series are linked together; an evaluation step of evaluating parameters set in the object tracker using the set of time series; a searching step of searching for new parameters to be set in the object tracker using the evaluation result in the evaluation step; and an output step of outputting the parameters searched for in the search step. Parameter optimization methods.

9. A computer program for causing a computer to execute optimization of parameters set in an object tracker that tracks an object, comprising: a first function of obtaining an object time series including information regarding the object's position at each time; a second function of acquiring a ground truth time series for an object for which the object time series is acquired, the ground truth time series including information on a movement start time, a movement end time, and an identification number, the information being measured in parallel with the acquisition of the object time series; a third function of acquiring a tracking time series output from the object tracker by inputting the object time series to the object tracker, the tracking time series including information about a tracking result of the object calculated using the parameters and the object time series; a storage function for storing a time series set in which the object time series, the correct answer time series, and the tracking time series are linked together; an evaluation function that evaluates parameters set in the object tracker using the time series set; A search function that searches for new parameters to be set in the object tracker using the evaluation result by the evaluation function; and an output function for outputting the parameters searched for by the search function. Computer program.

Images