Tracking method, tracking program, and information processing device

Abstract

This information processing device detects regions of objects from a plurality of time-series frames. The information processing device determines whether a first region of an object detected from a first frame included in the plurality of frames and a second region of an object detected from a second frame, which is the next frame after the first frame, are regions of the same object, on the basis of a first buffer region obtained by adding a buffer region having a first scale to the first region, and a second buffer region obtained by adding a buffer region having the first scale to the second region. If it is determined that said regions are not regions of the same object, the information processing device once again determines whether the first region and the second region are regions of the same object, on the basis of a third buffer region obtained by adding a buffer region having a second scale larger than the first scale to the first region, and a fourth buffer region obtained by adding a buffer region having the second scale to the second region.

Description

[Technical Field]

[0001] The present invention relates to a tracking method and the like. [Background technology]

[0002] Tracking technologies for tracking multiple objects in a video include MOT (Multi-object tracking), SORT (Simple Online and Realtime Tracking), etc. For example, conventional tracking technologies generate track data by setting a bbox (Bounding Box) for the area of an object detected from each frame of a time series included in the video, and assigning the same ID to the bbox of the same object.

[0003] In conventional tracking technology, IoU (Intersection over Unit) is used to determine whether bboxes in previous and subsequent frames are the same object. Fig. 13 is a diagram for explaining IoU. The bbox detected from the frame with frame number n is designated bbox10A. The bbox detected from the frame with frame number n+1 is designated bbox10B.

[0004] 13, IoU is a value obtained by dividing overlapping region 11 by combined region 12. Overlapping region 11 is the region where bbox 10A and bbox 10B overlap. Combined region 12 is the region where bbox 10A and bbox 10B are combined.

[0005] For example, if the IoU in FIG. 13 is equal to or greater than a threshold, bbox 10A and bbox 10B are bboxes corresponding to the same object and are assigned the same track ID. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-45501 Summary of the Invention [Problem to be solved by the invention]

[0007] However, with the above-mentioned conventional technology, when objects are densely packed, when the objects move discontinuously, or when the objects move quickly, it is not possible to properly identify the bbox of the same object, resulting in reduced tracking accuracy.

[0008] FIG. 14 is a diagram illustrating an example of a problem with conventional tracking technology. As shown in FIG. 14, bbox_f1 of person 3a is detected from frame f1. Furthermore, bbox_f2 of person 3b is detected from frame f2, which is the frame following frame f1. Here, even if person 3a and person 3b are the same person, if person 3a (3b) moves quickly, the overlapping area between bbox_f1 and bbox_f2 will be "0." This causes the IoU value to fall below the threshold, and different track IDs are assigned to bbox_f1 and bbox_f2. In the example shown in FIG. 14, track ID "1" is assigned to bbox_f1, and track ID "2" is assigned to bbox_f2. If person 3a and person 3b are the same person, it can be said that tracking accuracy is reduced.

[0009] In one aspect, the present invention aims to provide a tracking method, a tracking program, and an information processing device that can improve tracking accuracy. [Means for solving the problem]

[0010] In the first proposal, a computer is caused to perform the following process. The computer detects an object region from multiple frames in a time series. The computer determines whether the first region and the second region are regions of the same object based on a first buffer region obtained by adding a buffer region of a first scale to a first region of the object detected from a first frame included in the multiple frames, and a second buffer region obtained by adding the buffer region of the first scale to a second region of the object detected from a second frame subsequent to the first frame. If the computer determines that the first region and the second region are not regions of the same object, the computer again determines whether the first region and the second region are regions of the same object based on a third buffer region obtained by adding a buffer region of a second scale larger than the first scale to the first region, and a fourth buffer region obtained by adding the buffer region of the second scale to the second region. [Effects of the Invention]

[0011] The tracking accuracy can be improved. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram for explaining BIoU. [Figure 2] FIG. 2 is a diagram for explaining the buffer scale. [Figure 3] FIG. 3 is a diagram illustrating tracking using BIoU. [Figure 4] FIG. 4 is a diagram illustrating an example of track data executed by the information processing device. [Figure 5] FIG. 5 is a diagram illustrating the tracking process executed by the information processing device. [Figure 6] FIG. 6 is a functional block diagram showing the configuration of an information processing device according to this embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of the data structure of the track table. [Figure 8] FIG. 8 is a diagram illustrating an example of detection data. [Figure 9] FIG. 9 is a flowchart illustrating a processing procedure of the information processing device according to the present embodiment. [Figure 10] FIG. 10 is a diagram for explaining the effect of the information processing device of this embodiment. [Figure 11] FIG. 11 is a diagram for explaining the process of identifying the optimum combination of matching scales. [Figure 12] FIG. 12 is a diagram illustrating an example of a hardware configuration of a computer that realizes the same functions as the information processing device. [Figure 13] FIG. 13 is a diagram for explaining IoU. [Figure 14] FIG. 14 is a diagram for explaining an example of a problem with the conventional tracking technology. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, a tracking method, a tracking program, and an information processing device disclosed in the present application will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to these embodiments. [Example]

[0014] First, BIoU (Buffered Intersection over Unit) used by the information processing device of this embodiment will be described. Fig. 1 is a diagram for explaining BIoU. A bbox detected from a frame with frame number n is defined as bbox20A. A buffer_bbox based on bbox20A is defined as buffer_bbox21A. A bbox detected from a frame with frame number n+1 is defined as bbox20B. A buffer_bbox based on bbox20B is defined as buffer_bbox21B.

[0015] 1, BIoU is a value obtained by dividing overlapping area 22 by combined area 23. Overlapping area 22 is an area where buffer_bbox21A and buffer_bbox21B overlap. Combined area 23 is an area where buffer_bbox21A and buffer_bbox21B are combined.

[0016] Here, the relationship between the original bbox and buffer_bbox is defined in advance by the buffer scale. FIG. 2 is a diagram for explaining the buffer scale. As shown in FIG. 2, the center of the original bbox 25 and the center of the buffer_bbox 26 are assumed to be the same. The height and width of the original bbox 25 are assumed to be "h" and "w", respectively. The height and width of the buffer_bbox 26 are assumed to be "bh" and "bw". Then, the buffer scale is defined by equation (1).

[0017] Buffer_scale=(bw-w) / w=(bh-h) / h···(1)

[0018] FIG. 3 is a diagram illustrating tracking using BIoU. Here, the description will be given using frame f1 with frame number n and frame f2 with frame number n+1. The information processing device detects bbox_f1 of person 3a from frame f1. The information processing device detects bbox_f2 of person 3b from frame f2.

[0019] The information processing device sets buffer_bboxf1 based on bbox_f1 of person 3a, based on the buffer scale. The information processing device sets buffer_bboxf2 based on bbox_f2 of person 3b, based on the buffer scale. The information processing device calculates the BIoU value based on buffer_bboxf1 and buffer_bboxf2, and if the BIoU value is equal to or greater than a threshold, assigns the same track ID to bbox_f1 and bbox_f2.

[0020] In the example shown in FIG. 3, the BIoU value is equal to or greater than the threshold, so the information processing device assigns the same track ID "1" to bbox_f1 and bbox_f2.

[0021] 14, even if person 3a and person 3b are the same person, if person 3a (3b) moves quickly, the IoU value will be below the threshold, and different track IDs will be assigned to bbox_f1 and bbox_f2. In contrast, by using BIoU, it is possible to assign the same track ID to bbox_f1 and bbox_f2 even if person 3a (3b) moves quickly.

[0022] Next, an example of track data generated by the information processing device according to this embodiment will be described. Fig. 4 is a diagram for explaining an example of track data executed by the information processing device. Here, for convenience, the description will be given using frame f1 with frame number n, frame f2 with frame number n+1, and frame f3 with frame number n+2.

[0023] The information processing device detects an object from frame f1 and sets bboxes 30a, 30b, and 30c, detects an object from frame f2 and sets bboxes 31a, 31b, and 31c, and detects an object from frame f3 and sets bboxes 32a, 32b, and 32c.

[0024] Assume that the information processing device determines, through tracking processing shown in Fig. 5 (to be described later), that bbox30a and bbox31a are regions of the same object, and that bbox31a and bbox32a are regions of the same object. In this case, the information processing device assigns the same track ID "Tr1" to bbox30a, 31a, and 32a. As a result, track data with the track ID "Tr1" is generated.

[0025] Assume that the information processing device determines, through tracking processing shown in Fig. 5 (to be described later), that bbox30b and bbox31b are regions of the same object, and that bbox31b and bbox32b are regions of the same object. In this case, the information processing device assigns the same track ID "Tr2" to bbox30b, 31b, and 32b. As a result, track data with the track ID "Tr2" is generated.

[0026] Assume that the information processing device determines, through tracking processing shown in Fig. 5 (described later), that bbox30c and bbox31c are regions of the same object, and that bbox31c and bbox32c are regions of the same object. In this case, the information processing device assigns the same track ID "Tr3" to bbox30c, 31c, and 32c. As a result, track data with the track ID "Tr3" is generated.

[0027] Next, an example of tracking processing executed by the information processing device according to this embodiment for generating the track data described in Fig. 4 will be described. Fig. 5 is a diagram for explaining the tracking processing executed by the information processing device. The information processing device sets buffer_bbox for the bbox of an object detected from multiple frames in a time series, and executes first BIoU matching and second BIoU matching in that order to generate track data.

[0028] For example, in the first BIoU matching, a buffer_bbox with a matching scale of "0.3" is set and the BIoU is calculated. In the second BIoU matching, a buffer_bbox with a matching scale of "0.6" is set and the BIoU is calculated. The size of the buffer_bbox used in the first BIoU matching is smaller than the size of the buffer_bbox used in the second BIoU matching.

[0029] First, the first BIoU matching performed by the information processing device will be described. The information processing device performs the first BIoU matching based on detection data 40 and multiple pieces of alive track data 41. For example, the detection data 40 includes bbox data of multiple objects detected from a frame with frame number n. The alive track data 41 is track data up to frame number n-1. The alive track data 41 includes bbox data of objects detected from each frame, and a unique track ID is set. In addition, a track age is set in the alive track data 41.

[0030] The information processing device selects one bbox included in the detection data 40 and one piece of alive track data 41. In the following description, one bbox included in the detection data 40 selected by the information processing device will be referred to as the "first bbox." The bbox of frame number n-1 of the piece of alive track data 41 selected by the information processing device will be referred to as the "second bbox."

[0031] The information processing device sets a buffer_bbox (hereinafter referred to as the first buffer_bbox) with a matching scale of "0.3" for the first bbox. The information processing device sets a buffer_bbox (hereinafter referred to as the second buffer_bbox) with a matching scale of "0.3" for the second bbox. The information processing device calculates the BIoU value based on the first buffer_bbox and the second buffer_bbox.

[0032] If the BIoU value is equal to or greater than a threshold, the information processing device determines that the object in the first bbox and the object in the second bbox are the same object, and generates match track data 42. The match track data 42 is track data in which the data in the first bbox is added to the alive track data 41 selected by the information processing device.

[0033] The information processing device repeatedly executes the above process while changing the first bbox selected from the detection data 40 and the selected alive track data 41 (second bbox). The information processing device generates, as unmatched detection data 44, a bbox that does not match a bbox of the alive track data 41, among the multiple bboxes included in the detection data 40. The information processing device generates, as unmatched track data 43, alive track data 41 that does not match any bbox included in the detection data 40, among the multiple alive track data 41.

[0034] Next, the second BIoU matching performed by the information processing device will be described. The information processing device performs the second BIoU matching based on the unmatch detection data 44 and the plurality of unmatch track data 43.

[0035] The information processing device selects one bbox included in the unmatch detection data 44 and one unmatch track data 43. In the following description, the one bbox included in the unmatch detection data 44 selected by the information processing device will be referred to as the "third bbox." The bbox of frame number n-1 of the one unmatch track data selected by the information processing device will be referred to as the "fourth bbox."

[0036] The information processing device sets a buffer_bbox (hereinafter referred to as the third buffer_bbox) with a matching scale of "0.6" for the third bbox. The information processing device sets a buffer_bbox (hereinafter referred to as the fourth buffer_bbox) with a matching scale of "0.6" for the fourth bbox. The information processing device calculates the BIoU value based on the third buffer_bbox and the fourth buffer_bbox.

[0037] If the BIoU value is equal to or greater than a threshold, the information processing device determines that the object in the third bbox and the object in the fourth bbox are the same object, and generates match track data 45. The match track data 45 is track data in which the data in the third bbox is added to the unmatch track data 43 selected by the information processing device.

[0038] The information processing device repeatedly executes the above process while changing the third bbox selected from the unmatch detection data 44 and the unmatch track data 43 (fourth bbox) to be selected. The information processing device generates, as unmatch detection data 46, a bbox that does not match a bbox in the unmatch track data 43, among the multiple bboxes included in the unmatch detection data 44. The information processing device generates new track data 48 by assigning a unique track ID to each bbox included in the unmatch detection data 46. The information processing device sets an initial value of the track age in the new track data 48.

[0039] The information processing device generates, as unmatched track data 47, unmatched track data 43 that does not match any of the bboxes included in the unmatched detection data 44, from among the plurality of unmatched track data 43.

[0040] If the track age set in the unmatched track data 47 is greater than "max_age", the information processing device deletes the unmatched track data 47. If the track age set in the unmatched track data 47 is equal to or less than "max_age", the information processing device leaves the unmatched track data 47.

[0041] Meanwhile, the information processing device performs an update process on the unmatched track data 47. In the update process, the information processing device adds 1 to the track age of the unmatched track data 47. The information processing device predicts the movement of the target object based on the matched track data 42. After the update process, the information processing device outputs the matched track data 42 as alive track data 41 and performs the first BIoU matching described above. The information processing device also performs the update process on the matched track data 45, the new track data 48, and the remaining unmatched track data 47.

[0042] The information processing device generates track data by repeatedly executing the above process every time detection data 40 detected from the subsequent frame is acquired.

[0043] As described above, according to the information processing device of this embodiment, by first performing a narrow-range first BIoU matching, it becomes possible to match bboxes of the same object that moves slowly. The information processing device can match bboxes of the same object that moves quickly that were not matched in the first BIoU matching by performing a wide-range second BIoU matching. That is, it is possible to assign the same track ID to the same bbox detected from each frame, thereby improving tracking accuracy.

[0044] Next, a description will be given of an example configuration of an information processing device that executes the processing described in Fig. 5. Fig. 6 is a functional block diagram showing the configuration of an information processing device according to this embodiment. As shown in Fig. 6, this information processing device 100 has a communication unit 110, an input unit 120, a display unit 130, a storage unit 140, and a control unit 150.

[0045] The communication unit 110 is connected to the camera 90 via a network or the like. The communication unit 110 receives video data captured by the camera 90 via the network. The video data includes frames in time series. A frame is a still image captured by the camera 90 at a certain timing. A frame number is assigned to each frame in ascending order.

[0046] The input unit 52 corresponds to an input device that inputs various types of information to the information processing device 100 .

[0047] The display unit 53 displays information output from the control unit 150.

[0048] The storage unit 140 includes a video buffer 141 and a track table 142. The storage unit 54 corresponds to a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as an HDD (Hard Disk Drive).

[0049] The video buffer 141 is a buffer that stores video data. The video data includes frames in time series, and each frame is assigned a frame number.

[0050] The track table 142 is a table that stores multiple track data. FIG. 7 is a diagram showing an example of the data structure of the track table. As shown in FIG. 7, the track table 142 associates track IDs, frame numbers, and bbox data. The track ID is information that uniquely identifies track data. The frame number is a number assigned to a frame. The bbox data is bbox data of an object detected from a frame. For example, the bbox data is the data of a bbox on a frame.

[0051] The frame numbers and bbox data belonging to the same track ID correspond to one piece of track data. The track data corresponds to the alive track data 41 and the like described with reference to FIG.

[0052] Returning to the description of Fig. 6, the control unit 150 includes an acquisition unit 151, a detection unit 152, a first matching unit 153, a second matching unit 154, and an update processing unit 155. The control unit 155 corresponds to a CPU (Central Processing Unit) or the like.

[0053] Acquisition unit 151 acquires video data from camera 90 via communication unit 110. Acquisition unit 151 registers the video data in video buffer 141.

[0054] The detection unit 152 obtains a frame from the video buffer 141 and detects an object region included in the frame. For example, when detecting an object region, the detection unit 152 may use a trained learning model using a conventional technology similar to YOLO (You Only Look Once). Such a learning model is a learning model that receives a frame as input and outputs the object region.

[0055] The detection unit 152 generates detection data 40 by setting a bbox in the area of each object detected from the frame. FIG. 8 is a diagram showing an example of detection data. As shown in FIG. 8, a plurality of objects (balls in FIG. 8) are detected in the detection data 40, and a bbox is set for each object. The detection unit 152 outputs the detection data 40 to the first matching unit 153.

[0056] The detection unit 152 retrieves the frames stored in the video buffer 141 in the order of frame numbers, and repeatedly executes the above process.

[0057] The first matching unit 153 performs first BIoU matching based on the detection data 40 and the alive track data 41 acquired from the track table 142. The first BIoU matching performed by the first matching unit 153 is similar to the processing described in FIG.

[0058] For example, the first matching unit 153 sets a first buffer_bbox with a matching scale of "0.3" for the first bbox. The first matching unit 153 sets a second buffer_bbox with a matching scale of "0.3" for the second bbox. The first matching unit 153 calculates the BIoU value based on the first buffer_bbox and the second buffer_bbox.

[0059] If the BIoU value is equal to or greater than a threshold, the first matching unit 153 determines that the object in the first bbox and the object in the second bbox are the same object, and generates match track data 42. The match track data 42 is track data in which the data in the first bbox is added to the alive track data 41 selected by the first matching unit 153. The first matching unit 153 outputs the match track data 42 to the update processing unit 155.

[0060] The first matching unit 153 repeatedly executes the above process while changing the first bbox selected from the detection data 40 and the selected alive track data 41 (second bbox). The first matching unit 153 generates, as unmatched detection data 44, a bbox that does not match a bbox of the alive track data 41, among the multiple bboxes included in the detection data 40. The first matching unit 153 generates, as unmatched track data 43, alive track data 41 that does not match any bbox included in the detection data 40, among the multiple alive track data 41.

[0061] The first matching unit 153 outputs the unmatched track data 43 and the unmatched detection data 44 to the second matching unit 154.

[0062] The second matching unit 154 performs second BIoU matching based on the unmatched track data 43 and the unmatched detection data 44. The second BIoU matching performed by the second matching unit 154 is similar to the processing described with reference to FIG.

[0063] For example, the second matching unit 154 sets a third buffer_bbox with a matching scale of "0.6" for the third bbox. The second matching unit 154 sets a fourth buffer_bbox with a matching scale of "0.6" for the fourth bbox. The second matching unit 154 calculates the BIoU value based on the third buffer_bbox and the fourth buffer_bbox.

[0064] If the BIoU value is equal to or greater than a threshold, the second matching unit 154 determines that the object in the third bbox and the object in the fourth bbox are the same object, and generates match track data 45. The match track data 45 is track data in which the data in the third bbox is added to the unmatched track data 43 selected by the second matching unit 154. The second matching unit 154 outputs the match track data 45 to the update processing unit 155.

[0065] The second matching unit 154 repeatedly executes the above process while changing the third bbox selected from the unmatch detection data 44 and the unmatch track data 43 (fourth bbox) to be selected. The second matching unit 154 generates, as unmatch detection data 46, bboxes that do not match with bboxes in the unmatch track data 43, among the multiple bboxes included in the unmatch detection data 44. The second matching unit 154 generates new track data 48 by assigning a unique track ID to each bbox included in the unmatch detection data 46. The second matching unit 154 outputs the new track data 48 to the update processing unit 155.

[0066] The second matching unit 154 generates, as unmatched track data 47, the unmatched track data 43 that does not match any of the bboxes included in the unmatched detection data 44, from among the plurality of unmatched track data 43.

[0067] If the track age set in the unmatched track data 47 is greater than "max_age", the second matching unit 154 deletes the unmatched track data 47. If the track age set in the unmatched track data 47 is equal to or less than "max_age", the second matching unit 154 leaves the unmatched track data 47. The second matching unit 154 outputs the left unmatched track data 47 to the update processing unit 155.

[0068] The update processing unit 155 acquires each track data from the first matching unit 153 and the second matching unit 154, and performs update processing on the track data. The update processing performed by the update processing unit 155 is the same as the update processing described in Fig. 5. The update processing unit 155 updates the track table 142 with the track data for which the update processing has been performed.

[0069] Next, an example of a processing procedure of the information processing device according to this embodiment will be described. Fig. 9 is a flowchart showing the processing procedure of the information processing device according to this embodiment. As shown in Fig. 9, the acquisition unit 151 of the information processing device 100 acquires video data from a camera and stores it in the video buffer 141 (step S101).

[0070] The detection unit 152 of the information processing device 100 acquires a frame from the video buffer 141 (step S102). The detection unit 152 detects an object area from the frame and sets a bbox (step S103).

[0071] The first matching unit 153 of the information processing device 100 performs first BIoU matching (step S104). The second matching unit 154 of the information processing device 100 performs second BIoU matching (step S105).

[0072] The update processing unit 155 of the information processing device 100 executes update processing for each track data (step S106). The information processing device 100 updates the track table 142 (step S107).

[0073] If the information processing device 100 continues the process (step S108, Yes), the process proceeds to step S102. If the information processing device 100 does not continue the process (step S108, No), the process ends.

[0074] Next, a description will be given of the effects of the information processing device 100 according to this embodiment. The information processing device 100 sets a bbox for the area of an object detected from a frame of video data, and first performs a first BIoU matching with a narrower range, and then performs a second BIoU matching with a wider range, thereby identifying a bbox corresponding to the same object and generating track data.

[0075] First, by first performing a narrow-range first BIoU matching, it becomes possible to match bboxes of the same object that moves slowly. Furthermore, for bboxes of the same object that moves quickly that were not matched in the first BIoU matching, it becomes possible to match them by performing a wider-range second BIoU matching. In other words, the same track ID can be assigned to the same bbox detected from each frame, improving tracking accuracy.

[0076] 10 is a diagram for explaining the effect of the information processing device of this embodiment. The explanation will be made focusing on bbox60 set in frame f10 and bbox61 set in frame f11, the frame following frame f10. The object in bbox60 and the object in bbox61 are considered to be the same object. In the conventional technology that performs matching using normal IoU, a track ID different from the track ID of bbox60 is set in bbox61, resulting in a decrease in tracking accuracy.

[0077] On the other hand, in the information processing device 100 of the present invention, the same track ID as that of the bbox 60 is set in the bbox 61, thereby improving tracking accuracy.

[0078] Incidentally, the information processing device 100 according to the present embodiment sets the matching scale when performing the first BIoU matching to "0.3" and the matching scale when performing the second BIoU matching to "0.6," but this is not limited to this. For example, the information processing device 100 may identify an optimal matching scale combination based on training data. In the following description, the matching scale when performing the first BIoU matching will be referred to as the "first scale." The matching scale when performing the second BIoU matching will be referred to as the "second scale."

[0079] Fig. 11 is a diagram illustrating a process for identifying an optimal matching scale combination. The information processing device 100 acquires sample data 70 in which multiple pairs of first and second scales are set. In the explanation of Fig. 11, one pair showing the relationship between the first and second scales is referred to as a sample. For example, sample Sa1 is a sample showing a pair of the first scale "bs1" and the second scale "bs2."

[0080] On the other hand, video data and correct track data generated from the video data are stored in the training data 80. In the correct track data, bboxes of the same object are set, and the bboxes are arranged in the order of frame numbers.

[0081] The information processing device 100 sets a first scale corresponding to the selected sample Sa1 as the matching scale when performing first BIoU matching. A BIoU threshold is associated with the sample Sa1. The information processing device 100 sets a second scale corresponding to the selected sample Sa1 as the matching scale when performing second BIoU matching. The information processing device 100 generates track data from the video data included in the training data 80 using the set matching scale and BIoU threshold, and evaluates the generated track data. For example, the information processing device 100 calculates the match rate between multiple bboxes included in the generated track data and bboxes included in the correct track data.

[0082] The information processing device 100 repeatedly performs the above process for each sample included in the sample data 70, and identifies the sample with the highest match rate. The information processing device 100 sets the first scale corresponding to the identified sample as the matching scale when performing actual first BIoU matching. The information processing device 100 sets the second scale corresponding to the identified sample as the matching scale when performing actual second BIoU matching, and simultaneously determines the BIoU threshold for use in matching determination. Note that the information processing device 100 may previously set a BIoU threshold corresponding to the size of the matching scale, and after setting the matching scale, select the BIoU threshold corresponding to the size of the matching scale.

[0083] As described above, the information processing device 100 can further improve tracking accuracy by specifying the matching scale and BIoU threshold when performing the first BIoU matching and the second BIoU matching based on the sample with the highest matching rate.

[0084] Next, a description will be given of an example of the hardware configuration of a computer that realizes the same functions as the information processing device 100 described in the above embodiment. Fig. 12 is a diagram showing an example of the hardware configuration of a computer that realizes the same functions as the information processing device.

[0085] 12, computer 200 has CPU 201 that executes various types of arithmetic processing, input device 202 that accepts data input from a user, and display 203. Computer 200 also has communication device 204 that receives video data from camera 90, and interface device 205 that connects to various devices. Computer 200 also has RAM 206 that temporarily stores various types of information, and hard disk drive 207. Each of devices 201 to 207 is connected to bus 208.

[0086] The hard disk drive 207 stores an acquisition program 207a, a detection program 207b, a first matching program 207c, a second matching program 207d, and an update processing program 207e. The CPU 201 reads out the acquisition program 207a, the detection program 207b, the first matching program 207c, the second matching program 207d, and the update processing program 207e and loads them into the RAM 206.

[0087] The acquisition program 207a functions as an acquisition process 206a. The detection program 207b functions as a detection process 206b. The first matching program 207c functions as a first matching process 206c. The second matching program 207d functions as a second matching process 206d. The update processing program 207e functions as an update processing process 206e.

[0088] The processing of the acquisition process 206a corresponds to the processing of the acquisition unit 151. The processing of the detection process 206b corresponds to the processing of the detection unit 152. The processing of the first matching process 206c corresponds to the processing of the first matching unit 153. The processing of the second matching process 206d corresponds to the processing of the second matching unit 154. The processing of the update processing process 206e corresponds to the processing of the update processing unit 155.

[0089] It should be noted that each of the programs 207a to 207e does not necessarily have to be stored in the hard disk drive 207 from the beginning. For example, each of the programs may be stored in a "portable physical medium" such as a flexible disk (FD), CD-ROM, DVD disk, magneto-optical disk, or IC card that is inserted into the computer 200. Then, the computer 200 may read and execute each of the programs 207a to 207e. [Explanation of symbols]

[0090] 100 Information processing device 110 Communications Department 120 Input section 130 Display section 140 Storage section 141 Video Buffer 142 Track Table 150 control section 151 Acquisition Department 152 Detection unit 153 First Matching Section 154 Second Matching Section 155 Update processing section

Claims

1. Detect the object area from multiple frames in a time series, determining whether the first area and the second area are areas of the same object based on a first buffer area obtained by adding a buffer area of a first scale to a first area of the object detected from a first frame included in the plurality of frames, and a second buffer area obtained by adding a buffer area of the first scale to a second area of the object detected from a second frame subsequent to the first frame; if it is determined that the first area and the second area are not areas of the same object, it is determined again whether or not the first area and the second area are areas of the same object based on a third buffer area obtained by adding a buffer area of a second scale larger than the first scale to the first area, and a fourth buffer area obtained by adding a buffer area of the second scale to the second area; A tracking method characterized in that processing is performed by a computer.

2. 2. The tracking method according to claim 1, wherein the determining process determines that the first area and the second area are areas of the same object if a value obtained by dividing the overlapping area of the first buffer area and the second buffer area by the combined area of the first buffer area and the second buffer area is equal to or greater than a threshold value.

3. The tracking method described in claim 2, characterized in that the re-determination process determines that the first area and the second area are areas of the same object if the value obtained by dividing the overlapping area of the third buffer area and the fourth buffer area by the combined area of the third buffer area and the fourth buffer area is greater than or equal to a threshold.

4. The tracking method described in claim 3, characterized in that if it is determined that the first area and the second area are areas of the same object, a process of generating track data is further performed by assigning the same identification number to the first area and the second area.

5. The tracking method according to claim 4, further comprising the steps of: selecting one set from a plurality of sets of the first scale and the second scale; evaluating the track data generated based on the first scale and the second scale of the selected set based on training data; and selecting an optimal set of the first scale and the second scale and a threshold value from the plurality of sets based on the evaluation results.

6. Detect the object area from multiple frames in a time series, determining whether the first area and the second area are areas of the same object based on a first buffer area obtained by adding a buffer area of a first scale to a first area of the object detected from a first frame included in the plurality of frames, and a second buffer area obtained by adding a buffer area of the first scale to a second area of the object detected from a second frame subsequent to the first frame; if it is determined that the first area and the second area are not areas of the same object, it is determined again whether or not the first area and the second area are areas of the same object based on a third buffer area obtained by adding a buffer area of a second scale larger than the first scale to the first area, and a fourth buffer area obtained by adding a buffer area of the second scale to the second area; A tracking program that causes a computer to execute a process.

7. Detect the object area from multiple frames in a time series, determining whether the first area and the second area are areas of the same object based on a first buffer area obtained by adding a buffer area of a first scale to a first area of the object detected from a first frame included in the plurality of frames, and a second buffer area obtained by adding a buffer area of the first scale to a second area of the object detected from a second frame subsequent to the first frame; if it is determined that the first area and the second area are not areas of the same object, it is determined again whether or not the first area and the second area are areas of the same object based on a third buffer area obtained by adding a buffer area of a second scale larger than the first scale to the first area, and a fourth buffer area obtained by adding a buffer area of the second scale to the second area; An information processing device comprising a control unit that executes processing.

Images