Object tracking device, object tracking method, and program

Abstract

A grouping unit of this object tracking device groups a plurality of tracked bodies into a plurality of assessment groups on the basis of a plurality of similarity scores that respectively correspond to a plurality of tracked-body pairs. An evaluation value calculation unit calculates a group evaluation value for each of the plurality of assessment groups. An identical-object-group determination unit determines, on the basis of the calculated plurality of group evaluation values for the plurality of assessment groups, at least one identical-object group pertaining to an identical object from among the plurality of assessment groups.

Description

Technical Field

[0001] The present disclosure relates to an object tracking device, an object tracking method, and a program.

Background Art

[0002] Techniques have been proposed for detecting an image of a region corresponding to a target object (object) in a captured image (that is, an object image) and tracking the target object (hereinafter sometimes referred to as a "tracking object") (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The present inventor has found that there is a possibility that a plurality of tracking objects are related to the same object, and there is a need to group (integrate) the plurality of tracking objects related to the same object into one object.

[0005] One object of the present disclosure is to provide an object tracking device, an object tracking method, and a program that can accurately integrate a plurality of tracking objects related to the same object. It should be noted that this object is only one of the plurality of objects that the plurality of embodiments disclosed in this specification are intended to achieve. Other objects or problems and novel features will be clarified from the description of this specification or the accompanying drawings.

Means for Solving the Problems

[0006] In one aspect, the object tracking device is An acquisition unit that acquires similarity scores between the two trackers included in each of the multiple tracker pairs, each of which consists of two trackers in multiple trackers, A grouping unit that groups the plurality of tracked object pairs into a plurality of determination groups, which are groups for determining whether each determination group is a group of common objects, based on a plurality of similarity scores corresponding to each of the plurality of tracked object pairs. A group evaluation value calculation unit that calculates a group evaluation value for each of the plurality of judgment groups, Based on the multiple group evaluation values ​​of the multiple judgment groups calculated above, the same object group determination unit determines at least one same object group for the same object from among the multiple judgment groups, It is equipped with.

[0007] In other embodiments, an object tracking method performed by an object tracking device is: Each tracker pair consists of multiple tracker pairs, each comprising two trackers in multiple trackers. The process involves obtaining similarity scores between the two trackers included in each tracker pair. Based on the multiple similarity scores corresponding to each of the multiple tracked object pairs, the multiple tracked objects are grouped into multiple determination groups, which are groups for determining whether each determination group is a group about a common object. To calculate the group evaluation value for each of the aforementioned multiple judgment groups, Based on the multiple group evaluation values ​​of the multiple judgment groups calculated above, at least one identical object group is determined from among the multiple judgment groups for the same object. Includes.

[0008] In other embodiments, the program Each tracker pair consists of multiple tracker pairs, each comprising two trackers in multiple trackers. The process involves obtaining similarity scores between the two trackers included in each tracker pair. Based on the multiple similarity scores corresponding to each of the multiple tracked object pairs, the multiple tracked objects are grouped into multiple determination groups, which are groups for determining whether each determination group is a group about a common object. To calculate the group evaluation value for each of the aforementioned multiple judgment groups, Based on the multiple group evaluation values ​​of the multiple judgment groups calculated above, at least one identical object group is determined from among the multiple judgment groups for the same object. The object tracking device is instructed to perform the process that includes this. [Effects of the Invention]

[0009] This disclosure provides an object tracking device, an object tracking method, and a program that can accurately integrate multiple tracking objects related to the same object. [Brief explanation of the drawing]

[0010] [Figure 1] A block diagram showing an example of an object tracking device in this disclosure. [Figure 2] This flowchart shows an example of the processing operation of the object tracking device disclosed herein. [Figure 3] A block diagram showing another example of an object tracking device in this disclosure. [Figure 4] This flowchart shows an example of the processing operation of the acquisition processing unit of the object tracking device disclosed herein. [Figure 5] This figure shows an example of a tracking information unit. [Figure 6] This flowchart shows an example of the processing operation of the similarity calculation unit of the object tracking device disclosed herein. [Figure 7] This figure shows an example of a similarity score table. [Figure 8] This flowchart shows an example of the processing operation of a specific part of the object tracking device disclosed herein. [Figure 9] This is a diagram used to explain the condition of non-identical objects. [Figure 10] It is a diagram showing an example of a similar tracking object list. [Figure 11] It is a flowchart showing an example of the processing operation of the determination group forming unit of the object tracking device of the present disclosure. [Figure 12] It is a flowchart showing an example of another processing operation of the determination group forming unit of the object tracking device of the present disclosure. [Figure 13] It is a flowchart showing an example of another processing operation of the determination group forming unit of the object tracking device of the present disclosure. [Figure 14] It is a diagram used for explaining the processing operation of the determination group forming unit of the object tracking device of the present disclosure. [Figure 15] It is a diagram used for explaining the processing operation of the determination group forming unit of the object tracking device of the present disclosure. [Figure 16] It is a diagram used for explaining the processing operation of the determination group forming unit of the object tracking device of the present disclosure. [Figure 17] It is a diagram used for explaining the processing operation of the determination group forming unit of the object tracking device of the present disclosure. [Figure 18] It is a flowchart showing an example of the processing operation of the evaluation value calculation unit of the object tracking device of the present disclosure. [Figure 19] It is a diagram showing an example of a group evaluation value table. [Figure 20] It is a flowchart showing an example of the processing operation of the same object group determination unit of the object tracking device of the present disclosure. [Figure 21] It is a diagram used for explaining the processing operation of the same object group determination unit of the object tracking device of the present disclosure. [Figure 22] It is a diagram showing the tracking object information unit after replacement. [Figure 23] It is a diagram showing an example of the hardware configuration of the object tracking device.

Embodiments for Carrying Out the Invention

[0011] The embodiments will be described below with reference to the drawings. In this disclosure, the drawings may be associated with one or more embodiments. Also, each element in the drawings may correspond to one or more embodiments. Furthermore, in the embodiments, the same or equivalent elements are denoted by the same reference numerals, and redundant descriptions are omitted.

[0012] <First Embodiment> <Example of object tracking device configuration> Figure 1 is a block diagram showing an example of an object tracking device according to the present disclosure. In Figure 1, the object tracking device 10 includes an acquisition unit 11, a grouping unit 12, an evaluation value calculation unit 13, and a unit 14 for determining the same object group.

[0013] The acquisition unit 11 acquires the "similarity score" between the two "trackers" included in each of the multiple "tracker pairs".

[0014] Each "tracking object pair" consists of two tracking objects from multiple tracking objects. That is, multiple tracking object pairs are formed by selecting all combinations of two tracking objects from multiple tracking objects. A "tracking object" means one or more objects (or one or more object images) that are captured in one or more image frames and are considered to be the same object. Each tracking object is assigned a tracking ID. The "tracking ID" is a tracking object identifier to which the same value is assigned to multiple objects captured in multiple image frames and considered to be the same, and different values ​​are assigned to multiple objects captured in multiple image frames and considered to be different from each other. As described above, two tracking objects may actually relate to the same object.

[0015] The "similarity score" for a tracked object pair is an index of the similarity between the tracked object feature information relating to the features of the first tracked object included in the tracked object pair and the tracked object feature information relating to the features of the second tracked object. For example, the "similarity score" for a tracked object pair may be the magnitude of the difference vector between the tracked object feature vector of the first tracked object included in the tracked object pair and the tracked object feature vector of the second tracked object included in the tracked object pair. That is, the "similarity score" for a tracked object pair may be the Euclidean distance between the endpoint of the tracked object feature vector of the first tracked object included in the tracked object pair and the endpoint of the tracked object feature vector of the second tracked object included in the tracked object pair. Note that the tracked object feature vector for a single tracked object may be calculated by averaging multiple object features (e.g., multiple object feature vectors) included in multiple tracked object information units related to a single tracked object, for example, as will be explained in detail in the second embodiment.

[0016] The grouping unit 12 groups multiple tracked objects into multiple decision groups based on multiple similarity scores corresponding to each of the multiple tracked object pairs. Each decision group is a group used to determine whether it is a group about a common object. In other words, each decision group is the subject of a determination as to whether one or more tracked objects included in that group relate to a common object.

[0017] The evaluation value calculation unit 13 calculates a "group evaluation value" for each of the multiple judgment groups. For example, the evaluation value calculation unit 13 may calculate a "group evaluation value" for each of the multiple judgment groups based on multiple similarity scores. Furthermore, the evaluation value calculation unit 13 may calculate the group evaluation value of a judgment group by taking into account the time and location where the tracked object included in the judgment group was detected (for example, the plausibility of the tracked object's movement trajectory). For example, the group evaluation value of a judgment group that includes a tracked object whose movement trajectory shows the tracked object suddenly moving to a distant location may be weighted to be lower.

[0018] The identical object group determination unit 14 determines at least one "identical object group" for the same object from among the multiple determination groups based on the multiple group evaluation values ​​of the multiple determination groups calculated.

[0019] <Example of object tracking device operation> Figure 2 is a flowchart showing an example of the processing operation of the object tracking device of this disclosure.

[0020] The acquisition unit 11 acquires similarity scores between the two trackers included in each of the multiple tracker pairs (step S11).

[0021] The grouping unit 12 groups multiple trackers into multiple decision groups based on multiple similarity scores corresponding to each of the multiple tracker pairs (step S12).

[0022] The evaluation value calculation unit 13 calculates a group evaluation value for each of the multiple judgment groups (step S13).

[0023] The identical object group determination unit 14 determines at least one identical object group for the same object from among the multiple determination groups based on the multiple group evaluation values ​​of the multiple determination groups calculated (step S14).

[0024] As described above, according to the first embodiment, in the object tracking device 10, the grouping unit 12 groups multiple tracked objects into multiple determination groups based on multiple similarity scores corresponding to each of the multiple tracked object pairs. The evaluation value calculation unit 13 calculates a group evaluation value for each of the multiple determination groups based on the multiple similarity scores. The identical object group determination unit 14 determines at least one identical object group for the same object among the multiple determination groups based on the multiple group evaluation values ​​of the multiple determination groups calculated.

[0025] The configuration of this object tracking device 10 allows for the determination of identical object groups based on the group evaluation values ​​of each of the multiple determination groups formed by grouping multiple tracked objects. This enables the integration of multiple tracked objects that are highly likely to be related to the same object. In other words, it allows for the accurate integration of multiple tracked objects related to the same object.

[0026] <Second Embodiment> The second embodiment relates to a more specific embodiment.

[0027] <Example of object tracking device configuration> Figure 3 is a block diagram showing an example of the object tracking device of the present disclosure. In Figure 3, the object tracking device 20 includes an acquisition unit 21, a grouping unit 22, an evaluation value calculation unit 23, and a same object group determination unit 24.

[0028] The acquisition unit 21, similar to the acquisition unit 11 in the first embodiment, acquires the "similarity score" between the two "trackers" included in each of the multiple "tracker pairs".

[0029] For example, as shown in Figure 3, the acquisition unit 21 includes an acquisition processing unit 21A and a similarity calculation unit 21B.

[0030] The acquisition processing unit 21A acquires multiple tracking information units. Each "tracking information unit" includes, for example, a tracking ID, location information, and object feature information. The "tracking ID" is a tracking identifier to which the same value is assigned to multiple objects considered to be the same as each other and captured in multiple image frames, and different values ​​are assigned to multiple objects considered to be different as they are captured in multiple image frames, as described above. The "location information" is, for example, information about the location of the object image corresponding to each tracking information unit in the image frame. The "object feature information" is information about the object features in the object image corresponding to each tracking information unit. Each "tracking information unit" may also include, for example, a camera ID. The "camera ID" is information that identifies the camera that took the image frame. Each "tracking information unit" may also include, for example, timing information. The "timing information" is information about the timing when the image frame was taken (for example, the time of capture).

[0031] The similarity calculation unit 21B calculates a similarity score between the two trackers included in each of the multiple tracker pairs, for each of the multiple tracker pairs, where each tracker pair consists of two trackers from the multiple trackers, based on the multiple tracker information units obtained.

[0032] For example, if there are multiple tracker information units containing the same tracking ID, the similarity calculation unit 21B calculates the average of the multiple object features (e.g., multiple object feature vectors) contained in each of the multiple tracker information units as the tracker feature (e.g., tracker feature vector) corresponding to that tracking ID. The similarity calculation unit 21B may then calculate the similarity score for each tracker pair using the two tracker features (e.g., two tracker feature vectors) of the two trackers contained in each tracker pair. Specifically, if the tracker information unit contains object feature vectors as object feature information, the similarity calculation unit 21B may calculate the magnitude of the difference vector between the tracker feature vector of the first tracker and the tracker feature vector of the second tracker in the tracker pair.

[0033] The grouping unit 22, similar to the grouping unit 12 in the first embodiment, groups multiple trackers into multiple decision groups based on multiple similarity scores corresponding to each of the multiple tracker pairs.

[0034] For example, as shown in Figure 3, the grouping unit 22 has a selection unit 22A and a determination group formation unit 22B.

[0035] The identification unit 22A uses multiple tracker information units and multiple similarity scores calculated for multiple tracker pairs to identify tracker pairs consisting of two trackers that satisfy the "non-identical object condition" and whose similarity score is equal to or greater than the "first threshold". Hereinafter, this identified tracker pair may be referred to as a "non-identical object related similar tracker pair" or simply a "similar tracker pair". The "non-identical object condition" is a condition that is not satisfied when two objects are the same object. The "non-identical object condition" includes at least one of the temporal and spatial conditions.

[0036] For example, the "non-identical object condition" may include at least one of the following "first condition" and "second condition". The "first condition" is that both of the two tracked objects included in the tracked object pair are present in the first image frame in which the first area is captured by the first camera at the first timing. The "second condition" is that the first tracked object, one of the two tracked objects included in the tracked object pair, is present in the first image frame in which the first area is captured by the first camera at the first timing, and the second tracked object is present in the second image frame in which the second area, which does not overlap with the first area, is captured by the second camera at the second timing, and the absolute value of the difference between the first timing and the second timing is greater than or equal to zero and less than or equal to a predetermined value.

[0037] The determination group formation unit 22B performs a grouping process that groups multiple tracked objects into multiple determination groups.

[0038] For example, the determination group formation unit 22B performs grouping using tracked object pairs that correspond to similarity scores that have a value of "second threshold" or higher among multiple similarity scores. In this case, the determination group formation unit 22B performs grouping on tracked object pairs in order from those with the highest similarity scores.

[0039] Furthermore, the determination group formation unit 22B, in the grouping process, creates a first determination group that includes a first tracker included in the "similar tracker pair" and at least one other tracker, and a second determination group that includes a second tracker included in the "similar tracker pair" and at least one other tracker, the same as in the first determination group, as part of a plurality of determination groups.

[0040] Furthermore, if the grouping process finds that two tracking objects satisfying the "non-identical object condition" are included in the target group, the grouping unit 22B executes a process to remove one of the two tracking objects from the target group.

[0041] The evaluation value calculation unit 23, similar to the evaluation value calculation unit 13 in the first embodiment, calculates a "group evaluation value" for each of the multiple judgment groups based on multiple similarity scores. For example, the evaluation value calculation unit 23 identifies similarity scores that are equal to or greater than the "second threshold" among the multiple similarity scores corresponding to each of the multiple tracked object pairs in each judgment group. Then, the evaluation value calculation unit 23 calculates a group evaluation value for each judgment group by accumulating the identified similarity scores that are equal to or greater than the second threshold.

[0042] The identical object group determination unit 24, similar to the identical object group determination unit 14 in the first embodiment, determines at least one "identical object group" for the same object from among the multiple determination groups based on the multiple group evaluation values ​​of the multiple determination groups calculated.

[0043] For example, the identical object group determination unit 24 creates a graph as follows: The identical object group determination unit 24 creates a graph in which each judgment group is a "node," two judgment groups that cannot exist simultaneously are connected by "edges," and the group evaluation value of each judgment group is the "node weight." Then, the identical object group determination unit 24 applies a Maximum Weight Independent Set (MWIS) solver to the created graph to calculate the set of independent weights. In other words, the identical object group determination unit 24 determines the combination of judgment groups that can exist simultaneously and have the highest sum of group evaluation values ​​as the "identical object group."

[0044] The identical object group determination unit 24 then includes an object ID unique to the "identical object group" in each of the one or more tracking information units corresponding to the tracking objects included in each determination group determined to be an "identical object group".

[0045] <Example of object tracking device operation> An example of the processing operation of the object tracking device 20 having the above configuration will be described.

[0046] (Example of processing operation of acquisition processing unit 21A) Figure 4 is a flowchart showing an example of the processing operation of the acquisition processing unit of the object tracking device of this disclosure.

[0047] The acquisition processing unit 21A receives video footage captured by a camera (not shown) and extracts image frames from the video in chronological order (step S21). This results in multiple image frames.

[0048] The acquisition processing unit 21A selects the image frame with the earliest timestamp from among the multiple image frames obtained as the "current image frame" (step S22).

[0049] The acquisition processing unit 21A detects all objects of all object classes (object regions, object images) based on the selected "current image frame" and the tracking results from one or more frames preceding this current image frame (hereinafter sometimes simply referred to as "previous frames") (step S23). The acquisition processing unit 21A then extracts feature quantities related to the objects contained in the detected object images (step S23). The acquisition processing unit 21A also assigns an object number to each detected object (object image) (step S23). The acquisition processing unit 21A also assigns a tracking ID to each detected object (object image) (step S23). The acquisition processing unit 21A may also identify the position of each object image within the image frame.

[0050] For example, the image acquisition processing unit 21A detects the region corresponding to the target object (i.e., the "object region (object image)") in each of the multiple image frames. The image frame is labeled with, for example, the time or frame number of the image frame as image identification information. If the target object is a person, the image acquisition processing unit 21A may use a detector that has learned the image features of a person to detect the object region (i.e., the person region). For example, the image acquisition processing unit 21A may use a detector that detects based on HOG (Histograms of Oriented Gradients) features or a detector that detects directly from the image using a CNN (Convolutional Neural Network). Alternatively, the image acquisition processing unit 21A may detect a person using a detector that has learned a part of a person (e.g., the head) rather than the whole person. For example, the image acquisition processing unit 21A may use a detector that has learned the head and feet, and identify the person region by detecting the head position and the foot position. For example, the acquisition processing unit 21A may determine the human region by combining silhouette information (information about the region that differs from the background model) obtained by background subtraction with head detection information.

[0051] The acquisition processing unit 21A then identifies the position of each object region in the image frame. For example, the acquisition processing unit 21A may identify the position of the rectangular region surrounding the object region (for example, the region inside the outline of the object region) as the "position of the object region in the image frame". The position of this rectangular region may be represented, for example, by the coordinates of the vertices of the rectangular region (for example, the coordinates of the top-left vertex and the coordinates of the bottom-right vertex). Alternatively, the position of this rectangular region may be represented, for example, by the coordinates of one vertex (for example, the coordinates of the top-left vertex) and the width and height of the rectangular region.

[0052] Then, the acquisition processing unit 21A assigns an "object number" to each object region. The acquisition processing unit 21A assigns a different object number to each object region.

[0053] Then, the acquisition processing unit 21A performs a "tracking process" using the information about each detected object region. By performing the "tracking process," the acquisition processing unit 21A assigns the same "tracking ID" to all object regions of the same target object (for example, person A).

[0054] For example, the acquisition processing unit 21A applies a Kalman filter or a particle filter to object regions detected in one or more image frames that are temporally prior to the first image frame and that have been assigned tracking ID #1, in order to predict the region (predicted region) in the first image frame where the object region corresponding to tracking ID #1 exists. Then, the acquisition processing unit 21A assigns tracking ID #1 to the object regions in the first image frame that overlap with the predicted region, for example. Alternatively, the acquisition processing unit 21A may perform the tracking process using the KLT (Kanade-Lucas-Tomasi) algorithm.

[0055] The acquisition processing unit 21A determines whether there is an image frame among the multiple image frames that has not been selected as the "current image frame" (step S24). If there is an image frame that has not yet been selected (step S24 YES), the acquisition processing unit 21A selects the image frame with the earliest time among the unselected image frames as the "current image frame" (step S22). If there is no image frame that has not yet been selected (step S24 NO), the processing flow shown in Figure 4 ends.

[0056] The above processing operations yield multiple tracking information units. Figure 5 shows an example of a tracking information unit. In Figure 5, multiple tracking information units are shown in a table format. Each entry (i.e., each row) in Figure 5 corresponds to one tracking information unit.

[0057] In the example shown in Figure 5, the tracking information unit includes values ​​for the items "Object Number," "Time of Appearance," "Camera ID," "Tracking ID," "Location Information," and "Feature Information." The value of the item "Time of Appearance" corresponds, for example, to the time of the image frame shown above. The value of the item "Camera ID" is the identification information of the camera (not shown) that captured the video received by the acquisition processing unit 21A. That is, for example, when the acquisition processing unit 21A receives video from cameras 1 and 2, the tracking information unit obtained from the video from camera 1 includes "1" as the value of the item "Camera ID," and the tracking information unit obtained from the video from camera 2 includes "2" as the value of the item "Camera ID." In the example in Figure 5, the tracking information unit includes the coordinates (Left, Top) of the top-left vertex of the rectangular area surrounding the object area, as well as the width (Width) and height (Height) of the rectangular area, as "Position of the object area in the image frame." Furthermore, in the example shown in Figure 5, the tracker information unit includes an N-dimensional vector as feature information.

[0058] (Example of processing operation of the similarity calculation unit 21B) Figure 6 is a flowchart showing an example of the processing operation of the similarity calculation unit of the object tracking device of this disclosure.

[0059] The similarity calculation unit 21B calculates the tracker feature quantity for each tracker (i.e., each tracker ID) based on the multiple tracker information units acquired by the acquisition processing unit 21A (step S31).

[0060] The similarity calculation unit 21B forms multiple tracker pairs by selecting all combinations of two trackers from multiple trackers (multiple tracker IDs) included in multiple tracker information units acquired by the acquisition processing unit 21A (step S32).

[0061] The similarity calculation unit 21B selects one of the multiple tracked object pairs formed as the tracked object pair to be used for similarity calculation (step S33).

[0062] The similarity calculation unit 21B calculates the similarity score of the selected tracked object pair (step S34).

[0063] The similarity calculation unit 21B determines whether there are any unselected tracked object pairs (step S35).

[0064] If there are unselected tracked object pairs (step S35YES), the similarity calculation unit 21B selects one of the unselected tracked object pairs as the tracked object pair to be used for similarity calculation (step S33). On the other hand, if there are no unselected tracked object pairs (step S35NO), the processing flow in Figure 6 ends.

[0065] Through the above processing steps, a similarity score is obtained for each of the multiple tracked object pairs. Figure 7 shows an example of a similarity score table.

[0066] In the example shown in Figure 7, the similarity score table contains multiple entries, each entry associating information about a tracker pair (i.e., the IDs of the first and second trackers included in the tracker pair) with the similarity score corresponding to that tracker pair.

[0067] (Example of processing operation of specific unit 22A) Figure 8 is a flowchart showing an example of the processing operation of the specific unit of the object tracking device of this disclosure.

[0068] The identification unit 22A selects one of several entries in the similarity score table (step S41).

[0069] The identification unit 22A determines whether the two tracked objects of the selected entry's tracked object pair satisfy the "non-identical object condition" (step S42). Figure 9 is a diagram illustrating the non-identical object condition. Figure 9 shows information regarding temporal and spatial constraints (spatiotemporal constraint information). Each entry in Figure 9 associates information about the camera pair (ID of the first camera and ID of the second camera) with the "no appearance period". For example, the first entry indicates that in order for the same tracked object (object) that was photographed by camera ID 1 at the first timing (appearance time) to be photographed by camera ID 5 at the second timing (appearance time), the interval between the first timing (appearance time) and the second timing (appearance time) must not be less than 10 seconds. In other words, a tracked object captured by camera ID 1 at the first timing (appearance time) and a tracked object captured by camera ID 5 within 10 seconds of the first timing (appearance time) are determined to be non-identical tracked objects (non-identical objects). For example, this means that it takes at least 10 seconds to move from the shooting area of ​​camera ID 1 to the shooting area of ​​camera ID 5.

[0070] Furthermore, the third entry indicates that for the same tracked object (object) captured by camera ID1 at the first timing (appearance time) to be captured by camera ID2 at the second timing (appearance time), the interval between the first timing (appearance time) and the second timing (appearance time) must not be less than -1 second. This means that the shooting areas of camera ID1 and camera ID2 overlap, and it is permissible for the same tracked object (object) to be captured by both camera ID1 and camera ID2 at the same timing.

[0071] If the two tracked objects of the selected entry's tracked object pair satisfy the "non-identical object condition" (step S42YES), the identification unit 22A determines whether the similarity score of the tracked object pair that satisfies the "non-identical object condition" is equal to or greater than the first threshold (step S43).

[0072] If the similarity score of a pair of tracked objects that satisfy the "non-identical object condition" is equal to or greater than the first threshold (step S43 YES), the identification unit 22A adds the pair of tracked objects that satisfy the "non-identical object condition" and whose corresponding similarity score is equal to or greater than the first threshold to the "similar tracked object list" (step S44).

[0073] The identification unit 22A determines whether there are any entries in the similarity score table that have not yet been selected (step S45).

[0074] If there are entries in the similarity score table that have not yet been selected (step S45YES), the identification unit 22A selects one of the unselected entries (step S41). If there are no entries in the similarity score table that have not yet been selected (step S45NO), the processing flow ends.

[0075] If the two tracked objects in the selected entry's tracked object pair do not satisfy the "non-identical object condition" (step S42NO), the processing flow proceeds to step S45. Also, if the similarity score of the tracked object pair that satisfies the "non-identical object condition" is less than the first threshold (step S43NO), the processing flow proceeds to step S45.

[0076] Figure 10 shows an example of a similar tracker list. The similar tracker list shown in Figure 10 indicates that the tracker corresponding to tracker ID 3 and the tracker corresponding to tracker ID 4 are a "non-identical object related similar tracker pair".

[0077] (Example of processing operation of the determination group formation unit 22B) Figure 11 is a flowchart illustrating an example of the processing operation of the determination group formation unit of the object tracking device of this disclosure. Figure 11 particularly relates to the formation of a list of tracked object pairs to be grouped, which includes the similarity scores of the tracked object pairs to be grouped.

[0078] The judgment group formation unit 22B sorts the entries in the similarity score table in descending order of similarity score (step S51).

[0079] The determination group formation unit 22B selects the entry with the highest similarity score in the similarity score table after sorting (step S52).

[0080] The determination group formation unit 22B determines whether the similarity score of the selected entries is equal to or greater than the second threshold (step S53).

[0081] If the similarity score of the selected entry is equal to or greater than the second threshold (step S53YES), the determination group formation unit 22B adds the entry to the grouping target tracker pair list (step S54).

[0082] The determination group formation unit 22B determines whether there are any entries in the similarity score table after sorting that have not yet been selected (step S55).

[0083] If there are unselected entries in the sorted similarity score table (step S55YES), the determination group formation unit 22B selects the entry with the highest similarity score among the unselected entries in the sorted similarity score table (step S52). If there are no unselected entries in the sorted similarity score table (step S55NO), the processing flow ends.

[0084] If the similarity score of the selected entry is less than the second threshold (step S53NO), the processing flow proceeds to step S55.

[0085] For example, if the second threshold value is set to 0.6, entries 1 through 5 in the similarity score table in Figure 7 will be added to the grouping target tracker pair list. In the grouping target tracker pair list as well, the entries are sorted in descending order of similarity score.

[0086] Figure 12 is a flowchart illustrating an example of another processing operation of the determination group formation unit of the object tracking device of this disclosure. Figure 12 particularly relates to the formation of a queue for group update processing. Note that the processing flow in Figure 12 is executed for one entry (pair of tracked objects), and the processing flow in Figure 13 is executed for the group update processing queue formed thereby. This series of processes may be repeatedly executed while sequentially changing the entry (pair of tracked objects) to be executed.

[0087] The determination group formation unit 22B selects the entry with the highest similarity score from the grouping target tracker pair list (step S61).

[0088] The determination group formation unit 22B adds the selected entry's tracking pair to the group update processing queue (step S62). However, if the selected entry's tracking pair is the same as any of the tracking pair formed in step S64 based on a previously selected entry's tracking pair, the determination group formation unit 22B does not need to add the selected entry's tracking pair to the group update processing queue. Furthermore, the determination group formation unit 22B may skip the processing in steps S63 and S64 for the selected entry's tracking pair.

[0089] The determination group formation unit 22B determines whether the selected entry's tracker pair includes a tracker corresponding to a similar tracker (step S63). Hereafter, the tracker corresponding to a similar tracker may simply be referred to as a "similar tracker". For example, in the example shown in Figure 10, the tracker for tracker ID 3 and the tracker for tracker ID 4 are similar trackers. If the selected entry's tracker pair does not include a tracker corresponding to a similar tracker (step S63NO), the processing flow proceeds to step S65.

[0090] If the selected entry's tracker pair includes a tracker corresponding to a similar tracker (step S63 YES), the determination group formation unit 22B adds the resulting tracker pair to the group update processing queue by replacing the similar tracker in that tracker pair with the similar tracker associated with that similar tracker in the similar tracker list (step S64). Furthermore, the determination group formation unit 22B adds the resulting tracker pair to the group update processing queue by replacing the similar tracker in that tracker pair with a dummy tracker (step S64). Hereinafter, the tracker pair obtained by replacing the similar tracker as described above may be referred to as a "similar replacement tracker pair". Furthermore, the determination group formation unit 22B adds the resulting tracker pair to the group update processing queue by replacing the similar tracker in the similar replacement tracker pair with a dummy tracker (step S64). If, in step S64, the tracking object pair to be added to the group update processing target queue is already held in the group update processing target queue, the determination group formation unit 22B may skip adding that tracking object pair to the group update processing target queue.

[0091] The determination group formation unit 22B determines whether there are any entries in the grouping target tracker pair list that have not yet been selected (step S65).

[0092] If there are entries that have not yet been selected in the grouping target tracking pair list (step S65YES), the determination group formation unit 22B selects the entry with the highest similarity score among the entries that have not yet been selected in the grouping target tracking pair list (step S61). If there are no entries that have not yet been selected in the grouping target tracking pair list (step S65NO), the processing flow in Figure 12 ends.

[0093] Figure 13 is a flowchart illustrating an example of another processing operation of the determination group formation unit of the object tracking device of this disclosure. Figure 13 particularly relates to the formation (updating) of determination groups.

[0094] The determination group formation unit 22B selects the tracker pair that was added earliest from among the tracker pairs included in the queue to be updated (step S71).

[0095] The determination group formation unit 22B determines whether a determination group already exists that includes each of the two trackers included in the selected tracker pair (step S72).

[0096] If neither of the two trackers included in the selected tracker pair is present in any determination group (step S72NO), the determination group formation unit 22B forms a new determination group that includes the selected tracker pair (step S73).

[0097] The determination group formation unit 22B determines whether there are any tracked object pairs that have not yet been selected in the queue for group update processing (step S74).

[0098] If there are unselected tracker pairs in the queue for group update processing (step S74YES), the determination group formation unit 22B selects the tracker pair that was added earliest among the unselected tracker pairs (step S71). If there are no unselected tracker pairs in the queue for group update processing (step S74NO), the processing flow in Figure 13 ends.

[0099] If a determination group exists that includes one of the selected tracked objects from the pair (step S72YES), the determination group formation unit 22B adds the other tracked object from that pair to the determination group (step S75).

[0100] The determination group formation unit 22B determines whether the determination group updated in step S75 includes a combination of tracked objects that satisfy the "non-identical object condition" (step S76).

[0101] If the combination of tracked objects that satisfies the "non-identical object condition" is included (step S76YES), the determination group formation unit 22B removes the tracked object added in step S75 from the determination group updated in step S75 (step S77). Furthermore, the determination group formation unit 22B forms a new determination group that includes only the removed tracked object (step S77). The processing flow then proceeds to step S74.

[0102] If no combination of tracked objects satisfying the "non-identical object condition" is found (step S76NO), the processing flow proceeds to step S74.

[0103] Figures 14 to 17 illustrate the processing operation of the determination group formation unit of the object tracking device of this disclosure.

[0104] First, as shown in Figure 14, let's assume that over time, the acquisition processing unit 21A detects six tracked objects, IDs 1 to 6. In reality, IDs 1, 3, and 5 are associated with the same object, while IDs 2, 4, and 6 are associated with different objects. However, this is not known at this time. However, since tracked objects IDs 1 and 2 are detected at overlapping times, they are not associated with the same object. The same can be said for tracked objects 3 and 4, and tracked objects 5 and 6.

[0105] Furthermore, it is assumed that the grouping target tracking pair list shown in Figure 15 is formed based on the processing of the similarity calculation unit 21B and the processing of the determination group formation unit 22B shown in Figure 11. It is also assumed that the similar tracking list shown in Figure 10 is formed by the processing of the identification unit 22A.

[0106] In this situation, the determination group formation unit 22B selects the entry with the highest similarity score in the grouping target tracker pair list (i.e., a tracker pair consisting of two trackers with tracker IDs 1 and 5) (step S61).

[0107] Then, the determination group formation unit 22B adds the tracker pair with tracker IDs 1 and 5 to the queue for group update processing (step S62). The tracker pair with tracker IDs 1 and 5 does not include any trackers corresponding to similar trackers (step S63NO). Therefore, after step S65, the determination group formation unit 22B selects the entry with the highest similarity score among the entries that have not yet been selected in the grouping target tracker pair list (i.e., the tracker pair consisting of the two trackers with tracker IDs 1 and 4) (step S61).

[0108] Then, the determination group formation unit 22B adds the tracker pair with tracker IDs 1 and 4 to the group update processing queue (step S62). Tracker IDs 1 and 4 include trackers corresponding to similar trackers (i.e., the tracker of tracker ID 4 included in the similar tracker list) (step S63 YES). The determination group formation unit 22B adds the tracker pair obtained by replacing the similar tracker in that tracker pair with a similar tracker associated with this similar tracker in the similar tracker list (i.e., the tracker pair with tracker IDs 1 and 3) to the group update processing queue (step S64). Furthermore, the determination group formation unit 22B adds the tracker obtained by replacing the similar tracker in that tracker pair with a dummy tracker (i.e., the tracker pair with tracker IDs 0 and 3) to the group update processing queue (step S64). Here, tracker ID 0 is the ID of the dummy tracker. Furthermore, the determination group formation unit 22B adds the tracked object obtained by replacing the similar tracked object in the similar replacement tracked object pair with a dummy tracked object (i.e., the tracked object pair with tracked object IDs 0 and 4) to the group update processing target queue (step S64).

[0109] This process is repeated for the other entries in the grouping target tracker pair list (i.e., the tracker pair with tracker IDs 2 and 4, and the tracker pair with tracker IDs 2 and 6). As mentioned above, the tracker pair with tracker IDs 1 and 3 has already been considered in the processing for tracker IDs 1 and 4. Therefore, processing for the tracker pair with tracker IDs 1 and 3 may be skipped.

[0110] In this way, a queue for group update processing is formed.

[0111] The determination group formation unit 22B selects the tracking object pair that was added earliest among the tracking object pairs included in the queue for group update processing (the tracking object pair with tracking object IDs 1 and 5) (step S71).

[0112] No determination group exists that includes each of the two trackers in the selected tracker pair (step S72NO). Therefore, the determination group formation unit 22B forms a new determination group (the leftmost determination group in Figure 16) that includes the selected tracker pair (the tracker pair with tracker IDs 1 and 5) (step S73).

[0113] The determination group formation unit 22B selects the earliest added tracker pair among the tracker pairs that have not yet been selected (the tracker pair with tracker IDs 1 and 4) (step S71). A determination group containing tracker ID 1 exists (step S74 YES). Therefore, the determination group formation unit 22B adds the tracker with tracker ID 4 to that determination group (step S75).

[0114] The determination group formation unit 22B selects the earliest added tracker pair among the tracker pairs that have not yet been selected (the tracker pair with tracker IDs 1 and 3) (step S71). A determination group containing tracker ID 1 exists (step S74 YES). Therefore, the determination group formation unit 22B adds the tracker with tracker ID 3 to that determination group (step S75). Here, the tracker with tracker ID 3 and the tracker with tracker ID 4 are a "similar tracker pair". For this reason, as shown in Figure 16, the system is branched into determination group A, consisting of trackers with tracker IDs 1, 5, and 3, and determination group B, consisting of tracker IDs 1, 5, and 4.

[0115] The determination group formation unit 22B selects the earliest added tracker pair among the tracker pairs that have not yet been selected (the tracker pair with tracker IDs 0 and 3) (step S71). Here, when the tracker pair to be processed includes dummy tracker ID 0, both of the trackers included in the selected tracker pair are treated as not existing in any determination group (step S72NO). For this reason, a new determination group (the third determination group from the left in Figure 16) is formed that includes the selected tracker pair (the tracker pair with tracker IDs 0 and 3) (step S73).

[0116] The determination group formation unit 22B selects the earliest added tracker pair among the tracker pairs that have not yet been selected (the tracker pair with tracker IDs 0 and 4) (step S71). Here, when the tracker pair to be processed includes dummy tracker ID 0, both of the trackers included in the selected tracker pair are treated as not existing in any determination group (step S72NO). Therefore, a new determination group (the fourth determination group from the left in Figure 16) is formed that includes the selected tracker pair (the tracker pair with tracker IDs 0 and 4) (step S73).

[0117] The determination group formation unit 22B selects the earliest added tracked object pair among the tracked object pairs that have not yet been selected (the tracked object pair with tracked object IDs 2 and 4) (step S71). A determination group (determination groups B and F in Figure 16) that includes tracked object ID 4 exists (step S74 YES). Therefore, the determination group formation unit 22B adds the tracked object with tracked object ID 2 to determination groups B and F (step S75). However, in determination group B, the tracked object with tracked object ID 1 and the tracked object with tracked object ID 2 satisfy the "non-identical object condition" (step S76 YES). Therefore, the determination group formation unit 22B removes the added tracked object with tracked object ID 2 from determination group B (step S77). Furthermore, the determination group formation unit 22B forms a new determination group (determination group G in Figure 16) that includes only the removed tracked object with tracked object ID 2 (step S77).

[0118] The determination group formation unit 22B selects the earliest added tracked body pair (the tracked body pair with tracked body IDs 2 and 3) from among the tracked body pairs that have not yet been selected (step S71). A determination group exists that includes tracked body ID 3 (determination groups A and D in Figure 16) (step S74 YES). Therefore, the determination group formation unit 22B adds the tracked body with tracked body ID 2 to determination group A (step S75). However, in determination group A, the tracked body with tracked body ID 1 and the tracked body with tracked body ID 2 satisfy the "non-identical object condition" (step S76 YES). Therefore, the determination group formation unit 22B removes the added tracked body with tracked body ID 2 from determination group A (step S77).

[0119] The determination group formation unit 22B selects the earliest added tracker pair among the tracker pairs that have not yet been selected (the tracker pair with tracker IDs 2 and 6) (step S71). A determination group containing tracker ID 2 exists (determination groups D, F, and G in Figure 16) (step S74 YES). Therefore, the determination group formation unit 22B adds the tracker with tracker ID 6 to determination groups D, F, and G (step S75).

[0120] Through this process, the determination group formation unit 22B creates a "determination group list," for example, as shown in Figure 17. As shown in Figure 17, the "determination group list" associates each of the multiple determination groups with the IDs of the tracking entities that constitute each determination group.

[0121] (Example of processing operation of the evaluation value calculation unit 23) Figure 18 is a flowchart showing an example of the processing operation of the evaluation value calculation unit of the object tracking device of this disclosure.

[0122] The evaluation value calculation unit 23 selects one judgment group from the "judgment group list" (step S81).

[0123] The evaluation value calculation unit 23 identifies a pair of trackers that has a similarity score of 2 or higher from among all the tracker pairs that can be formed from the multiple trackers included in the judgment group selected in step S81 (step S82). Note that if the judgment group contains only one tracker, it is not possible to form a tracker pair. Therefore, the group evaluation value for this judgment group becomes zero.

[0124] The evaluation value calculation unit 23 calculates a "group evaluation value" for the judgment group selected in step S81 based on the similarity scores of the tracked object pairs identified in step S82 (step S83). For example, the evaluation value calculation unit 23 may calculate the group evaluation value for the judgment group selected in step S81 by accumulating the similarity scores of the tracked object pairs identified in step S82. Furthermore, the evaluation value calculation unit 23 may calculate the group evaluation value of the judgment group by taking into account the time and location where the tracked objects included in the judgment group were detected (for example, the plausibility of the tracked object's movement trajectory). For example, the group evaluation value of a judgment group that includes a tracked object whose movement trajectory shows the tracked object suddenly moving to a distant location may be weighted to be lower. That is, the evaluation value calculation unit 23 may calculate the group evaluation value by adding the evaluation value calculated based on the similarity scores of the tracked object pairs and the evaluation value based on the trajectory, or by multiplying the evaluation value calculated based on the similarity scores of the tracked object pairs and the evaluation value based on the trajectory.

[0125] The evaluation value calculation unit 23 determines whether there are any judgment groups in the "judgment group list" that have not yet been selected (step S84).

[0126] If there are judgment groups in the "Judgment Group List" that have not yet been selected (step S84YES), the evaluation value calculation unit 23 selects one judgment group that has not yet been selected in the "Judgment Group List" (step S81). If there are no judgment groups in the "Judgment Group List" that have not yet been selected (step S84NO), the processing flow in Figure 18 ends.

[0127] For example, in the "Determination Group List" in Figure 17, determination group A includes trackers with tracking IDs 1, 5, and 3. In determination group A, the tracker pairs of tracking IDs 1 and 3, tracking IDs 1 and 5, and tracking IDs 3 and 5 can be formed. Among the tracker pairs of tracking IDs 1 and 3, tracking IDs 1 and 5, and tracking IDs 3 and 5, the tracker pairs corresponding to a similarity score of 0.6 or higher are the tracker pairs of tracking IDs 1 and 3 and the tracker pairs of tracking IDs 1 and 5. Therefore, the evaluation value calculation unit 23 calculates the group evaluation value (1.42) for determination group A by multiplying the similarity score of the tracker pair of tracking IDs 1 and 3 (0.62) and the similarity score of the tracker pair of tracking IDs 1 and 5 (0.8).

[0128] By calculating the group evaluation value for each judgment group in this way, a "group evaluation value table" like the one shown in Figure 19 is obtained. As shown in Figure 19, the "group evaluation value table" associates multiple judgment groups with the corresponding tracking ID and group evaluation value for each judgment group. Figure 19 is a diagram showing an example of a group evaluation value table.

[0129] (Example of processing operation of the identical object group determination unit 24) Figure 20 is a flowchart illustrating an example of the processing operation of the identical object group determination unit of the object tracking device of this disclosure. Figure 21 is a diagram illustrating the processing operation of the identical object group determination unit of the object tracking device of this disclosure.

[0130] The identical object group determination unit 24 uses a group evaluation value table to create a graph in which each determination group is a "node," two determination groups that cannot exist simultaneously are connected by "edges," and the group evaluation value of each determination group is used as the "node weight" (step S91). An example of this graph is shown at the bottom of Figure 21.

[0131] The identical object group determination unit 24 applies a Maximum Weight Independent Set (MWIS) solver to the created graph to calculate the independent weight sets and determine the "identical object groups" (step S92). In other words, the identical object group determination unit 24 determines the combination of group evaluation values ​​that has the highest sum among all possible combinations of judgment groups as the "identical object group".

[0132] The identical object group determination unit 24 assigns the group ID of the identical object group as the object ID to the multiple tracking IDs included in the identical object group (step S93). In the example in Figure 21, determination group A and determination group F are determined to be identical object groups. Then, object ID "A" is assigned to tracking IDs 1, 5, and 3 included in determination group A, respectively. Also, object ID "F" is assigned to tracking IDs 4, 2, and 6 included in determination group F, respectively.

[0133] Furthermore, the same object group determination unit 24 may replace the "Tracking ID" item in the tracking information unit shown in Figure 5 with the "Object ID" item. Figure 22 shows the tracking information unit after the replacement.

[0134] As described above, according to the second embodiment, the grouping unit 22 in the object tracking device 20 performs grouping processing on tracked object pairs in order from those with the highest similarity scores.

[0135] The configuration of this object tracking device 20 allows for the priority gathering of multiple tracking objects that are likely to be involved with the same object.

[0136] Furthermore, the grouping unit 22 in the object tracking device 20 identifies a pair of tracked objects (a pair of similar tracked objects related to a non-identical object) that satisfies the non-identical object condition that is not satisfied when the two objects are the same object, and whose similarity score is equal to or greater than the first threshold. The grouping unit 22 then creates a first determination group, which includes the first tracked object included in the identified pair of tracked objects and at least one other tracked object, and a second determination group, which includes the second tracked object included in the identified pair of tracked objects and at least one other tracked object, as part of a plurality of determination groups.

[0137] The configuration of this object tracking device 20 allows for the formation of two separate determination groups, each containing two tracked objects whose similarity scores have coincidentally become high due to environmental factors such as lighting. This reduces the possibility that determination groups containing two tracked objects whose similarity scores have coincidentally become high due to environmental factors will be determined to be the same object group. In other words, it allows for the accurate grouping of multiple tracked objects that are highly likely to be related to the same object.

[0138] <Other Embodiments> Figure 23 shows an example of the hardware configuration of an object tracking device. In Figure 23, the object tracking device 100 has a processor 101 and a memory 102. The processor 101 may be, for example, a microprocessor, an MPU (Micro Processing Unit), or a CPU (Central Processing Unit). The processor 101 may include multiple processors. The memory 102 is composed of a combination of volatile memory and non-volatile memory. The memory 102 may include storage located away from the processor 101. In this case, the processor 101 may access the memory 102 via an I / O interface that is not shown.

[0139] The object tracking devices 10 and 20 of the first and second embodiments may each have the hardware configuration shown in Figure 23. The acquisition units 11 and 21, the grouping units 12 and 22, the evaluation value calculation units 13 and 23, and the identical object group determination units 14 and 24 of the object tracking devices 10 and 20 of the first and second embodiments may be implemented by the processor 101 reading and executing a program stored in the memory 102. The program can be stored using various types of non-transitory computer-readable media and supplied to the object tracking devices 10 and 20. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives) and magneto-optical recording media (e.g., magneto-optical disks). Furthermore, examples of non-transitory computer-readable media include CD-ROMs (Read Only Memory), CD-Rs, and CD-R / Ws. Furthermore, examples of non-transitory computer-readable media include semiconductor memory. Semiconductor memory includes, for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, and RAM (Random Access Memory). The program may also be supplied to the object tracking devices 10, 20 by various types of transient computer-readable medium. Examples of transient computer-readable medium include electrical signals, optical signals, and electromagnetic waves. The transient computer-readable medium can supply the program to the object tracking devices 10, 20 via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.

[0140] While the present disclosure has been described above with reference to embodiments, the disclosure is not limited thereto. Various modifications to the structure and details of the present disclosure are possible, as can be understood by those skilled in the art within the scope of the disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0141] Some or all of the above embodiments may also be described as follows, but are not limited to the following: (Note 1) An acquisition unit that acquires similarity scores between the two trackers included in each of the multiple tracker pairs, each of which consists of two trackers in multiple trackers, A grouping unit that groups the plurality of tracked object pairs into a plurality of determination groups, which are groups for determining whether each determination group is a group of common objects, based on a plurality of similarity scores corresponding to each of the plurality of tracked object pairs. A group evaluation value calculation unit that calculates a group evaluation value for each of the plurality of judgment groups, Based on the multiple group evaluation values ​​of the multiple judgment groups calculated above, the same object group determination unit determines at least one same object group for the same object from among the multiple judgment groups, An object tracking device equipped with the following. (Note 2) The grouping unit performs the grouping process on the tracked object pairs in order, starting with the pair with the highest similarity score. The object tracking device described in Appendix 1. (Note 3) The grouping unit identifies a pair of trackers consisting of two trackers that satisfy the non-identical object condition not met when the two objects are identical and whose similarity score is equal to or greater than a first threshold. It then creates a first determination group as part of the plurality of determination groups, which includes a first tracker included in the identified pair of trackers and at least one other tracker, and a second determination group which includes a second tracker included in the identified pair of trackers and the same at least one other tracker as in the first determination group. The object tracking device described in Appendix 1. (Note 4) The grouping unit groups the plurality of trackers into the plurality of judgment groups using tracker pairs corresponding to similarity scores having a value of 2 or higher among the plurality of similarity scores. An object tracking device as described in Appendix 1 or 2. (Note 5) The grouping unit, when it finds that two tracking objects satisfy the non-identical object condition that is not met when the two objects are the same object, includes in the target determination group, executes a process to remove one of the two tracking objects from the target determination group. The object tracking device described in Appendix 1. (Note 6) The group evaluation value calculation unit identifies similarity scores that are above a second threshold from among a plurality of similarity scores corresponding to each of the plurality of tracked object pairs in each judgment group, and calculates a group evaluation value for each judgment group by accumulating the identified similarity scores that are above the second threshold. An object tracking device as described in Appendix 1 or 2. (Note 7) The acquisition unit is, An acquisition processing unit acquires multiple tracked object information units, each of which includes a tracking ID, which is a tracking identifier assigned to multiple objects considered identical across multiple image frames and assigned to multiple objects considered different across multiple image frames; position information relating to the position of the object image corresponding to each tracked object information unit in the image frame; and feature information relating to the object features in the object image corresponding to each tracked object information unit. A similarity calculation unit calculates a similarity score between the two trackers included in each of the multiple tracker pairs, where each tracker pair consists of two trackers in the multiple trackers, based on the multiple tracker information units. Equipped with, An object tracking device as described in Appendix 1 or 2. (Note 8) Each tracker information unit includes a feature vector as its feature information. The similarity calculation unit calculates the magnitude of the difference vector between the two feature vectors corresponding to the two trackers included in each tracker pair as the similarity score between the two trackers included in each tracker pair. The object tracking device described in Appendix 7. (Note 9) The aforementioned non-identical object condition includes at least one of the temporal and spatial conditions. An object tracking device as described in Appendix 3 or 5. (Note 10) The aforementioned non-identical object condition is, The first condition is that both of the aforementioned two tracking objects are present in the first image frame in which the first area is captured by the first camera at the first timing, and, The second condition is that, of the two tracked objects, the first tracked object is present in the first image frame in which the first area is captured by the first camera at the first timing, and the second tracked object is present in the second image frame in which the second area, which does not overlap with the first area, is captured by the second camera at the second timing, and the absolute value of the difference between the first timing and the second timing is greater than or equal to zero and less than or equal to a predetermined value. Including at least one of the following, An object tracking device as described in Appendix 3 or 5. (Note 11) An object tracking method performed by an object tracking device, Each tracker pair consists of multiple tracker pairs, each comprising two trackers in multiple trackers. The process involves obtaining similarity scores between the two trackers included in each tracker pair. Based on the multiple similarity scores corresponding to each of the multiple tracked object pairs, the multiple tracked objects are grouped into multiple determination groups, which are groups for determining whether each determination group is a group about a common object. To calculate the group evaluation value for each of the aforementioned multiple judgment groups, Based on the multiple group evaluation values ​​of the multiple judgment groups calculated above, at least one identical object group is determined from among the multiple judgment groups for the same object. A method for tracking objects, including the tracking of objects. (Note 12) The grouping described above includes performing the grouping process on the tracker pairs in order from the pair with the highest similarity score, The object tracking method described in Appendix 11. (Note 13) The aforementioned grouping is, Identifying a pair of trackers consisting of two trackers that satisfy the non-identical object condition not met when the two objects are identical, and whose similarity score is equal to or greater than the first threshold, To create a first determination group, which includes a first tracker included in the identified tracker pair and at least one other tracker, and a second determination group, which includes a second tracker included in the identified tracker pair and the same at least one other tracker as in the first determination group, as part of the plurality of determination groups, including, The object tracking method described in Appendix 11. (Note 14) Each tracker pair consists of multiple tracker pairs, each comprising two trackers in multiple trackers. The process involves obtaining similarity scores between the two trackers included in each tracker pair. Based on the multiple similarity scores corresponding to each of the multiple tracked object pairs, the multiple tracked objects are grouped into multiple determination groups, which are groups for determining whether each determination group is a group about a common object. To calculate the group evaluation value for each of the aforementioned multiple judgment groups, Based on the multiple group evaluation values ​​of the multiple judgment groups calculated above, at least one identical object group is determined from among the multiple judgment groups for the same object. A program that causes an object tracking device to perform a process that includes [specific actions]. (Note 15) The grouping described above includes performing the grouping process on the tracker pairs in order from the pair with the highest similarity score, The program described in Appendix 14. (Note 16) The aforementioned grouping is, Identifying a pair of trackers consisting of two trackers that satisfy the non-identical object condition not met when the two objects are identical, and whose similarity score is equal to or greater than the first threshold, To create a first determination group, which includes a first tracker included in the identified tracker pair and at least one other tracker, and a second determination group, which includes a second tracker included in the identified tracker pair and the same at least one other tracker as in the first determination group, as part of the plurality of determination groups, including, The program described in Appendix 14.

[0142] This application claims priority based on Japanese Patent Application No. 2023-020151, filed on 13 February 2023, and incorporates all of its disclosures herein. [Explanation of Symbols]

[0143] 10 Object Tracking Device 11 Acquisition Department 12 Grouping section 13. Evaluation Value Calculation Unit 14. Identical Object Group Determination Unit 20 Object Tracking Devices 21 Acquisition Department 21A Acquisition Processing Unit 21B Similarity calculation part 22 Grouping section 22A Specific part 22B Judgment group formation unit 23 Evaluation Value Calculation Unit 24. Identical Object Group Determination Unit

Claims

1. An acquisition unit that acquires similarity scores between the two trackers included in each of the multiple tracker pairs, each of which consists of two trackers in multiple trackers, A grouping unit that groups the plurality of tracked object pairs into a plurality of determination groups, which are groups for determining whether each determination group is a group of common objects, based on a plurality of similarity scores corresponding to each of the plurality of tracked object pairs. A group evaluation value calculation unit that calculates a group evaluation value for each of the plurality of judgment groups, Based on the multiple group evaluation values ​​of the multiple judgment groups calculated above, the same object group determination unit determines at least one same object group for the same object among the multiple judgment groups, An object tracking device equipped with the following.

2. The grouping unit performs the grouping process on the tracked object pairs in order from the pair with the highest similarity score. The object tracking device according to claim 1.

3. The grouping unit identifies a pair of trackers consisting of two trackers that satisfy the non-identical object condition not satisfied when the two objects are the same object, and whose similarity score is equal to or greater than a first threshold. It then creates a first determination group as part of the plurality of determination groups, which includes a first tracker included in the identified pair of trackers and at least one other tracker, and a second determination group which includes a second tracker included in the identified pair of trackers and the same at least one other tracker as in the first determination group. The object tracking device according to claim 1.

4. The grouping unit groups the plurality of trackers into the plurality of judgment groups using tracker pairs corresponding to similarity scores that have a value of 2 or higher among the plurality of similarity scores. The object tracking device according to claim 1 or 2.

5. The grouping unit, when it finds that two tracking objects satisfy the non-identical object condition that is not met when the two objects are the same object, executes a process to remove one of the two tracking objects from the target determination group. The object tracking device according to claim 1.

6. The group evaluation value calculation unit identifies similarity scores that are above a second threshold from among a plurality of similarity scores corresponding to each of the plurality of tracked object pairs in each judgment group, and calculates a group evaluation value for each judgment group by accumulating the identified similarity scores that are above the second threshold. The object tracking device according to claim 1 or 2.

7. The acquisition unit is, An acquisition processing unit acquires multiple tracked object information units, each of which includes a tracking ID, which is a tracking identifier assigned to multiple objects considered identical in multiple image frames and assigned to multiple objects considered different in multiple image frames; position information relating to the position of the object image corresponding to each tracked object information unit in the image frame; and feature information relating to the object features in the object image corresponding to each tracked object information unit. A similarity calculation unit calculates a similarity score between the two trackers included in each of the multiple tracker pairs, based on the multiple tracker information units, for each of the multiple tracker pairs, each consisting of two trackers in the multiple trackers. Equipped with, The object tracking device according to claim 1 or 2.

8. Each tracker information unit includes a feature vector as its feature information. The similarity calculation unit calculates the magnitude of the difference vector between the two feature vectors corresponding to the two trackers included in each tracker pair as the similarity score between the two trackers included in each tracker pair. The object tracking device according to claim 7.

9. An object tracking method performed by an object tracking device, Each tracker pair consists of multiple tracker pairs, each comprising two trackers in multiple trackers. The process involves obtaining similarity scores between the two trackers included in each tracker pair. Based on the multiple similarity scores corresponding to each of the multiple tracked object pairs, the multiple tracked objects are grouped into multiple determination groups, which are groups for determining whether each determination group is a group about a common object. To calculate the group evaluation value for each of the aforementioned multiple judgment groups, Based on the multiple group evaluation values ​​of the multiple judgment groups calculated above, at least one identical object group is determined from among the multiple judgment groups for the same object. A method for tracking objects, including the tracking of objects.

10. Each tracker pair consists of multiple tracker pairs, each comprising two trackers in multiple trackers. The process involves obtaining similarity scores between the two trackers included in each tracker pair. Based on the multiple similarity scores corresponding to each of the multiple tracked object pairs, the multiple tracked objects are grouped into multiple determination groups, which are groups for determining whether each determination group is a group about a common object. To calculate the group evaluation value for each of the aforementioned multiple judgment groups, Based on the multiple group evaluation values ​​of the multiple judgment groups calculated above, at least one identical object group is determined from among the multiple judgment groups for the same object. A program that causes an object tracking device to perform a process that includes [specific actions].

Images