Moving object tracking system

Abstract

To track an identical moving object appearing in video data acquired by a plurality of cameras arranged over a wide area using the video data.SOLUTION: A tracking system for a moving object includes a graph. In the graph, a node representing a single camera and a node representing a common tracking ID assigned to a moving object appearing in video data acquired by the single camera are connected via an edge. In the graph, nodes representing respective single cameras and having a relationship between at least two single cameras are connected via an edge representing the relationship. In the graph, nodes representing at least two common tracking IDs and recognized as an identical moving object are connected via an edge representing that at least two moving objects appearing in each video data captured by the at least two single cameras are the identical moving objects.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to a system that uses video data acquired by multiple cameras to track a moving object captured in the video data. [Background technology]

[0002] Japanese Patent Application Laid-Open Publication No. 2017-021753 discloses a technique for using video data acquired by multiple cameras to graph images of people captured in the video data. In this conventional technique, frames in which people are detected are extracted from frames (images) included in the video data. Next, a rectangular area (i.e., a bounding box) containing the detected people is trimmed from the extracted frames. The extraction of frames in which people are detected and the trimming of the bounding box are performed on multiple frames that differ in at least one of the time and location of capture. As a result, multiple bounding boxes are extracted.

[0003] In the technology described in the above publication, an image sequence consisting of five bounding boxes is then extracted from the extracted bounding boxes by referring to the Euclidean distance between the cameras. Then, the similarity of these person images included in the image sequence is calculated based on the feature quantities of the person images. This similarity is used to determine whether the person images included in the five bounding boxes represent the same person.

[0004] In the technology described in the above publication, when it is determined that the person images contained in five bounding boxes are of the same person, a graph is generated for the five bounding boxes. This graph is expressed using nodes (vertices, nodal points) and edges (edges, branches) in graph theory. Figures 7-9 of the above publication show an example of a graph in which five bounding boxes (person images) determined to be of the same person through data processing of video data acquired by multiple cameras installed in different locations are treated as nodes, and these bounding boxes are connected by edges.

[0005] Japanese Patent Application Laid-Open Publication No. 2022-086650 discloses a technology for dividing surveillance camera video data into predetermined time intervals and generating a graph from the video data for each divided time interval. In this conventional technology, the graph includes a node graph and an edge graph. The node graph includes a unique node ID assigned to each monitored element captured in the surveillance camera video data, and tracking information and attribute information for the element linked to the node ID. The edge graph includes information on the interaction behavior of the monitored elements when the monitored elements include multiple people, and information on the relationship between the monitored elements when the monitored elements include people and objects.

[0006] In the technology described in the above publication, the generated graphs are stored in databases. The node graph is stored in a node database, and the edge graph is stored in an edge database. In other words, the generated node graph and edge graph are stored in separate databases. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Application Publication No. 2017-021753 [Patent Document 2] Japanese Patent Publication No. 2022-086650 Summary of the Invention [Problem to be solved by the invention]

[0008] Consider tracking moving objects (people, robots, vehicles, etc.) captured in video data acquired by multiple cameras. According to the technology described in Japanese Patent Laid-Open Publication No. 2017-021753, a graph is generated for the same person captured in the video data. Therefore, using this graph, it may be possible to track the same moving object captured in the video data. However, since the technology in this publication extracts image sequences based on Euclidean distance, this extraction is not performed between cameras with large Euclidean distances. Therefore, the graph generated by the technology in this publication is not suitable for tracking moving objects moving over a wide area.

[0009] Furthermore, JP 2022-086650 A describes generating a graph for a single surveillance camera, but does not describe generating a graph for multiple surveillance cameras. Even if graphs were generated for multiple surveillance cameras, the relationships between these graphs would likely not be defined, and the node and edge graphs generated for each surveillance camera would likely be stored in separate databases. Therefore, the technology in this publication cannot track the same moving object captured in video data acquired by multiple cameras.

[0010] One object of the present disclosure is to provide a technology that can track the same moving object captured in video data acquired by multiple cameras placed over a wide area. [Means for solving the problem]

[0011] The present disclosure is a moving object tracking system having the following features. The tracking system includes a storage device that stores image data captured by at least two cameras, and a processor that processes the image data. In the data processing, the processor generates a graph consisting of at least two nodes and at least one edge indicating the relationship between the at least two nodes, and stores the graph in a database in the storage device. In the graph, a node representing a single camera included in the at least two cameras is connected via at least one edge to a node representing a tracking identification number (tracking ID) assigned to a moving object captured in video data acquired by the single camera. The tracking identification number includes a common tracking identification number (common tracking ID) assigned to the same moving object captured in video data acquired by the single camera. In the graph, at least one edge indicating a relationship between at least two of the single cameras connects nodes representing these single cameras that have the relationship. In the graph, nodes that represent the at least two common tracking identification numbers and are recognized as the same moving object are connected via at least one edge that indicates that the at least two moving objects captured in each of the video data acquired by the at least two single cameras are the same moving object. [Effects of the Invention]

[0012] According to the present disclosure, a graph consisting of at least two nodes and at least one edge indicating the relationship between the at least two nodes is generated and stored in a database in a storage device. In this graph, a node representing a single camera is connected via at least one edge to a node representing a tracking identification number assigned to a moving object captured in video data acquired by the single camera. Furthermore, this tracking identification number includes a common tracking identification number assigned to the same moving object captured in video data acquired by the single camera.

[0013] In this graph, nodes representing at least two single cameras that have a relationship are connected to each other via at least one edge indicating the relationship between these single cameras. Furthermore, in this graph, nodes representing at least two common tracking identification numbers that are recognized as the same moving object are connected to each other via at least one edge indicating that at least two moving objects captured in each of the video data acquired by the at least two single cameras are the same moving object.

[0014] Therefore, by using such a graph, it becomes possible to track the same moving object captured in the video data obtained from at least two single cameras. This effect is expected to be obtained even when multiple cameras are placed over a wide area. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a diagram illustrating an example of the overall configuration of a tracking system according to an embodiment; [Figure 2] 10A and 10B are diagrams illustrating a person detection process and a person re-identification process performed by a graph generation processing unit. [Figure 3] FIG. 2 is a diagram illustrating an example of a basic configuration of a graph generated by a graph generation processing unit. [Figure 4] FIG. 10 is a diagram illustrating an example of a detailed configuration of a graph generated by a graph generation processing unit. [Figure 5] 10A and 10B are diagrams illustrating an example of a verification process performed by a graph verification processing unit. [Figure 6] 10A and 10B are diagrams illustrating an example of a verification process performed by a graph verification processing unit. [Figure 7] 10A and 10B are diagrams illustrating an example of a verification process performed by a graph verification processing unit. [Figure 8] 10A and 10B are diagrams illustrating an example of a verification process performed by a graph verification processing unit. [Figure 9]10A and 10B are diagrams illustrating an example of a tracking process performed by a tracking processing unit. [Figure 10] 10A and 10B are diagrams illustrating an example of a tracking process performed by a tracking processing unit. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In each drawing, the same or corresponding parts are denoted by the same reference numerals, and the description thereof will be simplified or omitted.

[0017] 1. Overall configuration example FIG. 1 is a diagram illustrating an example of the overall configuration of a tracking system (hereinafter also simply referred to as "system") according to an embodiment. The system according to the embodiment is a system for tracking a moving object moving within a city CT. There is no limitation on the size of the city CT in the present disclosure. A so-called smart city is an example of a large-scale city CT, an underground mall is an example of a medium-scale city CT, and a large building is an example of a small-scale city CT. Examples of moving objects include people, robots, and vehicles. In the embodiment, the moving object is assumed to be a person (pedestrian PD). In FIG. 1, pedestrians PD1-PD3 are depicted as examples of pedestrians PD.

[0018] The system according to the embodiment includes at least two cameras arranged in the city CT. In FIG. 1, cameras CA1-CA6 are depicted as an example of the at least two cameras. Camera CA1 acquires video data VD_CA1. Like camera CA1, cameras CA2-CA6 acquire video data VD_CA2-VD_CA6, respectively. Video data VD_CAn of any one camera CAn (n is a natural number) arranged in the city CT is transmitted to the management server 10 via a communication network. The communication network is not particularly limited, and wired and wireless networks can be used.

[0019] The management server 10 is a computer including at least one processor 11, at least one storage device 12, and at least one interface 13. The processor 11 performs various types of data processing. The processor 11 includes a CPU (Central Processing Unit). The storage device 12 stores various types of data required for data processing. Examples of the storage device 12 include an HDD, an SSD, a volatile memory, and a non-volatile memory. The interface 13 receives various types of data from the outside and outputs various types of data to the outside. The various types of data received from the outside by the interface 13 include video data VD_CAn. This video data VD_CAn is stored in the storage device 12. A graph DB (database) 17 is formed in the storage device 12. The graph DB 17 may be formed in an external device that can communicate with the management server 10.

[0020] 2. Management Server Configuration Example Fig. 1 shows an example of the functional configuration of the management server 10. In the example shown in Fig. 1, the management server 10 includes a graph generation processing unit 14, a graph verification processing unit 15, and a tracking processing unit 16. These functions are realized by the processor 11 executing various programs stored in the storage device 12.

[0021] 2-1.Graph generation processing section The graph generation processing unit 14 performs processing to generate a graph GPH based on the video data VD_CAn. To generate the graph GPH, the graph generation processing unit 14 performs person detection and extraction processing and person re-identification processing. Figure 2 is a diagram explaining the person detection processing and person re-identification processing performed by the graph generation processing unit 14.

[0022] 2 shows video data VD_CAi from camera CAi and video data VD_CAj from camera CAj as an example of video data VD_CAn. The video data VD_CAi and VD_CAj are each separated by a predetermined time interval, and FIG. 2 shows a set of frames FR in a section time bt. Each frame FR of the video data VD_CAi is, for example, CAi As with each frame FR of the video data VD_CAi, each frame FR of the video data VD_CAj also includes the ID of the camera CAj. CAj and timestamp ts.

[0023] In the detection and extraction process, first, a frame FR in which a person is detected is extracted. Next, the detected person is extracted from this extracted frame. In the example shown in FIG. 2, pedestrians PDx and PDy are detected in each frame within section time bt1 (timestamp ts1-ts5) of video data VD_CAi. Furthermore, pedestrian PDz is detected in each frame within section time bt2 (timestamp ts6-ts10) of video data VD_CAj. Bounding boxes surrounding these pedestrians PD are added to the positions where pedestrians PDx, PDy, and PDz are detected. By trimming this bounding box, an image IM of pedestrians PDx, PDy, and PDz is obtained. PDx , I.M. PDy and IM PDz The image IM of the pedestrian PD is extracted. PDx and IM PDy For example, the ID of the camera CAi CAi The data of the timestamp ts and the coordinate CD in the frame FR of the extracted image PD and data of

[0024] In the re-identification process, each image IM extracted in the detection and extraction process is PDFrom each image, a feature for re-identification processing (hereinafter also referred to as "Re-ID feature") is extracted. The Re-ID feature is extracted using a Re-ID model based on machine learning. Note that the technique for extracting Re-ID feature using the Re-ID model is well known, and the method is not particularly limited. PD Once the Re-ID features are extracted, it is determined whether the people included in the image sequences are the same person by comparing the Re-ID features.

[0025] In the example shown in FIG. 2, each image IM PD Since the Re-ID feature values ​​of the people extracted from the image sequence in the section time bt1 are similar to each other, the pedestrians PDx and PDy included in the image sequence in the section time bt1 are determined to be the same person. PD Since the Re-ID features of the people extracted from are similar to each other, the pedestrian PDz included in the image sequence in the interval time bt2 is determined to be the same person.

[0026] When the detection process is performed, a tracking ID is assigned to the person captured in the video data VD_CAn. PD Among these, if a person is determined to be the same person through the re-identification process, a tracking ID common to this person is assigned. PD (hereinafter referred to as "Common Tracking ID PD ") will be assigned a common tracking ID. PD is also called a universally unique ID (UUID). In the example shown in FIG. 2, for pedestrians PDx, PDy, and PDz, a common tracking ID PDx(bt1) , ID PDy(bt1) and ID PDz(bt2) Each common tracking ID is assigned. PD is a combination of data for a section time bt and data representative of the Re-ID feature data extracted from the image sequence for this section time bt. Note that there are no particular limitations on the example of selecting the representative data for the Re-ID feature, and any method can be used.

[0027] Common Tracking IDPD is generated for each interval time bt. Therefore, if the same pedestrian PD continues to be captured by one camera (single camera), the common tracking ID assigned to this pedestrian PD PD can occur separately for as many interval times bt as there are interval times bt. Therefore, in the re-identification process, the Re-ID feature values ​​may be compared between multiple image sequences that differ in the interval time bt. For example, the Re-ID feature values ​​are compared between two image sequences of interval times bt that have close timestamps ts. If the Re-ID feature values ​​are similar between multiple image sequences, it is determined that the pedestrians PD included in these image sequences are the same person, and the common tracking IDs that were assigned separately to each pedestrian PD are used. PD may be combined into one.

[0028] The graph generation processing unit 14 uses the common tracking ID assigned to the pedestrian PD by the above-described re-identification process. PD and the IDs of at least two cameras placed in the city CT. The generated graph GRH is stored in the graph DB 17. As already explained, the graph GRH is expressed using nodes (vertices, nodal points) and edges (sides, branches) in graph theory. FIG. 3 is a diagram showing an example of the basic configuration of the graph GRH generated by the graph generation processing unit 14. In FIG. 3, a node N_ID representing each ID of the cameras CA1 to CA6 (however, CA2 is omitted) shown in FIG. 1 is shown. CA1 -N_ID CA6 and an edge E indicating the relationship between these nodes.

[0029] Here, the installation position of the camera CA1 is close to that of the camera CA3. Therefore, there is a relationship between these cameras. Therefore, in the graph GRH1 shown in Figure 3, there is a node N_ID representing the ID of the camera CA1. CA1 and a node N_ID representing the ID of camera CA3. CA3are connected via edge E_CA1-3. The meaning of this edge E_CA1-3 is "NEARBY." The "NEARBY" relationship also exists between camera CA1 and camera CA4, between camera CA4 and camera CA5, and between camera CA5 and camera CA6. Therefore, the nodes N representing the IDs of two related cameras CA are connected by one edge E (edge ​​E_CA1-4, edge E_CA4-5, and edge E_CA5-6).

[0030] Another example of the relationship between two cameras CA is that the imaging ranges of these cameras overlap in part or in whole. Here, part of the imaging range of camera CA3 overlaps with that of camera CA5. Therefore, in the graph GRH1 shown in FIG. 3, the node N_ID representing the ID of camera CA3 CA3 and a node N_ID representing the ID of the camera CA5. CA5 are connected via an edge E_CA3-5. The meaning of this edge E_CA3-5 is "OVERLAPPED."

[0031] A common tracking ID assigned to pedestrian PDs PD Represents the node N_ID PD This common tracking ID PD Node N_ID representing the camera CAn that acquired the video data VD_CAn that was the basis for the assignment CAn PDp, PDq, PDr, PDs, and PDu are connected to each other via at least one edge E. Figure 3 shows the common tracking IDs assigned to pedestrians PDp, PDq, PDr, PDs, and PDu. PD Represents the node N_ID PDp , N_ID PDq , N_ID PDr , N_ID PDs and N_ID PDu is drawn. Node N_ID PDp connects to node N_ID via edge E_CA1 CA1 Node N_ID PDq and N_ID PDr connects to node N_ID via two edges E_CA4 CA4Node N_ID PDs connects to node N_ID via edge E_CA5 CA5 Node N_ID PDu connects to node N_ID via edge E_CA3 CA3 is tied to.

[0032] As mentioned above, the common tracking ID PD The re-identification process combines data from the interval time bt and representative data of the Re-ID features from the image sequences. In the embodiment, the representative data of the Re-ID features is used to perform a re-identification process for the person captured by at least two cameras. This re-identification process is the same as the comparison of the Re-ID features between multiple different image sequences in the interval time bt. However, while the comparison of the Re-ID features between multiple image sequences targets one camera, the comparison of the representative data of the Re-ID features targets two cameras. If the representative data of the Re-ID features between the two cameras is similar, the pedestrian PD captured separately by these cameras is determined to be the same person.

[0033] When it is determined that the pedestrians PD captured by the two cameras are the same person, the graph generation processing unit 14 uses the common tracking IDs assigned to the pedestrians PD separately. PD Represents the node N_ID PD are connected via edge E. In the graph GRH1 shown in FIG. 3, pedestrians PDp and PDq are determined to be the same person, and pedestrians PDr and PDs are determined to be the same person. Therefore, the node N_ID PDp and node N_ID PDq are connected via edges E_IDp-q, and node N_ID PDr and node N_ID PDs are connected via edge E_IDr-s. The meaning of edges E_IDp-q and E_IDr-s is "SAME PERSON."

[0034] In the graph GRH1 shown in Figure 3, it is also determined that pedestrians PDq and PDr are the same person. This determination is based on the result of comparing Re-ID features between multiple image sequences that are different in the interval time bt, performed using a single camera. For this reason, the node N_ID PDq and node N_ID PDr are connected via an edge E_IDq-r which means "SAME PERSON".

[0035] 4 is a diagram showing a detailed configuration example of a graph generated by the graph generation processing unit 14. In the graph GRH2 shown in FIG. 4, a node N representing additional information ADD is added to the graph GRH1 shown in FIG. 3. This additional information ADD includes a common tracking ID PD The additional information ADD is the image IM of the pedestrian PD. PD , Pedestrian PD appearance characteristics AP PD , Pedestrian PD behavior AC PD , pedestrian PD's face image IMF PD Examples include:

[0036] Pedestrian PD Image IM PD are used to extract Re-ID features. Examples of appearance features of pedestrian PD include the color, clothing, and body shape of the pedestrian PD. These appearance features are estimated using a pre-trained appearance model. Examples of pedestrian PD behavior include "walking," as well as "carrying" and "opening" actions that pedestrian PD take with respect to stationary objects such as luggage. These actions are estimated using a pre-trained behavior model. These actions also include interaction actions such as "conversation" and "handing over" that multiple people take together. The facial image IMF of pedestrian PD PD is a pedestrian PD image IM PD The face image may be one in which the face portion has been trimmed from the original image, or may be a face image provided from outside in the tracking process (described later) by the tracking processing unit 16.

[0037] The graph GRH2 shown in Figure 4 has a node N_IDPDp Three edges E_ID extend from PDp Ahead, an image of a pedestrian PDp is displayed. PDp Node N_IM representing PDp(bt1) and N_IM PDp(bt2) and node N_AC representing the behavior of pedestrian PDp. PDp(bt1) Also, the node N_ID PDq Three edges E_ID extend from PDq Next, the face image IMF of pedestrian PDq PDq Node N_IMF representing PDq and the image IM of the pedestrian PDq PDq Node N_IM representing PDq(bt3) and node N_AC representing the behavior of pedestrian PDq. PDq(bt3) and are arranged.

[0038] Although detailed explanation is omitted, in the graph GRH2, the node N_ID PDr , N_ID PDs and N_ID PDu For example, the node N representing the additional information ADD for each pedestrian PD is connected to the node N_ID via the edge E. PD The node N_ID is connected to the node PDs and node N_ID PDu is the edge E_IA, which means the interaction action "TALKING". PDs-PDu Also, the node N_ID PDs Edge E_ID PDs Node N_AP connected via PDs(bt6) is the appearance feature AP of pedestrian PDs in section time bt6. PDs Represents.

[0039] 2-2.Graph verification processing section The graph verification processing unit 15 verifies the judgment made by the graph generation processing unit 14, specifically, the judgment made using the representative data of the Re-ID feature quantities. Judgment using the representative data of the Re-ID feature quantities provides a certain degree of certainty that the pedestrian PD captured separately by the two cameras is the same person. However, when the relative positions of the two cameras and the temporal or spatial positional relationship of the pedestrian PD judged to be the same person are taken into consideration, a contradiction may occur in the judgment results. The graph verification processing unit 15 detects this contradiction. If a contradiction is judged to exist, the edge E that means "SAME PERSON" is deleted. In other words, the two nodes N_ID based on the representative data of the Re-ID feature quantities are PD The binding is released.

[0040] FIG. 5-8 is a diagram illustrating an example of the verification process performed by the graph verification processing unit 15. The verification process is performed using the graph GRH. Each of the graphs GRH3-6 shown in FIG. 5-8 has a node N_ID representing each ID of at least two cameras. CA and an edge E indicating the relationship between these nodes. Also, at least two nodes N_ID CA Includes a common tracking ID PD Represents the node N_ID PD are connected to each other via edge E.

[0041] In the graph GRH3 shown in Figure 5, the node N_ID CA1 Node N_ID connected to PDp connects to node N_ID via edge E_CA4 CA4 Node N_ID connected to PDq and are connected via an edge E_IDp-q, which means "SAME PERSON." PDp also connects to node N_ID via edge E_CA3 CA3 Node N_ID connected to PDv are connected via an edge E_IDp-v which means "SAME PERSON".

[0042] However, when the relative positions of cameras CA1, CA3, and CA4 and the predicted movement direction of pedestrian PDp based on the section time bt (the direction of the arrows of edges E_IDp-q and E_IDp-v) are taken into consideration, the simultaneous existence of edges E_IDp-q and E_IDp-v is inconsistent. This is because graph GRH3 has connections that make it seem as if one pedestrian PDp has split into two pedestrians PDq and PDv. Therefore, in this case, edges E_IDp-q and E_IDp-v are deleted. In a more advanced example, node N_ID representing pedestrian PD who is determined to be the same person as pedestrian PDp PD In this case, the inappropriate edge E can be deleted and the appropriate edge E can be left.

[0043] The structure of graph GRH4 shown in FIG. 6 is basically the same as that of graph GRH3 shown in FIG. 5. The difference between these graphs is the predicted movement direction of pedestrian PDp based on the section time bt (the direction of the arrows of edges E_IDp-q and E_IDp-v). However, the verification results are the same. This is because graph GRH3 has connections that make it seem as if two pedestrians PDq and PDv were merged into one pedestrian PDp. Therefore, in this case as well, edges E_IDp-q and E_IDp-v are deleted.

[0044] In the graph GRH5 shown in Figure 7, the node N_ID CA4 Node N_ID connected to PDr connects to node N_ID via edge E_CA5 CA5 Node N_ID connected to PDs and are connected via edge E_IDr-s, which means "SAME PERSON". PDr Also, the node N_ID PDv and are connected via an edge E_IDr-v, which means "SAME PERSON".

[0045] Unlike the results of the verification explained with reference to Figures 5 and 6, the result of the verification based on the graph GRH5 shown in Figure 7 is "consistent". This is because the node N_ID CA5 and node N_ID CA3 are connected via edge E_CA3-5, which means "OVERLAPPED." Therefore, it is as if one pedestrian PDr has been split into two pedestrians, PDs and PDv, but the verification result shows that there is no contradiction in that pedestrians PDs and PDv are the same person.

[0046] In the graph GRH6 shown in Figure 8, the node N_ID PDp(bt1) and node N_ID PDq(bt3) In the graph GRH6, the nodes N_ID and N_IDp-q are connected via an edge E_IDp-q, which means "SAME PERSON". CA3 Node N_ID connected to PDu(bt7) and node N_ID via edge E_CA7 CA7 Node N_ID connected to PDw(bt8) are connected via an edge E_IDu-w that means "SAME PERSON."

[0047] In graph GRH6, furthermore, node N_ID PDq(bt3) and node N_ID PDw(bt8) However, there is no relationship between the positions of the cameras CA4 and CA7 (i.e., the node N_ID CA4 and node N_ID CA7 Considering the fact that there is no edge E between cameras CA4 and CA7 and the distance between cameras CA4 and CA7, it is determined that there is a contradiction in the existence of edge E_IDu-w. In this case, edge E_IDu-w is deleted.

[0048] 2-3. Tracking processing section The tracking processing unit 16 tracks the tracking target using the graph GPH stored in the graph DB 17. Figures 9 and 10 are diagrams for explaining an example of the tracking process performed by the tracking processing unit 16. In the example shown in Figure 9, an image IM of the tracking target is TGT The graph DB 17 is referenced using the query Q (input information). TGT is selected from the frame of the pedestrian PD used to extract the Re-ID features. TGT is the face image IMF of the tracked object. TGT In this case, the facial image IMF TGT may be trimmed from the frame of the pedestrian PD used to extract the Re-ID features, or may be provided from outside the tracking system.

[0049] In the example shown in FIG. TGT and its image IM PD The tracking data TRC of the pedestrian PD that matches the image IM is output as a search result R. TGT Matching images IM PD Node N_IM representing PD and each node N_IM PD Node N_ID connected to via edge E PD and each node N_ID PD Node N_ID connected to via edge E CA In another example, each node N_IM PD Additional information ADD (i.e., appearance feature AP) is connected to the PD , behavioral AC PD A set of nodes N representing the nodes 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32,

[0050] In the example shown in FIG. 10, the date and time DT, the location AR, and the image IM TGT In this example, the graph DB 17 is referenced using the query Q (input information). TGT and its image IM PDThe tracking data TRC of the pedestrian PD that matches the query Q is output as the search result R. However, in the example shown in FIG. 10, the date and time DT and the location AR are included in the query Q, so the tracking data TRC with limited date and location is output.

[0051] 3.Effects According to the embodiment described above, two common tracking IDs are generated based on the video data VD_CA obtained from at least two cameras. PD Represents the node N_ID PD A graph GRH is generated in which the images are connected by edges E. Therefore, by using this graph GRH for tracking processing, it becomes possible to track the same person (same moving object) that appears in these video data VD_CA. This effect is also expected to be obtained when multiple cameras are installed over a wide area.

[0052] Also, according to an embodiment, two common tracking IDs PD Represents the node N_ID PD The connections are verified. If the results of the verification performed during the connection are found to be inconsistent, the connection is removed. This verification is expected to help ensure the reliability of the graph GRH.

[0053] Furthermore, the generation of the graph GRH and the verification of the linkage can be performed in parallel with the acquisition of the video data VD_CA, which makes it possible to track the same person who appears in the video data VD_CA in real time with high accuracy. [Explanation of symbols]

[0054] 10 Management server, 11 Processor, 12 Storage device, 14 Graph generation processing unit, 15 Graph verification processing unit, 16 Tracking processing unit, 17 Graph DB, CA, CA1-CA6, CAi, CAj, CAn Camera, FR Frame, PD, PDp-PDs, PDu-PDz Pedestrian, VD, VD_CA1-CA6, VD_CAn Video data, AC PD behavior, AP PDAppearance features, IM PD ,IM PDp ,IM PDq ,IM PDx -IM PDz ,IM TGT Image, IMF PD ,IMF PDq ,IMF TGT Face image, E,E_CA1,E_CA3-CA7,E_CA1-3,E_CA1-4,E_CA3-5,E_CA4-5,E_CA5-6,E_CA3-7,E_ID PDp -E_ID PDs ,E_ID PDu ,E_IDp-q,E_IDq-r,E_IDr-s,E_IDq-w,E_IDu-w,E_IA PDs - PDu Edge, N, N_AC PDp -N_AP PDs ,N_ID CA ,N_ID CA1 -N_ID CA7 ,N_ID PD ,N_ID PDp -N_ID PDs ,N_ID PDu -N_ID PDw ,N_IMF PDq Node, ADD additional information, GRH, GRH1-GRH6 graph

Claims

1. a storage device in which video data captured by at least two cameras is stored; a processor configured to perform data processing based on each of the video data; Equipped with The processor, in the data processing, Generate a graph consisting of at least two nodes and at least one edge indicating a relationship between the at least two nodes; storing the graph in a database in the storage device; In the graph, a node representing a single camera included in the at least two cameras and a node representing a tracking identification number assigned to a moving object captured in video data acquired by the single camera are connected via at least one edge; the tracking identification number includes a common tracking identification number assigned to the same moving object captured in the video data acquired by the single camera, At least one edge indicating a relationship between the at least two single cameras connects nodes having the relationship among the nodes representing these single cameras, and At least two nodes representing the common tracking identification numbers that are recognized as the same moving object are connected to each other via at least one edge that indicates that at least two moving objects captured in each of the image data acquired by the at least two single cameras are the same moving object. A moving object tracking system comprising:

2. In the graph, a node representing the common tracking identification number and a node representing additional information about the same moving object to which the common tracking identification number is assigned are connected via at least one edge; The additional information includes at least one of an image of the same moving object to which the common tracking identification number is assigned, an appearance feature of the same moving object, a behavior of the same moving object, and, if the same moving object is a person, a face image of the person.

2. The moving object tracking system according to claim 1.

3. The processor, in generating the graph, determining whether the at least two moving objects are the same moving object based on the feature amounts of the at least two moving objects; If it is determined that the at least two moving objects are the same moving object, linking the common tracking identification numbers assigned to these moving objects, respectively; When the common tracking identification numbers assigned to the at least two moving objects are linked, the nodes representing the common tracking identification numbers are connected to each other in the graph via at least one edge indicating that the at least two moving objects are the same moving object.

3. A moving object tracking system according to claim 1 or 2.

4. The processor, in generating the graph, further verifying a determination based on each feature amount of the at least two moving objects; If it is determined that there is a contradiction in the determination, the association between the nodes representing the common tracking identification numbers assigned to the at least two moving objects is released.

4. The moving object tracking system according to claim 3.

5. The processor further comprises: The tracking process of the tracking target is performed by referring to the graph using a query as an input, The query includes at least one of a date and time, a location, an image of the tracked object, and, if the tracked object is a person, a face image of the person.

3. A moving object tracking system according to claim 1 or 2.

Images