Object matching system, object matching method, and object matching program

Abstract

An input means 81 receives an input of a plurality of images in which a target object that is a kind of the object to be collated has been imaged. An object detection means 82 detects the target object from the images. An execution instruction means 83 transmits, to a quantum computer that executes minimum maximal matching, first node information that indicates the detected position of the target object, and second node information that indicates the position of the target object, which is predicted on the basis of the past collation result, and the similarity between the position of the first node and the position of the second node, and determines the first node corresponding to the second node by executing collation processing between the past collation result and the target object detected from the images.

Description

Technical Field

[0001] The present invention relates to an object matching system, an object matching method, and an object matching program for matching objects existing in a plurality of images.

Background Art

[0002] Generally, tracking a number of objects such as vehicles and people from an image captured by a single camera, or tracking objects from images captured by a plurality of cameras is performed. Various methods for solving such object matching by maximum matching or 0-1 integer programming problems have been proposed.

[0003] For example, Non-Patent Document 1 describes a method for tracking a plurality of objects. Specifically, Non-Patent Document 1 describes a framework (SORT: Simple online and realtime tracking) that performs Kalman filtering in the image space and data association for each frame by using an association metric that measures the overlap of bounding boxes by the Hungarian method.

[0004] Note that Non-Patent Document 2 describes a method for calculating the Wasserstein distance of persistent diagrams by a quantum computer.

Prior Art Documents

Non-Patent Documents

[0005]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] On the other hand, since the amount of calculation when performing object matching is polynomial time, it is also known that the calculation time increases as the number of objects to be detected increases. For example, when solving the maximum matching of an n×n bipartite graph by the Hungarian method described in Non-Patent Document 1, the calculation cost is O(n 3 ) is known. In general, there is a trade-off relationship between calculation speed and accuracy.

[0007] Also, it is possible to perform online and real-time tracking by SORT described in Non-Patent Document 1. However, in so-called Tracking by Detection (a method of tracking based on the result of object detection) as described in Non-Patent Document 1, there is a problem that the tracking accuracy also decreases when the detection accuracy decreases.

[0008] Here, in order to suppress the increase in calculation time, it is conceivable as an idea to use a quantum computer. However, for example, the solution obtained by quantum annealing is unstable and not necessarily always the optimal solution. Therefore, it is difficult to say that the accuracy can be improved simply by applying a quantum computer. Therefore, it is preferable to be able to perform object matching that can maintain the accuracy while improving the calculation speed.

[0009] Therefore, an object of the present invention is to provide an object matching system, an object matching method, and an object matching program that can perform object matching while improving the calculation speed and maintaining the accuracy.

Means for Solving the Problem

[0010] The object matching system according to the present invention includes an input means for receiving an input of a plurality of images obtained by imaging a target object, which is an object of a type to be matched; an object detection means for detecting the target object from the images; and, for a quantum computer that executes minimum-maximum matching, information of a first node representing the position of the detected target object, information of a second node representing the position of the target object predicted based on past matching results, and a similarity between the position of the first node and the position of the second node are transmitted to execute a matching process between the past matching results and the target object detected from the images, and an execution instruction means for determining a first node corresponding to the second node.

[0011] The object matching method according to the present invention includes receiving an input of a plurality of images obtained by imaging a target object, which is an object of a type to be matched, detecting the target object from the images, and for a quantum computer that executes minimum-maximum matching, transmitting information of a first node representing the position of the detected target object, information of a second node representing the position of the target object predicted based on past matching results, and a similarity between the position of the first node and the position of the second node to execute a matching process between the past matching results and the target object detected from the images, and determining a first node corresponding to the second node.

[0012] The object matching program according to the present invention causes a computer to execute an input process for receiving input of a plurality of images obtained by imaging a target object, which is an object of a type to be matched, an object detection process for detecting the target object from the images, and a quantum computer that executes minimum-maximum matching. Information on a first node representing the position of the detected target object, information on a second node representing the position of the target object predicted based on past matching results, and the similarity between the position of the first node and the position of the second node are transmitted to execute a matching process between the past matching results and the target object detected from the images, and an execution instruction process for determining a first node corresponding to the second node is executed.

Effect of the Invention

[0013] According to the present invention, object matching can be performed while improving the calculation speed and maintaining the accuracy.

Brief Description of the Drawings

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Embodiments for Carrying Out the Invention

[0015] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0016] Embodiment 1. FIG. 1 is a block diagram showing a configuration example of a first embodiment of the object matching system of the present invention. The object matching system 100 of the present embodiment includes an imaging device 10, an object matching device 20, a quantum computer 30, and a browsing terminal 40. The object matching device 20 is connected to the imaging device 10, the quantum computer 30, and the browsing terminal 40, respectively.

[0017] The quantum computer 30 is a computer that realizes parallel computing by utilizing quantum mechanical phenomena. For example, a quantum computer of the quantum annealing method (hereinafter referred to as a quantum annealing machine) is a dedicated device for obtaining the ground state of the Hamiltonian of the Ising model, and is a device for executing annealing based on the Ising model. More specifically, the quantum annealing machine is a device that probabilistically obtains the values of binary variables that minimize or maximize the objective function (i.e., the Hamiltonian) of the Ising model with binary variables as arguments. Note that the binary variables may be realized by classical bits or quantum bits.

[0018] The mode of the quantum computer 30 in the present embodiment is arbitrary. The quantum computer 30 may be configured by any hardware as long as it probabilistically obtains the values of binary variables that minimize or maximize the objective function with binary variables as arguments. The quantum computer 30 may be, for example, a non-Neumann type computer in which the objective function is implemented by hardware in the form of an Ising model. Further, the quantum computer 30 may be a quantum annealing machine or a general annealing machine.

[0019] The imaging device 10 is installed at a predetermined position and periodically images objects of a type to be collated (hereinafter referred to as target objects). Then, the imaging device 10 transmits the captured image to the object collation device 20. The imaging device 10 may transmit, for example, an image of each frame captured as video to the object collation device 20.

[0020] In addition, in this embodiment, the case where one imaging device 10 is installed will be described. The imaging device 10 may be connected to a switching hub having, for example, a PoE (Power over Ethernet) function and be powered from a LAN (Local Area Network) cable.

[0021] In this embodiment, a vehicle will be mainly described as an example of a target object. FIG. 2 is an explanatory diagram showing an example of a situation in which the object collation system 100 of this embodiment is used. In FIG. 2, the imaging device 10 is installed on a road where a plurality of vehicles pass, and an example of a situation in which a plurality of vehicles as target objects are imaged is illustrated. Then, the object collation system 100 collates the vehicle in the image T1 captured at time t with the vehicle in the image T2 captured at time t + 1 to determine whether they are the same vehicle.

[0022] However, the target object is not limited to a vehicle as long as it is a moving object, and may be, for example, a human. Also, as long as it is an object that can be imaged as an image, the target object may be an object such as a particle that cannot be seen by the human eye. When the target object is a particle, for example, it becomes possible to infer the trajectory of the particle in radiation therapy or an experiment using an accelerator.

[0023] The object collation device 20 of this embodiment is a device that discriminates and collates the same target object between frames at different times of a moving image captured by one imaging device 10. The object collation device 20 includes a storage unit 21, an input unit 22, an object detection unit 23, an execution instruction unit 24, and an output unit 25.

[0024] The storage unit 21 stores various information used by the object matching device 20 for processing. The storage unit 21 may store, for example, the input image. The storage unit 21 is realized by, for example, a magnetic disk or the like.

[0025] The input unit 22 receives the input of a plurality of images obtained by imaging the target object from the imaging device 10. The input unit 22 may store the received images in the storage unit 21.

[0026] The object detection unit 23 detects the target object from the image. Note that the method by which the object detection unit 23 detects the target object from the image is arbitrary, and a known method such as YOLOv5 may be used. The object detection unit 23 may output, as detection results, the coordinates of the upper left vertex and the lower right vertex of, for example, a rectangle (that is, a bounding box) surrounding the detected target object in the image.

[0027] The execution instruction unit 24 issues an instruction to cause the quantum computer 30 to execute the matching process of the target object. Here, as one of the matching problems, there is the minimum-maximum matching problem. Since the minimum-maximum matching can be represented by an Ising model, it can be formulated as an optimization problem that can be solved by quantum annealing. Therefore, the quantum computer 30 of the present embodiment is assumed to operate as a quantum computer that executes the minimum-maximum matching.

[0028] Hereinafter, an example of a method for representing the minimum-maximum matching problem by an Ising model will be described. Here, it is assumed that the minimum variable maximum matching is performed when an undirected graph G = (V, E) is given. For each edge e ∈ E, a binary variable x representing whether or not the edge is included in the matching C is defined. e Let it be. Also, for each vertex v, let ∂v be the set of edges having v as an end point. Then, for each vertex, a variable y represented by the following Equation 1 is defined. v is defined.

[0029]

Equation

[0030] X e When the set D of edges that becomes is a matching of the graph, y v becomes a variable that takes 1 when the vertex v is an endpoint of an edge in the matching and 0 otherwise. Also, let A, B, and C be positive constants, and assume that A / Δ - 2 > B > C. Here, Δ is the maximum degree of the graph. At this time, the energy function H is represented by Equation 2 shown below.

[0031]

Equation

[0032] Note that the method of expressing the energy function is just an example, and alternatively, the minimum-maximum matching problem may be formulated by the method described in Non-Patent Document 2.

[0033] The execution instruction unit 24 transmits information necessary to solve the bipartite graph problem (here, the minimum-maximum matching problem) to the quantum computer and causes the verification process to be executed.

[0034] Note that the verification process here can be said to be a process of determining a likely first node corresponding to the second node. As a result of the verification process by the quantum computer, for example, candidates for the first node corresponding to the second node are identified.

[0035] First, the execution instruction unit 24 generates information on the nodes on both sides of the bipartite graph. Specifically, the execution instruction unit 24 generates information representing the position of the target object at time t detected from the image as information on one node (hereinafter referred to as the first node). Also, the execution instruction unit 24 generates information representing the position of the target object at time t predicted based on the past verification results as information on the other node (hereinafter referred to as the second node). Hereinafter, the target object itself may also be referred to as a node.

[0036] Note that the past matching result means the result of identifying the target object by the matching process performed before the current time. The identification here does not necessarily mean identifying each individual target object itself, but rather means the process of associating target objects that are assumed to match between a plurality of images.

[0037] A plurality of first nodes and second nodes are generated for each target object. Note that the information of the first node and the information of the second node do not necessarily include the specific position information (for example, coordinate information) detected by the object detection unit 23, and may include abstract information such that an identification index is assigned to each target object.

[0038] FIG. 3 is an explanatory diagram showing an example of a bipartite graph. The black circles on the upper side of the bipartite graph illustrated in FIG. 3 indicate nodes (that is, first nodes) representing the positions of the target objects detected in the frame at time t. Also, the black circles on the lower side indicate nodes (that is, second nodes) representing the predicted positions of the vehicles in the frame at time t based on the vehicle detection and tracking results (that is, the matching results) up to the frame at time t-1.

[0039] In addition, the similarity between the nodes is associated as a weight with the edge connecting the upper and lower nodes. Examples of the similarity between the nodes include the similarity between the images of the detected target objects and the IoU (Intersection over Union) indicating the overlap between the bounding boxes.

[0040] Note that the method by which the execution instruction unit 24 predicts the position of the target object at time t based on the past matching result is arbitrary. For example, the execution instruction unit 24 may predict the position of the target object at time t by linear regression using, as learning data, the coordinates of the positions where the same target object (specifically, the target object to which the same index is assigned) was detected in the images of the frames traced back in the past from the immediately previous time t-1. At this time, the time corresponds to the explanatory variable and the coordinates correspond to the objective variable.

[0041] In addition, the execution instruction unit 24 of the present embodiment creates a plurality of patterns of the information of the second node according to the length of the past collation results used for prediction. Note that the length of the collation results is arbitrary and may be determined in advance by an administrator or the like. Specifically, the execution instruction unit 24 generates information of the second node in a plurality of patterns, such as information of the second node based on the tracking results in the most recent period (for example, from time t-1 to time t-10), information of the second node based on the tracking results in the medium term (for example, from time t-1 to time t-20), and information of the second node based on the tracking results in the long term (for example, from time t-1 to time t-100). Then, the execution instruction unit 24 transmits the information of the second node in this plurality of patterns to the quantum computer 30 all at once to execute the collation process.

[0042] FIG. 4 is an explanatory diagram showing an example of a process of creating a plurality of patterns of the information of the second node. The information of the second node illustrated in FIG. 4 indicates that a plurality of patterns of bipartite graphs G1 to GN are created for different periods.

[0043] Similar to the bipartite graph illustrated in FIG. 3, the black circles on the upper side of the bipartite graph G1 indicate the nodes representing the positions of the target objects detected in the frame at time t, and the black circles on the lower side indicate the nodes representing the predicted vehicle positions in the frame at time t based on the collation results of the frames from time t-1 to t-10.

[0044] Also, for example, the black circles on the upper side of the bipartite graph G2 indicate the nodes representing the positions of the target objects detected in the frame at time t, and the black circles on the lower side indicate the nodes representing the predicted vehicle positions in the frame at time t based on the collation results of the frames from time t-1 to t-20.

[0045] The execution instruction unit 24 transmits the information of the second node in a plurality of patterns to the quantum computer 30 in a lump and executes the collation process for each piece of information of the second node. As a result, a plurality of collation results are obtained from the information of the second node in each pattern.

[0046] Then, the execution instruction unit 24 determines the first node corresponding to (i.e., matching) the second node by using a plurality of collation results. Specifically, the execution instruction unit 24 may determine to associate nodes that match more closely. For example, the execution instruction unit 24 may determine the first node corresponding to the second node by a majority vote of candidates for the first node corresponding to the second node from the results of the collation process. At this time, the execution instruction unit 24 may assign the index given to the second node as the index of the first node determined to match.

[0047] For example, in a device such as a so-called classical computer, if collation processing (i.e., multiplexing of matching) is performed on information of a plurality of patterns of second nodes, the amount of calculation becomes enormous. Therefore, it is not realistic to perform collation processing on information of a plurality of second nodes all at once. On the other hand, in the present embodiment, the collation processing is executed by the quantum computer 30. Therefore, an increase in calculation cost can be suppressed.

[0048] It is also conceivable to stabilize the solutions obtained by solving the information of one type of second node multiple times. However, for example, when the target object is a vehicle, it is difficult to improve the accuracy no matter how many times the solution is recalculated only with an image (undetected) hidden behind other vehicles or obstacles.

[0049] On the other hand, in the present embodiment, a plurality of patterns of second node information are created according to the length of the past collation results used for prediction. Therefore, for example, even if the solution of the quantum annealing is unstable, it is possible to mitigate the influence by a majority vote. Furthermore, for example, even in a situation where it becomes difficult to accurately predict the position because the target object cannot be detected halfway, it is similarly possible to mitigate the influence.

[0050] The output unit 25 outputs the result of the collation process. In the present embodiment, the output unit 25 transmits the result of the collation process to the browsing terminal 40 for display. Note that the output unit 25 may output the result of the collation process to other devices (not shown) or the like.

[0051] Specifically, the output unit 25 may transmit the detected position information of the target object and the information for identifying the target object (for example, an index) to the browsing terminal 40 in association with each other. The position information at this time may be information converted into coordinates on a plane map illustrated in FIG. 2, for example. Further, the output unit 25 may output not only the position information and the index but also information indicating the attributes of the target object obtained as a result of image recognition (for example, the type of vehicle such as a large vehicle, a small vehicle, a motorcycle, etc.).

[0052] The input unit 22, the object detection unit 23, the execution instruction unit 24, and the output unit 25 are realized by a processor (for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit)) of a computer that operates according to a program (object matching program).

[0053] For example, the program may be stored in the storage unit 21 of the object matching device 20, and the processor may read the program and operate as the input unit 22, the object detection unit 23, the execution instruction unit 24, and the output unit 25 according to the program. Further, the functions of the object matching device 20 may be provided in the form of SaaS (Software as a Service).

[0054] Further, the input unit 22, the object detection unit 23, the execution instruction unit 24, and the output unit 25 may each be realized by dedicated hardware. Further, some or all of the components of each device may be realized by general-purpose or dedicated circuitry, a processor, etc. or a combination thereof. These may be configured by a single chip or by a plurality of chips connected via a bus. Some or all of the components of each device may be realized by a combination of the circuitry, etc. described above and a program.

[0055] In addition, when some or all of the components of the object matching device 20 are realized by a plurality of information processing devices, circuits, etc., the plurality of information processing devices, circuits, etc. may be centrally arranged or may be distributed. For example, the information processing devices, circuits, etc. may be realized in a form in which each is connected via a communication network, such as a client-server system, a cloud computing system, etc.

[0056] The browsing terminal 40 visualizes the matching result based on the information transmitted from the object matching device 20 (more specifically, the output unit 25). More specifically, the browsing terminal 40 visualizes the movement status of the target object. For example, when the target object is a vehicle, the browsing terminal 40 may visualize the traffic flow status.

[0057] FIG. 5 is an explanatory diagram showing an example of visualizing the traffic flow status. In the example shown in FIG. 5, a line is set at an appropriate location (for example, near a crosswalk) on the plane map of the intersection as exemplified in FIG. 2, and the number of vehicles that have crossed the set line is shown in tabular form at a preset time (for example, from 12:00 to 13:00). More specifically, the number of vehicles flowing into the intersection across the line and the number of vehicles flowing out of the intersection across the line are counted.

[0058] Next, the operation of the object matching system 100 of the present embodiment will be described. FIG. 6 is a flowchart showing an operation example of the object matching system 100 of the present embodiment. The input unit 22 receives the input of the image captured by the imaging device 10 (step S11). The object detection unit 23 detects the target object from the image (step S12). The execution instruction unit 24 transmits the information of the first node and the second node, and the similarity of the positions between the nodes to the quantum computer 30 to execute the matching process (step S13), and determines the first node corresponding to the second node (step S14).

[0059] As described above, in this embodiment, the input unit 22 receives the input of a plurality of images obtained by imaging a target object, and the object detection unit 23 detects the target object from the images. Then, the execution instruction unit 24 transmits the information of the first node and the information of the second node, as well as the similarity between the position of the first node and the position of the second node, to a quantum computer that executes the minimum-maximum matching, and causes the quantum computer to execute a matching process to determine the first node corresponding to the second node. Therefore, object matching can be performed while improving the calculation speed and maintaining the accuracy.

[0060] Next, a modified example of the object matching system 100 of this embodiment will be described. In the above embodiment, the matching process was performed by focusing on the entire target object. On the other hand, when the target object can be divided into individual parts (hereinafter referred to as parts), the matching process may be performed on the divided parts.

[0061] That is, the object detection unit 23 may detect individual parts (that is, parts) included in the target object from the image in which the target object is imaged. Note that a known method may be used for this detection process. Then, the execution instruction unit 24 may transmit the information of the first node representing the position of the detected part, the information of the second node representing the position of the part predicted based on the past matching result, and the similarity of each part to the quantum computer 30 to cause the quantum computer to execute a matching process.

[0062] For example, when the target object is a human, parts such as the head, abdomen, arms, and legs that make up the human body are assumed. In this case, the object detection unit 23 detects each part included in the human from the image in which the human is imaged, and transmits the position and similarity of the part predicted based on the past matching result to the quantum computer 30 as in the above embodiment to cause the quantum computer to execute a matching process. This makes it possible to track the movements of the human body.

[0063] Also, for example, when the target object is an organ, parts at the cell level that make up the organ are assumed. In this case, the object detection unit 23 may detect parts at the cell level from a biological image such as an organ image, and transmit the position and similarity of the cells predicted based on the past collation results to the quantum computer 30 and execute the collation process in the same manner as in the above embodiment. This makes it possible to analyze life phenomena such as cell division.

[0064] Embodiment 2. Next, a second embodiment of the object collation system of the present invention will be described. In the first embodiment, the case where one imaging device 10 is installed and the image captured by the imaging device 10 is used has been described. In the second embodiment, a configuration in which a plurality of imaging devices 10 for imaging the target object are installed will be described. Note that the ranges imaged by the imaging devices 10 may or may not overlap.

[0065] FIG. 7 is a block diagram showing a configuration example of a second embodiment of the object collation system of the present invention. The object collation system 200 of the present embodiment includes a plurality of imaging devices 10, an object collation device 50, a quantum computer 30, and a browsing terminal 40. The object collation device 50 is connected to each of the imaging devices 10, the quantum computer 30, and the browsing terminal 40. Since the contents of the imaging device 10, the quantum computer 30, and the browsing terminal 40 are the same as those in the first embodiment, the following description will be omitted.

[0066] FIG. 8 is an explanatory diagram showing an example of a situation in which the object collation system 200 of the present embodiment is used. In FIG. 8, a situation is exemplified in which a plurality of imaging devices 10 are installed on a road where a plurality of vehicles pass, and a plurality of vehicles as target objects are imaged. Then, the vehicles in each of the images T1, T2, T3, T4 captured at each time are collated to determine whether they are the same vehicle.

[0067] The object matching device 50 of this embodiment is a device that discriminates and matches the same target object between frames of different videos captured by a plurality of imaging devices 10. The object matching device 50 includes a storage unit 21, an input unit 52, an object detection unit 23, an execution instruction unit 54, and an output unit 25. The contents of the storage unit 21, the object detection unit 23, and the output unit 25 are the same as those in the first embodiment.

[0068] The input unit 52 receives the input of a plurality of images obtained by imaging a target object with a plurality of imaging devices 10. Note that, in the same manner as in the first embodiment, the object detection unit 23 detects the target object from each received image. In this embodiment, the object detection unit 23 detects the target object from a plurality of images obtained by imaging the target object with a plurality of imaging devices 10.

[0069] The execution instruction unit 54, in the same manner as in the first embodiment, transmits the information of the first node, the information of the second node, and the similarity to the quantum computer 30 to cause the execution of the matching process. Note that the method by which the execution instruction unit 54 generates the information of the first node, the information of the second node, and the similarity is the same as that in the first embodiment. Further, the execution instruction unit 54 of this embodiment may, for example, predict the position of the target object according to the distance of the range imaged by the imaging device 10 and generate the information of the second node. Also, the execution instruction unit 54 may determine the similarity in consideration of the accuracy according to the distance of the imaging range.

[0070] Hereinafter, the process of collectively transmitting the information of the second node in a plurality of patterns to the quantum computer 30 and causing the execution of the matching process for each piece of information of the second node is the same as that in the first embodiment.

[0071] As described above, in this embodiment, the input unit 52 receives the input of a plurality of images obtained by imaging a target object with a plurality of imaging devices 10, and the object detection unit 23 detects the target object from the plurality of images. Therefore, in addition to the effects of the first embodiment, it becomes possible to match the target objects between distant positions. Also, since the images captured by the plurality of imaging devices 10 can be used, it is also possible to take measures against the dead angles of the occlusion camera.

Example

[0072] The present invention will be described below with reference to specific examples, but the scope of the present invention is not limited to the content described below. Hereinafter, a general tracking process and a tracking process using the object matching system according to the present invention will be compared by taking the case of using a single camera as an example. In the evaluation of this embodiment, YOLOv5 was commonly used for vehicle detection in any tracking process.

[0073] As general tracking processes, two types of tracking processes will be exemplified. The first tracking process is the tracking process (DeepSort) described in Non-Patent Document 1. The second tracking process is a tracking process using maximum weight maximum matching (the number of edges at the time of matching is maximized), and matching is performed based on the IoU between the position prediction result and the detection result. In this embodiment, Linear Sum Assignment attached to the SciPy package was used. On the other hand, the tracking process used in the present invention is a tracking process using minimum weight maximum matching (the number of edges at the time of matching cannot be increased any further).

[0074] In this embodiment, the tracking process was evaluated using MOTA (Multi Object Tracking Accuracy) and the vehicle count ratio. MOTA evaluates the frequencies of non-detection, false detection, and switching of tracking IDs, and is calculated by Equation 3 shown below.

[0075]

Equation

[0076] In Equation 3, t is the frame number, and g t is the number of correct data in the t-th frame. Also, FP t , Miss t , IDSW t represent the frequencies of non-detection, false detection, and switching of tracking IDs in the t-th frame, respectively.

[0077] The vehicle count ratio is a value calculated as "the number of tracked vehicles / the number of correct vehicles", and the closer it is to 1.0, the higher the evaluation. In this case, the evaluation was performed based on the number of vehicles passing through the intersection. Note that the tracking process using the object matching system according to the present invention was substituted with a package that can be calculated even by a general personal computer (PC). In this embodiment, three types of videos were used for the evaluation.

[0078] FIG. 9 is an explanatory diagram showing the evaluation results. In the method of FIG. 9, "First" indicates the first tracking process, "Second" indicates the second tracking process, and "This case" indicates the tracking process using the object matching system according to the present invention. As shown in FIG. 9, it was confirmed that in the tracking process using the object matching system according to the present invention, the accuracy can be maintained particularly from the viewpoint of the count ratio.

[0079] Next, the outline of the present invention will be described. FIG. 10 is a block diagram showing the outline of the object matching system according to the present invention. The object matching system 80 according to the present invention includes an input means 81 (for example, input unit 22) that receives an input of a plurality of images obtained by imaging a target object, which is an object of a type to be matched (for example, a vehicle, a human, a particle, etc.), an object detection means 82 (for example, object detection unit 23) that detects the target object from the images, and a quantum computer (for example, quantum computer 30) that executes minimum-maximum matching. Information on the first node representing the position of the detected target object (for example, at time t), information on the second node representing the position of the target object predicted based on the past matching results (for example, at time t), and the similarity (for example, IoU) between the position of the first node and the position of the second node are transmitted to execute a matching process between the past matching results and the target object detected from the image, and an execution instruction means 83 (for example, execution instruction unit 24) that determines the first node corresponding to the second node.

[0080] With such a configuration, object matching can be performed while improving the calculation speed and maintaining the accuracy.

[0081] Further, the execution instruction means 83 may create a plurality of patterns of the information of the second node according to the length of the past collation result used for prediction, cause the quantum computer to execute the collation process for each pattern, and determine to associate nodes with more matching collation results for each pattern.

[0082] At this time, the execution instruction means 83 may collectively transmit the created information of the plurality of second nodes and cause the collation process to be executed in a batch. According to such a configuration, an increase in the calculation cost can be suppressed.

[0083] Furthermore, the execution instruction means 83 may determine the first node corresponding to the second node by a majority vote of the candidates of the first nodes corresponding to the second nodes from the results of the collation process.

[0084] Also, the object detection means 82 may detect parts (for example, head, cells, etc.) that are individual parts included in the target object from the image. Then, the execution instruction means 83 may transmit the information of the first node representing the position of the detected part, the information of the second node representing the position of the part predicted based on the past collation result, and the similarity of the positions of the respective parts to the quantum computer to cause the collation process to be executed.

[0085] Also, the input means 81 may receive the input of a plurality of images obtained by imaging the target object with a plurality of imaging devices (for example, the imaging device 10), and the object detection means 82 may detect the target object from the plurality of images.

[0086] As described above, the present invention has been described with reference to the embodiments and examples, but the present invention is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0087] This application claims the priority based on Japanese Patent Application No. 2021-201521 filed on December 13, 2021, and incorporates all of its disclosures herein.

Industrial Applicability

[0088] The present invention is suitably applicable to an object matching system that matches objects existing in a plurality of images. Further, the present invention is suitably applicable to a system that utilizes a quantum computer to grasp real-time traffic volume and perform traffic control accordingly.

Explanation of Reference Numerals

[0089] 10 Imaging device 20, 50 Object matching device 21 Storage unit 22, 52 Input unit 23 Object detection unit 24, 54 Execution instruction unit 25 Output unit 30 Quantum computer 40 Browsing terminal

Claims

1. Input means for receiving an input of a plurality of images obtained by imaging a target object, which is an object of a type to be collated; Object detection means for detecting the target object from the images; Execution instruction means for transmitting information of a first node representing the position of the detected target object, information of a second node representing the position of the target object predicted based on past collation results, and the degree of similarity between the position of the first node and the position of the second node to a quantum computer that executes minimum-maximum matching, causing the quantum computer to execute a collation process between the past collation results and the target object detected from the images, and determining the first node corresponding to the second node; An object collation system characterized by the above.

2. The execution instruction means creates a pattern of information of the second node for each length of the collation result with a gradually increasing period according to the length of the past collation result used for prediction, causes the quantum computer to execute a collation process for each pattern, and determines to associate nodes with more matching results for each pattern. The object collation system according to Claim 1.

3. The execution instruction means transmits the created information of the plurality of second nodes together and causes the collation process to be executed in a batch. The object collation system according to Claim 2.

4. The execution instruction means determines the first node corresponding to the second node by a majority vote of the candidates of the first node corresponding to the second node from the results of the collation process. The object collation system according to Claim 2 or Claim 3.

5. The object detection means detects parts, which are individual parts included in the target object, from the images, and the execution instruction means transmits information of a first node representing the position of the detected parts, information of a second node representing the position of the parts predicted based on past collation results, and the degree of similarity of the positions of the respective parts to a quantum computer to cause the collation process to be executed. The object collation system according to Claim 1 or Claim 2.

6. The input means receives an input of a plurality of images obtained by imaging a target object with a plurality of imaging devices, and the object detection means detects the target object from the plurality of images. The object collation system according to Claim 1 or Claim 2.

7. Receiving an input of a plurality of images obtained by imaging a target object, which is an object of a type to be collated; Detecting the target object from the images; For a quantum computer that performs minimum-maximum matching, information on a first node representing the position of the detected target object, information on a second node representing the position of the target object predicted based on past matching results, and the similarity between the position of the first node and the position of the second node are transmitted to execute a matching process between the past matching results and the target object detected from the image. Determine the first node corresponding to the second node. An object matching method characterized by the above.

8. According to the length of the past matching results used for prediction, create a pattern of information on the second node for each length of the matching results with the period gradually increased. Cause the quantum computer to execute a matching process for each pattern. Determine to associate nodes with more matching results of the matching process. The object matching method according to Claim 7.

9. On a computer, An input process for receiving an input of a plurality of images obtained by imaging a target object, which is an object of a type to be matched. An object detection process for detecting the target object from the image. For a quantum computer that performs minimum-maximum matching, information on a first node representing the position of the detected target object, information on a second node representing the position of the target object predicted based on past matching results, and the similarity between the position of the first node and the position of the second node are transmitted to execute a matching process between the past matching results and the target object detected from the image, and an execution instruction process for determining the first node corresponding to the second node is executed. An object matching program for the above purpose.

10. On a computer, In an execution instruction process, according to the length of the past matching results used for prediction, create a pattern of information on the second node for each length of the matching results with the period gradually increased, cause the quantum computer to execute a matching process for each pattern, and determine to associate nodes with more matching results of the matching process. The object matching program according to Claim 9.

Images