Learning device, matching device, learning method, matching method, and program

Abstract

Provided is a learning device that can improve the accuracy of matching. For each piece of tracked body data of tracked body information about a tracked body, a correct answer weight generation unit (12) uses correct answer tracked body pair information that is a set of tracked body information for the same tracked body or a set of tracked body information for different tracked bodies to generate a correct answer weight. An inference model training unit (14) uses machine learning to train an inference model that receives data related to tracked body information as input data and uses the correct answer weights generated for the tracked body information as correct answer data to output tracked body data weights that correspond to tracked body data included in the tracked body information. The tracked body data weights are used in association with the similarities between tracked body data included in tracked body information about a first tracked body of a pair of tracked bodies and tracked body data included in tracked body information about a second tracked body of the pair of tracked bodies when a tracked body matching score is to be calculated during matching processing for the pair of tracked bodies.

Description

[Technical Field]

[0001] The present invention relates to a learning device, a matching device, a learning method, a matching method, and program Regarding. [Background technology]

[0002] A method for matching objects such as people is known. In relation to this technology, Patent Document 1 discloses a match determination device that efficiently identifies the same analysis target from multiple pieces of sensing information. The device disclosed in Patent Document 1 identifies a selected feature selected from one or more features for the analysis target included in an analysis group, and evaluates whether the analysis targets of the multiple analysis groups match based on a combination of the selected features between different analysis groups. Furthermore, if the evaluation indicates a match between the analysis targets of the analysis groups, the device disclosed in Patent Document 1 identifies the analysis targets of the different analysis groups as the same target. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 138983 Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology disclosed in Patent Document 1, during matching, it is only evaluated whether the analysis targets of multiple analysis groups match based on the combination of selected features between different analysis groups. With this method, there is a risk that matching cannot be performed with high accuracy.

[0005] The purpose of the present disclosure has been made to solve such problems, and is to provide a learning device, a matching device, a learning method, a matching method, and a program that can improve the accuracy of matching. [Means for solving the problem]

[0006] The learning device according to the present disclosure includes a correct weight generating means for generating a correct weight corresponding to correct data of a tracked object data weight relating to importance indicating how well the tracked object data represents the characteristics of the tracked object corresponding to the tracked object in the tracked object information, using correct tracked object pair information which is a set of tracked object information of the same tracked object or a set of tracked object information of different tracked objects, for each of the tracked object data of tracked object information which includes at least feature amount information indicating the characteristics of the tracked object that is the object to be tracked and which includes one or more tracked object data obtained by tracking the tracked object using video; and an inference model learning means for learning, by machine learning, an inference model that uses the tracked object data as input data, the tracked object data weight generated for the tracked object information as correct answer data, and outputs a tracked object data weight corresponding to the tracked object data included in the tracked object information, wherein the correct answer weight generation means generates the tracked object data weight to be used in association with the similarity between the tracked object data included in the tracked object information for a first tracked object of the pair of tracked objects and the tracked object data included in the tracked object information for a second tracked object when calculating a tracked object matching score that is a matching score for the pair of tracked objects in a matching process for the pair of tracked objects.

[0007] The matching device according to the present disclosure also includes a weight inference means for inferring tracked body data weights corresponding to each of the tracked body data included in the tracked body information of a pair of tracked bodies to be matched, using an inference model that has been trained in advance by machine learning, the inference model being trained to use as input data data related to tracked body information that includes at least feature information indicating the characteristics of a tracked body that is an object to be tracked and includes one or more tracked body data obtained by tracking the tracked body using video, and using as correct answer data a correct answer weight corresponding to correct answer data for a tracked body data weight that is related to the importance of how well the tracked body data represents the characteristics of the tracked body that corresponds to the tracked body in the tracked body information, and outputting tracked body data weights corresponding to the tracked body data included in the tracked body information related to the input data; and a tracked body matching means for performing a matching process for the pair of tracked bodies by associating the similarity between the tracked body data included in the tracked body information related to a first tracked body of the pair and the tracked body data included in the tracked body information related to a second tracked body with the inferred tracked body data weight, and calculating a tracked body matching score that is a matching score for the pair of tracked bodies.

[0008] Furthermore, the learning method according to the present disclosure uses correct tracked body pair information, which is a set of tracked body information of identical tracked bodies or a set of tracked body information of distinct tracked bodies, to generate a correct answer weight corresponding to correct answer data of a tracked body data weight related to importance indicating how well the tracked body data represents the characteristics of the tracked body corresponding to the tracked body in the tracked body information, for each of the tracked body data of tracked body information which includes at least feature information indicating the characteristics of the tracked body, which is an object to be tracked, and which includes one or more tracked body data obtained by tracking the tracked body using video; the data related to the tracked body information is input data, and the correct answer weight generated for the tracked body information is used as correct answer data to learn an inference model by machine learning, which outputs a tracked body data weight corresponding to the tracked body data included in the tracked body information; and the tracked body data weight is used in association with the similarity between the tracked body data included in the tracked body information of a first tracked body and the tracked body data included in the tracked body information of a second tracked body when calculating a tracked body matching score, which is a matching score of the pair of tracked bodies in a matching process of the pair of tracked bodies.

[0009] Furthermore, the matching method according to the present disclosure is an inference model that has been trained in advance by machine learning, and uses as input data data relating to tracked body information that includes at least feature information indicating the characteristics of the tracked body, which is an object to be tracked, and includes one or more tracked body data obtained by tracking the tracked body using video, and uses as correct answer data a correct answer weight corresponding to the correct answer data of a tracked body data weight that is related to the importance of how well the tracked body data represents the characteristics of the corresponding tracked body in the tracked body information, and outputs tracked body data weights corresponding to the tracked body data included in the tracked body information relating to the input data, and uses the inference model trained to infer tracked body data weights corresponding to each of the tracked body data included in the tracked body information of a pair of tracked bodies to be matched, and performs a matching process for the pair of tracked bodies by associating the similarity between the tracked body data included in the tracked body information regarding the first tracked body of the pair and the tracked body data included in the tracked body information regarding the second tracked body with the inferred tracked body data weight and calculating a tracked body matching score, which is a matching score for the pair of tracked bodies.

[0010] A first program according to the present disclosure causes a computer to execute the above learning method.

[0011] A second program according to the present disclosure causes a computer to execute the above-described matching method. [Effects of the Invention]

[0012] According to the present disclosure, it is possible to provide a learning device, a matching device, a learning method, a matching method, and a program that can improve matching accuracy. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram illustrating an overview of a learning device according to an embodiment of the present disclosure. [Figure 2] 1 is a flowchart illustrating a learning method executed by a learning device according to an embodiment of the present disclosure. [Figure 3] 1 is a diagram illustrating an overview of a matching device according to an embodiment of the present disclosure. [Figure 4] 1 is a flowchart illustrating a matching method executed by a matching device according to an embodiment of the present disclosure. [Figure 5] FIG. 1 is a diagram illustrating a configuration of a matching system according to a first embodiment. [Figure 6] FIG. 1 illustrates a configuration of a learning device according to a first embodiment. [Figure 7] FIG. 2 is a diagram illustrating tracked object information according to the first embodiment. [Figure 8] FIG. 2 is a diagram illustrating correct tracking object pair information according to the first embodiment. [Figure 9] FIG. 2 is a diagram illustrating correct tracking object pair information according to the first embodiment. [Figure 10] 10 is a flowchart showing a process of a correct weight generating unit according to the first embodiment. [Figure 11] FIG. 10 is a diagram illustrating correct tracking weight information according to the first embodiment. [Figure 12] FIG. 4 is a diagram for explaining the processing of the correct weight generating unit according to the first embodiment. [Figure 13] 10 is a flowchart showing the processing of the inference model learning unit according to the first embodiment. [Figure 14] FIG. 1 is a diagram for explaining a learning method for an inference model according to a first embodiment; [Figure 15] FIG. 1 is a diagram illustrating a configuration of a matching device according to a first embodiment. [Figure 16] 10 is a flowchart showing a process of a weight inference unit according to the first embodiment; [Figure 17] 10 is a flowchart showing a process of a tracked object matching unit according to the first embodiment. [Figure 18] FIG. 10 is a diagram illustrating a configuration of a learning device according to a second embodiment. [Figure 19] 10 is a flowchart showing a learning method executed by the learning device according to the second embodiment. [Figure 20]10 is a flowchart showing a process of a tracked object clustering unit according to the second embodiment. [Figure 21] FIG. 10 is a diagram for explaining the processing of a tracked object clustering unit according to the second embodiment. [Figure 22] FIG. 11 is a diagram illustrating an example of tracked object information stored in a tracked object information storage unit according to the second embodiment. [Figure 23] FIG. 11 is a diagram illustrating a state in which tracked object information stored in a tracked object information storage unit according to the second embodiment is clustered. [Figure 24] 10 is a flowchart showing a process of a pseudo-correct tracking body pair information generating unit according to the second embodiment. [Figure 25] 10 is a flowchart showing a process of a pseudo-correct tracking body pair information generating unit according to the second embodiment. [Figure 26] FIG. 11 is a diagram illustrating pseudo-correct tracking entity pair information corresponding to identical correct tracking entity pair information according to the second embodiment. [Figure 27] FIG. 11 is a diagram illustrating pseudo-correct tracking entity pair information corresponding to another correct tracking entity pair information according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] (Summary of Embodiments of the Present Disclosure) Prior to describing the embodiments of the present disclosure, an overview of the embodiments of the present disclosure will be described. Fig. 1 is a diagram illustrating an overview of a learning device 10 according to the embodiments of the present disclosure. Fig. 2 is a flowchart illustrating a learning method executed by the learning device 10 according to the embodiments of the present disclosure.

[0015] The learning device 10 is, for example, a computer. The learning device 10 has a correct answer weight generation unit 12 and an inference model learning unit 14. The correct answer weight generation unit 12 functions as a correct answer weight generation means. The inference model learning unit 14 functions as an inference model learning means. The learning device 10 learns an inference model, which will be described later.

[0016] The correct weight generation unit 12 generates a correct weight for tracked object information related to the tracked object, which is an object to be tracked (a tracked object) (step S12). The tracked object is, for example, a person, but is not limited to this. The tracked object may be an animal or a moving object other than a living thing (for example, a vehicle, an aircraft, etc.). In the following embodiment, a case will be described where the tracked object is a person. In the following description, "the same tracked object as tracked object A" means, if the tracked object is a person, that the person is the same as tracked object A (person A). Furthermore, "a tracked object separate (different) from tracked object A" means, if the tracked object is a person, that the person is different from tracked object A (person A). The tracked object information and correct weight will be described below.

[0017] "Tracking object information" includes one or more pieces of tracking object data related to one tracking object. In other words, the tracking object data included in one piece of tracking object information relates to the same tracking object. For example, if the tracking object is a person, tracking object information related to a certain person A (tracking object A) includes one or more pieces of tracking object data related to that person A (tracking object A). In this embodiment, it is assumed that there are multiple different pieces of tracking object information related to a certain person X (tracking object X). The tracking object data includes at least feature amount information indicating the characteristics of the tracking object. The tracking object data is obtained by tracking the tracking object using video. The feature amount information may include multiple feature components (elements). In other words, the feature amount information corresponds to a feature amount vector. Furthermore, the feature amount information is information that makes it possible to calculate the similarity between two objects by comparing the feature amount information of each of the two objects. This will be described in more detail below.

[0018] The "correct weight" corresponds to the correct data (correct label) used in the learning stage of the inference model, which will be described later. The correct weight corresponds to the correct data of the tracked object data weight, which is a weight related to the tracked object data.

[0019] A "tracked object data weight" is associated with each piece of tracked object data included in the tracked object information. The tracked object data weight relates to the importance indicating how well the corresponding tracked object data represents the characteristics of the corresponding tracked object in the tracked object information in which the tracked object data is included. In other words, the tracking data weight may correspond to the relative importance of one or more tracked object data included in the tracked object information when matching between two pieces of tracked object information. The correct answer weight and the tracked object data weight will be described later. Note that the "tracked object data weight" corresponds to the output data of the inference model, as will be described later. In other words, the tracked object data weight is inferred by the inference model, which will be described later. That is, the inference model, which will be described later, outputs a tracked object data weight corresponding to the tracked object data included in the tracked object information.

[0020] Here, the tracked object data weight is used when calculating a tracked object matching score corresponding to the matching score (matching degree, similarity degree, etc.) of the pair of tracked objects in the matching process of the pair of tracked objects. Specifically, the tracked object data weight is used in association with the similarity between the tracked object data included in the tracked object information related to the first tracked object of the pair of tracked objects and the tracked object data included in the tracked object information related to the second tracked object. A specific method for calculating the tracked object matching score will be described later.

[0021] Furthermore, the correct weight generation unit 12 generates correct weights using the correct tracked body pair information. "Correct tracked body pair information" is information in which two pieces of tracked body information are paired. Correct tracked body pair information is a set of tracked body information of the same tracked body, or a set of tracked body information of different tracked bodies. The correct tracked body pair information will be described later. The process of S12 will be described later in detail.

[0022] The inference model learning unit 14 learns an inference model by machine learning such as a neural network (step S14). The inference model learning unit 14 uses data related to the tracked object information as input data and uses the correct answer weight generated for the tracked object information as correct answer data to learn an inference model that outputs a tracked object data weight corresponding to the tracked object data included in the tracked object information. The input data (features) of the inference model will be described later. The processing of S14 will be described in detail later.

[0023] Fig. 3 is a diagram illustrating an overview of the matching device 20 according to the embodiment of the present disclosure, and Fig. 4 is a flowchart illustrating a matching method executed by the matching device 20 according to the embodiment of the present disclosure.

[0024] The matching device 20 is, for example, a computer. The matching device 20 has a weight inference unit 22 and a tracked object matching unit 24. The weight inference unit 22 functions as a weight inference means (inference means). The tracked object matching unit 24 functions as a tracked object matching means (matching means). The matching device 20 matches the tracked object using a trained inference model.

[0025] The weight inference unit 22 infers tracked object data weights using an inference model trained in advance by machine learning as described above (step S22). Specifically, the weight inference unit 22 infers tracked object data weights corresponding to each of the tracked object data included in the tracked object information of each of the pair of tracked objects to be compared using the inference model trained as described above.

[0026] The tracked object matching unit 24 performs matching processing on the pair of tracked objects to be matched (step S24). Here, the pair of tracked objects consists of a first tracked object and a second tracked object. Then, the tracked object matching unit 24 associates the similarity between the tracked object data included in the tracked object information of the first tracked object and the tracked object data included in the tracked object information of the second tracked object with the inferred tracked object data weight, and calculates the tracked object matching score of the pair of tracked objects. In this way, the tracked object matching unit 24 performs matching processing on the pair of tracked objects.

[0027] Here, an example of a method for calculating a tracker matching score according to this embodiment will be described. In this embodiment, the tracker matching score is calculated, for example, as shown in the following formula (1). Formula (1) is a formula for calculating the matching score between tracker A and tracker B (tracker matching score).

number

[0028] In formula (1), "Score" is the tracked body matching score between tracked body A and tracked body B. The higher the Score, the higher the possibility that tracked body A and tracked body B are the same tracked body. Furthermore, n is the number of tracked body data in the tracked body information of tracked body A. m is the number of tracked body data in the tracked body information of tracked body B. Furthermore, i is the index of tracked body data in the tracked body information of tracked body A. j is the index of tracked body data in the tracked body information of tracked body B. Furthermore, w i A is the tracked object data weight corresponding to tracked object data i in the tracked object information of tracked object A. j B is the tracked object data weight corresponding to the tracked object data j in the tracked object information of tracked object B. Also, f i,j indicates the similarity between tracked object data i in the tracked object information of tracked object A and tracked object data j in the tracked object information of tracked object B. i,j may indicate, for example, the cosine similarity of the feature amount information (feature amount vector) included in the tracked object data.

[0029] As shown in formula (1), the tracked body matching score corresponds to the sum of the products of the similarity between the tracked body data in the tracked body information of tracked body A and the tracked body data in the tracked body information of tracked body B for each combination and the weights of the two tracked body data. In other words, the tracked body matching score corresponds to the sum of the products of the similarity between the tracked body data in the tracked body information of tracked body A and the tracked body data in the tracked body information of tracked body B and the weights of these two tracked body data for all combinations of tracked body data. In addition, the tracked body matching score, the weight w, and the similarity f i,j can take on values ​​in the range (0,1).

[0030] Here, for comparison with the present embodiment, a method for calculating a tracker matching score according to a comparative example will be described below. In the comparative example, the tracker matching score is calculated as shown in the following formula (2). Formula (2) is a formula for calculating the matching score between tracker A and tracker B (tracker matching score).

number

[0031] As shown in formula (2), in the comparative example, the tracked object matching score is calculated by averaging the similarities between the tracked object data for each combination of the tracked object data in the tracked object information of tracked object A and the tracked object data in the tracked object information of tracked object B. In the tracked object matching score calculated in this manner, the weights of all tracked object data are treated as equal. In other words, the tracked object matching score calculated by the method according to the comparative example does not take into account the weights of the tracked object data. Here, some tracked object data included in the tracked object information well represents the characteristics of the corresponding tracked object, while other tracked object data does not well represent the characteristics of the tracked object. Therefore, the importance (contribution degree) of the tracked object data included in the tracked object information is not constant. Therefore, a tracked object matching score calculated by treating the tracked object data equally may not have good matching accuracy.

[0032] In contrast, the tracked object matching score according to this embodiment corresponds to the sum of the products of the similarities for all combinations of the tracked object data in the tracked object information of tracked object A and the tracked object data in the tracked object information of tracked object B and the weights of the corresponding two tracked object data. In other words, the tracked object matching score according to this embodiment corresponds to the weighted average of the similarities for all combinations of the tracked object data in the tracked object information of tracked object A and the tracked object data in the tracked object information of tracked object B. Therefore, when calculating the tracked object matching score, the tracked object data weight is used in association with the similarity between the tracked object data included in the tracked object information related to the first tracked object of the pair of tracked objects and the tracked object data included in the tracked object information related to the second tracked object. This allows the weights of these tracked object data to be taken into account in the similarity between the two tracked object data. Therefore, in the tracked object matching score, importance is placed on the similarity for tracked object data that is important in the tracked object information (that well represents the characteristics of the tracked object). This allows the accuracy of the tracked object matching score to be improved.

[0033] Therefore, the matching device 20 according to this embodiment can perform matching with high accuracy. Moreover, the learning device 10 according to this embodiment can learn an inference model for inferring tracked object data weights necessary for performing matching with high accuracy. The learning device 10 according to this embodiment can generate correct weights corresponding to correct data for tracked object data weights, which are used in training the inference model. Therefore, the learning device 10 according to this embodiment can improve matching accuracy. Note that the matching accuracy can also be improved by a learning method that realizes the learning device 10 and a program that executes the learning method. Also, the matching method that realizes the matching device 20 and a program that executes the matching method can also perform matching with high accuracy.

[0034] Furthermore, the correct weight generating unit 12 may generate correct weights based on the similarity between each tracked body data included in the tracked body information of one tracked body in each of the plurality of correct tracked body pair information and each tracked body data included in the tracked body information of the other tracked body (S12). This makes it possible to generate correct weights more effectively. Details will be described later.

[0035] (Embodiment 1) Hereinafter, embodiments will be described with reference to the drawings. For clarity of explanation, the following description and drawings have been omitted and simplified as appropriate. In addition, the same elements in each drawing are designated by the same reference numerals, and duplicate explanations have been omitted as necessary.

[0036] 5 is a diagram showing the configuration of a verification system 50 according to the first embodiment. The verification system 50 has, as its main hardware components, a control unit 52, a storage unit 54, a communication unit 56, and an interface unit 58 (IF; Interface). The control unit 52, the storage unit 54, the communication unit 56, and the interface unit 58 are connected to each other via a data bus or the like.

[0037] The control unit 52 is a processor such as a CPU (Central Processing Unit). The control unit 52 functions as an arithmetic device that performs control processing, arithmetic processing, etc. The control unit 52 may have multiple processors. The storage unit 54 is a storage device such as a memory or a hard disk. The storage unit 54 is, for example, a ROM (Read Only Memory) or a RAM (Random Access Memory). The storage unit 54 has a function for storing control programs, arithmetic programs, etc. executed by the control unit 52. In other words, the storage unit 54 (memory) stores one or more instructions. The storage unit 54 also has a function for temporarily storing processing data, etc. The storage unit 54 may include a database. The storage unit 54 may have multiple memories.

[0038] The communication unit 56 performs processing necessary for communicating with other devices via a network. The communication unit 56 may include a communication port, a router, a firewall, etc. The interface unit 58 (IF; Interface) is, for example, a user interface (UI). The interface unit 58 has an input device such as a keyboard, a touch panel, or a mouse, and an output device such as a display or a speaker. The interface unit 58 may be configured such that the input device and the output device are integrated, such as a touch screen (touch panel). The interface unit 58 accepts data input operations by a user (operator) and outputs information to the user. The interface unit 58 may display the matching results.

[0039] The matching system 50 also includes a learning device 100 and a matching device 200. The learning device 100 corresponds to the learning device 10 described above. The matching device 200 corresponds to the matching device 20 described above. The learning device 100 and the matching device 200 are, for example, computers. The learning device 100 and the matching device 200 may be physically implemented as the same device. Alternatively, the learning device 100 and the matching device 200 may be physically implemented as separate devices (computers). In this case, the learning device 100 and the matching device 200 each have the hardware configuration described above.

[0040] The learning device 100 executes the learning method shown in Fig. 2. That is, the learning device 100 generates correct weights and learns an inference model used in matching tracked objects. The matching device 200 executes the matching method shown in Fig. 4. That is, the matching device 200 uses the trained inference model to infer weights of tracked object data (tracked object data weights) included in tracked object information for each of a pair of tracked objects to be matched, and calculates a matching score using the obtained tracked object data weights. Details of the learning device 100 and the matching device 200 will be described later.

[0041] FIG. 6 is a diagram illustrating a configuration of a learning device 100 according to the first embodiment. The learning device 100 may include, as a hardware configuration, the control unit 52, the storage unit 54, the communication unit 56, and the interface unit 58 illustrated in FIG. 5. The learning device 100 also includes, as components, a correct tracking entity pair information storage unit 110, a correct weight generation unit 120, a correct tracking weight information storage unit 130, an inference model learning unit 140, an inference model storage unit 150, and an input data designation unit 160. The learning device 100 does not need to be physically configured as a single device. In this case, the above-described components may be realized by multiple physically separate devices.

[0042] The correct tracking entity pair information storage unit 110 functions as correct tracking entity pair information storage means (information storage means). The correct weight generation unit 120 corresponds to the correct weight generation unit 12 shown in FIG. 1. The correct weight generation unit 120 functions as correct weight generation means. The correct tracking weight information storage unit 130 functions as correct tracking weight information storage means (information storage means). The inference model learning unit 140 corresponds to the inference model learning unit 14 shown in FIG. 1. The inference model learning unit 140 functions as inference model learning means. The inference model storage unit 150 functions as inference model storage means. The input data designation unit 160 functions as input data designation means (designation means).

[0043] Each of the above-described components can be realized, for example, by executing a program under the control of the control unit 52. More specifically, each component can be realized by the control unit 52 executing a program (instructions) stored in the storage unit 54. Alternatively, each component may be realized by recording the necessary programs on any non-volatile recording medium and installing them as needed. Each component may not necessarily be realized by software programs, but may be realized by any combination of hardware, firmware, and software. Each component may also be realized using a user-programmable integrated circuit, such as an FPGA (field-programmable gate array) or a microcomputer. In this case, the integrated circuit may be used to realize a program composed of each of the above-described components. The same applies to the collation device 200 and other embodiments described below.

[0044] The correct tracking entity pair information storage unit 110 stores a large number of correct tracking entity pair information. For example, the correct tracking entity pair information storage unit 110 may store approximately 100 to 1000 pieces of correct tracking entity pair information. As described above, the correct tracking entity pair information is information in which two pieces of tracking entity information are paired. Therefore, the correct tracking entity pair information includes a pair of tracking entity information.

[0045] Correct tracked body pair information is same correct tracked body pair information or different correct tracked body pair information. Same correct tracked body pair information is a set of tracked body information of the same tracked body. On the other hand, different correct tracked body pair information is a set of tracked body information of different tracked bodies. Therefore, in correct tracked body pair information, it is clear in advance whether the two pieces of tracked body information are tracked body information related to the same tracked body, or whether the two pieces of tracked body information are tracked body information related to different tracked bodies. In other words, same correct tracked body pair information is generated using tracked body information related to the same tracked body reliably (accurately). On the other hand, different correct tracked body pair information is generated using tracked body information related to different tracked bodies reliably (accurately).

[0046] Here, specific examples of the tracked body information and correct tracked body pair information will be described with reference to the drawings. Fig. 7 is a diagram illustrating an example of tracked object information according to the first embodiment. Fig. 7 shows tracked object information (tracked object information A) related to a certain tracked object A (for example, person A). The tracked object information illustrated in Fig. 7 includes eight pieces of tracked object data A1 to A8.

[0047] The tracked object data may be obtained, for example, from an image (video) of a single tracked object obtained by an imaging device such as a camera. Each of the multiple tracked object data included in one piece of tracked object information may correspond to, for example, a different frame (video frame) in a video (moving image). Note that a frame corresponds to each still image (frame) that makes up the video data. Each of the multiple tracked object data included in one piece of tracked object information may be obtained by performing object detection processing (image processing) on ​​each of the different frames. Note that the multiple tracked object data included in one piece of tracked object information may correspond to each frame of a video obtained by different imaging devices.

[0048] As described above, the tracked object information includes one or more pieces of tracked object data related to the same tracked object. Here, the tracked object information may include tracked object data of different frames related to the same tracked object through object tracking processing. That is, the tracked object information may be acquired by object tracking processing (video analysis processing) using, for example, an image sequence (video) acquired by an imaging device such as a camera as input. The object tracking processing may, for example, use an image sequence of objects in chronological order as input, and detect and track the same object as that detected in an image frame at a certain time in the next or subsequent time frame. Note that the object tracking processing may track the same object based on, for example, the similarity of the object's position and appearance in the image.

[0049] As described above, the tracked object data includes at least feature information indicating the features of the tracked object. The feature information can be acquired, for example, by performing an object detection process on a frame to detect the tracked object present in the frame, extracting image data of the detected tracked object, and acquiring the feature of the tracked object from the extracted image data. An existing algorithm may be used as a method for acquiring the feature of the tracked object from the image data of the tracked object. For example, the feature of the tracked object may be acquired using a trained model trained by machine learning such as a neural network to input image data and output feature of the object shown in the image. Components (elements) of the feature indicated by the feature information include, for example, the positions of feature points of a person's face, the reliability of human-likeness, the coordinate positions of skeletal points, and the reliability of clothing labels, but are not limited to these.

[0050] As described above, the tracked object data A1 to A8 may be acquired from different frames. Each of the tracked object data A1 to A8 includes at least feature amount information corresponding to the tracked object A. The tracked object data may also indicate the time at which the corresponding frame was acquired, and the position and size of the tracked object in the corresponding frame (image). The position and size of the tracked object may be the position coordinates and size of a rectangle surrounding the tracked object in the frame. Note that the components (elements) of the features indicated by the feature amount information included in each of the tracked object data A1 to A8 may be the same, but the values ​​of the components (component values) may be different.

[0051] The number of tracked object data included in one piece of tracked object information is not limited to eight and may be any number. Different pieces of tracked object information may include different numbers of tracked object data. For example, one piece of tracked object information may include eight pieces of tracked object data, another piece of tracked object information may include six pieces of tracked object data, and yet another piece of tracked object information may include one piece of tracked object data.

[0052] 8 and 9 are diagrams illustrating examples of correct tracking entity pair information according to the first embodiment. Fig. 8 is a diagram illustrating an example of same correct tracking entity pair information. Fig. 9 is a diagram illustrating an example of different correct tracking entity pair information.

[0053] The correct tracked body pair information (same correct tracked body pair information) illustrated in FIG. 8 includes tracked body information about tracked body A and tracked body B, which are the same tracked body. In other words, tracked body A and tracked body B are, for example, the same person X. Tracked body information about tracked body A (tracked body information A) includes eight pieces of tracked body data A1 to A8. Tracked body information about tracked body B (tracked body information B) includes eight pieces of tracked body data B1 to B8.

[0054] Tracked object information A and tracked object information B may be obtained, for example, from images taken at different time periods. For example, tracked object information A may include tracked object data obtained from images taken of person X from 11:00. Furthermore, tracked object information B may include tracked object data obtained from images taken of person X from 13:00. Alternatively, tracked object information A and tracked object information B may be obtained from images taken by imaging devices installed at different locations. For example, tracked object information A may include tracked object data obtained from images taken of person X from the left side or the front. Furthermore, tracked object information B may include tracked object data obtained from images taken of person X from the right side or the rear.

[0055] Furthermore, the correct tracked body pair information includes a tracked body pair type. The tracked body pair type indicates whether the pair of tracked body information included in the correct tracked body pair information is tracked body information related to the same tracked body or tracked body information related to different tracked bodies. The tracked body pair type included in the correct tracked body pair information (same correct tracked body pair information) illustrated in FIG. 8 indicates "same tracked body." In other words, the same correct tracked body pair information illustrated in FIG. 8 is definitely generated using tracked body information related to the same tracked body A and tracked body B.

[0056] The correct tracked body pair information (different correct tracked body pair information) illustrated in FIG. 9 includes tracked body information about tracked body A and tracked body C, which are different from each other. For example, tracked body A is person X, and tracked body C is person Y, which is different from person X. Tracked body information about tracked body A (tracked body information A) includes eight pieces of tracked body data A1 to A8. Tracked body information about tracked body C (tracked body information C) includes eight pieces of tracked body data C1 to C8. Furthermore, the tracked body pair type included in the correct tracked body pair information (different correct tracked body pair information) illustrated in FIG. 9 indicates "different tracked body." In other words, the different correct tracked body pair information illustrated in FIG. 9 is generated using tracked body information about tracked body A and tracked body C, which are definitely different from each other.

[0057] Here, the tracker information A included in the correct tracking entity pair information (different correct tracking entity pair information) exemplified in FIG. 9 is the same as the tracker information A included in the correct tracking entity pair information (same correct tracking entity pair information) exemplified in FIG. 8. In other words, the same tracker information regarding a certain tracker may be included in each of multiple correct tracking entity pair information. Therefore, the tracker information A may be included in same correct tracking entity pair information that is different from the same correct tracking entity pair information exemplified in FIG. 8. Similarly, the tracker information A may be included in different correct tracking entity pair information that is different from the different correct tracking entity pair information exemplified in FIG. 9.

[0058] The number of tracked object data included in each piece of tracked object information included in the correct tracked object pair information is arbitrary. For example, in the example of Fig. 8, tracked object information A may include six pieces of tracked object data, and tracked object information B may include four pieces of tracked object data. Also, in the example of Fig. 9, tracked object information A may include six pieces of tracked object data, and tracked object information C may include one piece of tracked object data. However, at least one of the tracked object information included in the correct tracked object pair information must include multiple pieces of tracked object data.

[0059] The correct weight generation unit 120 generates a correct weight using the correct tracked body pair information. Specifically, the correct weight generation unit 120 may calculate a similarity between each of the tracked body data included in the tracked body information of one tracked body in each of the plurality of correct tracked body pair information and each of the tracked body data included in the tracked body information of the other tracked body. Then, the correct weight generation unit 120 may generate a correct weight for the tracked body data based on the calculated similarity.

[0060] Furthermore, the correct weight generation unit 120 may assign points (weighted points) to the tracked object data based on the calculated similarity, and generate a correct weight for the tracked object data according to the number of assigned points. Furthermore, the correct weight generation unit 120 may assign points to tracked object data corresponding to the highest similarity among the similarities calculated using a set of tracked object information of the same tracked object (same correct tracked object pair information) among the correct tracked object pair information. Furthermore, the correct weight generation unit 120 may assign points to tracked object data corresponding to the lowest similarity among the similarities calculated using a set of tracked object information of another tracked object (different correct tracked object pair information) among the correct tracked object pair information.

[0061] The processing of the correct answer weight generating unit 120 will be described in detail below using a flowchart. Fig. 10 is a flowchart showing the processing of the correct weight generation unit 120 according to the first embodiment. The processing of the flowchart shown in Fig. 10 corresponds to the processing of S12 shown in Fig. 2. The correct weight generation unit 120 acquires one piece of correct tracked object pair information from the correct tracked object pair information storage unit 110 (step S102). As a result, a pair of tracked object information is acquired.

[0062] The correct weight generation unit 120 calculates all similarities between the tracked object data in the pair of tracked object information included in the acquired correct tracked object pair information (step S104). Here, the "similarity between tracked object data" is calculated by f i,jSpecifically, the correct weight generation unit 120 calculates the similarity for all combinations of each of the tracked body data included in one of the tracked body information pieces in the acquired correct tracked body pair information and each of the tracked body data included in the other tracked body information piece.

[0063] When the correct tracked object pair information exemplified in FIG. 8 is acquired, the correct weight generation unit 120 calculates the similarity between the tracked object data A1 and the tracked object data B1. The correct weight generation unit 120 also calculates the similarity between the tracked object data A1 and the tracked object data B2. Similarly, the correct weight generation unit 120 calculates the similarity between the tracked object data A1 and each of the tracked object data B1 to B8. Similarly, the correct weight generation unit 120 calculates the similarity between the tracked object data A2 and each of the tracked object data B1 to B8. Similarly, the correct weight generation unit 120 calculates the similarity between the tracked object data for all combinations of each of the tracked object data A1 to A8 and each of the tracked object data B1 to B8. In other words, the correct answer weight generation unit 120 calculates the similarity between the tracked body data for all 64 (=8×8) combinations of each of the eight tracked body data in tracked body information A and each of the eight tracked body data in tracked body information B.

[0064] 9 is acquired, the correct weight generation unit 120 calculates the similarity between the tracked object data A1 and the tracked object data C1. The correct weight generation unit 120 also calculates the similarity between the tracked object data A1 and the tracked object data C2. Similarly, the correct weight generation unit 120 calculates the similarity between the tracked object data A1 and each of the tracked object data C1 to C8. Similarly, the correct weight generation unit 120 also calculates the similarity between the tracked object data A2 and each of the tracked object data C1 to C8. Similarly, the correct weight generation unit 120 calculates the similarity between the tracked object data for all combinations of each of the tracked object data A1 to A8 and each of the tracked object data C1 to C8. In other words, the correct answer weight generation unit 120 calculates the similarity between the tracked body data for all 64 (=8×8) combinations of each of the eight tracked body data in tracked body information A and each of the eight tracked body data in tracked body information C.

[0065] The correct weight generation unit 120 determines whether the acquired correct tracked body pair information includes tracked body information of the same tracked body (step S106). Specifically, the correct weight generation unit 120 determines whether the tracked body pair type of the acquired correct tracked body pair information indicates "same tracked body". If the tracked body pair type of the acquired correct tracked body pair information indicates "same tracked body", the correct weight generation unit 120 determines that the acquired correct tracked body pair information includes tracked body information of the same tracked body. On the other hand, if the tracked body pair type of the acquired correct tracked body pair information indicates "different tracked bodies", the correct weight generation unit 120 determines that the acquired correct tracked body pair information includes tracked body information of different tracked bodies.

[0066] If the correct tracked body pair information includes tracked body information of the same tracked body (YES in S106), the correct weight generation unit 120 assigns points to the tracked body data with the highest similarity (step S108). Specifically, the correct weight generation unit 120 assigns points (weighted points) to each of the two (pair) tracked body data used when the highest similarity among all the calculated similarities was calculated.

[0067] 8, let us assume that the similarity between the tracked object data A2 and the tracked object data B7 is the highest among the 64 similarities calculated in the process of S104. In this case, the correct answer weight generation unit 120 assigns a weight point of "1" to each of the tracked object data A2 and the tracked object data B7.

[0068] When the tracked body pair type of the correct tracked body pair information is "same tracked body," it is desirable that one tracked body information and the other tracked body information are similar to each other. Therefore, it is desirable that the tracked body matching score between one tracked body information and the other tracked body information is high. From the above-described formula (1) or (2), the tracked body matching score can be higher as the similarity between each tracked body data of one tracked body information and the tracked body data of the other tracked body information increases. Therefore, among the combinations of each tracked body data of one tracked body information and the tracked body data of the other tracked body information, it can be said that two tracked body data constituting a combination with high similarity well represent the characteristics of the corresponding tracked body in the tracked body information to which they belong. Therefore, when the tracked body pair type of the correct tracked body pair information is "same tracked body," the correct weight generation unit 120 assigns weight points to each of the two tracked body data constituting the combination with the highest similarity among all combinations. This allows weight points to be assigned to tracked body data with high importance.

[0069] On the other hand, if the correct tracked body pair information includes tracked body information of separate tracked bodies (NO in S106), the correct weight generation unit 120 assigns points to the tracked body data with the lowest similarity (step S110). Specifically, the correct weight generation unit 120 assigns points (weighted points) to each of the two (pair) tracked body data used when the lowest similarity among all the calculated similarities was calculated.

[0070] 9, it is assumed that the similarity between the tracked object data A6 and the tracked object data C8 is the lowest among the 64 similarities calculated in the process of S104. In this case, the correct answer weight generation unit 120 assigns a weight point of "1" to each of the tracked object data A6 and the tracked object data C8.

[0071] When the tracked body pair type of the correct tracked body pair information is "different tracked body," it is desirable that one tracked body information and the other tracked body information are different (dissimilar) from each other. Therefore, it is desirable that the matching score between one tracked body information and the other tracked body information is low. From the above-mentioned formula (1) or formula (2), the matching score can be lower as the similarity between each tracked body data of one tracked body information and the tracked body data of the other tracked body information decreases. Therefore, among combinations of each tracked body data of one tracked body information and the tracked body data of the other tracked body information, it can be said that two tracked body data constituting a combination with low similarity well represent the characteristics of the corresponding tracked body in the tracked body information to which they belong. Therefore, when the tracked body pair type of the correct tracked body pair information is "different tracked body," the correct weight generation unit 120 assigns weight points to each of the two tracked body data constituting the combination corresponding to the lowest similarity among all combinations. This makes it possible to assign weight points to tracked body data with high importance.

[0072] The correct weight generation unit 120 determines whether there is correct tracking entity pair information that has not been acquired from the correct tracking entity pair information storage unit 110 (step S112). If there is correct tracking entity pair information that has not been acquired (YES in S112), the process flow returns to S102. Then, the processes of S102 to S112 are repeated. As a result, for each of the multiple correct tracking entity pair information stored in the correct tracking entity pair information storage unit 110, weight points are assigned to each tracking entity data of the tracking entity information included in the correct tracking entity pair information. Here, as described above, the same tracking entity information (e.g., tracking entity information A) related to a certain tracking entity may be included in each of the multiple correct tracking entity pair information. Therefore, by repeating the processes of S102 to S112, weight points related to each tracking entity data of each tracking entity information are added up.

[0073] On the other hand, if there is no unacquired correct tracked object pair information (NO in S112), the correct weight generation unit 120 generates a correct weight for each tracked object data for each tracked object information (step S114). Specifically, the correct weight generation unit 120 calculates the total value of weight points assigned to each tracked object data included in the tracked object information. The correct weight generation unit 120 generates a correct weight for each tracked object data in the tracked object information by normalizing the total value of weight points calculated for each tracked object data in the tracked object information within the range of 0 to 1. Specifically, the correct weight generation unit 120 generates a correct weight for each tracked object data by dividing the total value of weight points for each tracked object data by the sum of the total values ​​of weight points calculated for each tracked object data in the tracked object information. As a result, the sum of the correct weights for the tracked object data in the tracked object information becomes 1. The correct weight generation unit 120 generates correct tracked weight information corresponding to the tracked object information.

[0074] The correct tracking weight information storage unit 130 stores correct tracking weight information corresponding to each tracked object information. The correct tracking weight information storage unit 130 stores correct tracking weight information corresponding to each of the multiple tracked object information included in the multiple correct tracked object pair information stored in the correct tracked object pair information storage unit 110.

[0075] FIG. 11 is a diagram illustrating correct tracking weight information according to the first embodiment. FIG. 11 shows correct tracking weight information relating to tracked object information A (tracked object A) illustrated in FIG. 7 and the like. The correct tracking weight information illustrated in FIG. 11 includes tracked object data A1 to A8 and corresponding correct weights WA1 to WA8. The correct tracking weight information storage unit 130 stores the correct tracking weight information illustrated in FIG. 11 for each of a plurality of tracked object information (e.g., tracked object information A, tracked object information B, tracked object information C).

[0076] The process of S114 in Fig. 10 will now be described with reference to Fig. 11. It is assumed that weight points are assigned to each tracked object data in tracked object information A by repeating the processes of S102 to S112 as follows: The total value of weight points assigned to the tracked object data A1 is "1". The total value of weight points assigned to the tracked object data A2 is "4". The total value of weight points assigned to the tracked object data A3 is "0". The total value of weight points assigned to the tracked object data A4 is "0". The total value of weight points assigned to the tracked object data A5 is "1". The total value of weight points assigned to the tracked object data A6 is "3". The total value of weight points assigned to the tracked object data A7 is "0". The total value of weight points assigned to the tracked object data A8 is "1".

[0077] In the above example, the total sum of the weight points assigned to each tracked object data is 1+4+0+0+1+3+0+1=10. Therefore, the correct weight generation unit 120 calculates the correct weight WA1 for tracked object data A1 as 1 / 10=0.1. The correct weight generation unit 120 also calculates the correct weight WA2 for tracked object data A2 as 4 / 10=0.4. The correct weight generation unit 120 also calculates the correct weight WA5 for tracked object data A5 as 1 / 10=0.1. The correct weight generation unit 120 also calculates the correct weight WA6 for tracked object data A6 as 3 / 10=0.3. The correct weight generation unit 120 also calculates the correct weight WA8 for tracked object data A8 as 1 / 10=0.1. The correct weight generation unit 120 calculates the correct weights WA3, WA4, and WA7 for the tracked object data A3, A4, and A7, respectively, as 0 / 10=0. As a result, the total of the correct weights WA1 to WA8 becomes 1.

[0078] Fig. 12 is a diagram for explaining the processing of the correct weight generation unit 120 according to the first embodiment. Fig. 12 shows processing when two correct tracking body pair information are used: the correct tracking body pair information (same correct tracking body pair information) exemplified in Fig. 8 and the correct tracking body pair information (different correct tracking body pair information) exemplified in Fig. 9.

[0079] In the case of the same correct tracked object pair information exemplified in Fig. 8, the correct weight generation unit 120 calculates the similarity between the tracked object data for all combinations of each of the tracked object data A1 to A8 and each of the tracked object data B1 to B8. Then, as shown by arrow F11, it is assumed that the similarity between the tracked object data A2 and the tracked object data B7 is the highest. In this case, as shown by arrow F12, the correct weight generation unit 120 assigns a weight point of "1" to each of the tracked object data A2 and the tracked object data B7.

[0080] 9, the correct weight generation unit 120 calculates the similarity between the tracked object data for all combinations of each of the tracked object data A1 to A8 and each of the tracked object data C1 to C8. Then, as shown by arrow F13, it is assumed that the similarity between the tracked object data A6 and the tracked object data C8 is the highest. In this case, as shown by arrow F14, the correct weight generation unit 120 assigns a weight point of "1" to each of the tracked object data A6 and the tracked object data C8.

[0081] Through the above processing, the correct weight generation unit 120 calculates the total weight points of the tracked object data A2 as "1" and the total weight points of the tracked object data A6 as "1" for the tracked object information A related to tracked object A, as shown by arrow F15. Therefore, the total sum of the weight points is "2". Then, the correct weight generation unit 120 normalizes the total weight points as shown by arrow F16, and calculates the correct weight of the tracked object data A2 as "0.5" (= 1 / 2) and the correct weight of the tracked object data A6 as "0.5" (= 1 / 2).

[0082] The inference model learning unit 140 (FIG. 6) learns an inference model using the correct tracking weighting information. The inference model learning unit 140 takes data related to the tracked object information as input data, and uses the correct weights generated for the tracked object information as correct data to learn an inference model that outputs tracked object data weights corresponding to the tracked object data included in the tracked object information. For example, when the tracked object information A described above is used, the inference model learning unit 140 learns an inference model using data related to the tracked object information A as input data and the correct weights generated for the tracked object information A as correct data. In other words, the inference model learning unit 140 learns an inference model using the correct tracking weighting information exemplified in FIG. 11.

[0083] The inference model is trained by a machine learning algorithm such as a neural network. The input data (features) of the inference model may include, for example, feature information of each tracked object data included in the tracked object information. Furthermore, the input data (features) of the inference model may indicate, for example, a graph structure indicating a similarity relationship between the tracked object data included in the tracked object information. In this case, the inference model may be trained using, for example, a graph neural network or a graph convolutional neural network. This makes it possible to train an inference model with higher accuracy. The graph structure will be described later.

[0084] Fig. 13 is a flowchart showing the processing of the inference model learning unit 140 according to the first embodiment. The processing of the flowchart shown in Fig. 13 corresponds to the processing of S14 shown in Fig. 2. The inference model learning unit 140 acquires correct tracking weight information from the correct tracking weight information storage unit 130 (step S120). As a result, the inference model learning unit 140 acquires the tracked object data included in the tracked object information and the correct weight corresponding to each tracked object data.

[0085] The inference model learning unit 140 generates data (graph structure data) showing the graph structure of the tracked object data (step S122). Specifically, the inference model learning unit 140 calculates the similarity between each tracked object data included in the tracked object information and all other tracked object data. In the example of FIG. 11, the inference model learning unit 140 calculates the similarity between the tracked object data A1 and each of the tracked object data A2 to A8. Similarly, the inference model learning unit 140 calculates the similarity between the tracked object data A2 to A8 and each of the other tracked object data. Note that the "similarity between tracked object data" is calculated using f shown in formula (1). i,j The inference model learning unit 140 may assign data such as a flag indicating that a combination of tracked object data has a similarity equal to or greater than a predetermined threshold value, among the combinations of tracked object data. The inference model learning unit 140 then generates graph structure data indicating the combinations of tracked object data having a similarity equal to or greater than the threshold value.

[0086] The graph structure data may be included in the correct tracking weight information in advance. In this case, the graph structure data may be generated by the correct weight generating unit 120 (or another component).

[0087] The inference model learning unit 140 inputs input data related to the tracked object data into the inference model and infers tracked object data weights (step S124). Specifically, the inference model learning unit 140 inputs, as input data, feature information of the tracked object data included in the correct tracking weighting information (tracked object information) and the graph structure data generated by the processing of S122 into the inference model. As a result, the inference model outputs weights (tracked object data weights) corresponding to each of the tracked object data (tracked object information) included in the correct tracking weighting information. In this way, the inference model learning unit 140 infers tracked object data weights using the inference model.

[0088] The inference model learning unit 140 calculates a loss function using the tracked object data weight and the correct weight obtained by inference (step S126). Specifically, the inference model learning unit 140 calculates the loss function using the tracked object data weight obtained in the process of S124 and the correct weight included in the correct tracking weight information obtained in the process of S120. More specifically, the inference model learning unit 140 may calculate the loss function using, for example, a least square error. That is, the inference model learning unit 140 may calculate the loss function by the sum of the squares of the differences between the correct weight and the inferred tracked object data weight for each tracked object data. Note that the method of calculating the loss function is not limited to using a least square error, and any function used in machine learning may be used.

[0089] The inference model learning unit 140 adjusts the parameters of the inference model by backpropagation using a loss function (step S128). Specifically, the inference model learning unit 140 uses the loss function calculated in S126 to adjust the parameters of the inference model (such as the weights of neurons in a neural network) by backpropagation, which is commonly used in machine learning. This allows the inference model to be learned.

[0090] The inference model learning unit 140 determines whether the iterations (number of repetitions) exceed a specified value or whether the loss function has converged (step S130). If the iterations exceed a specified value or the loss function has converged (YES in S130), the inference model learning unit 140 ends the processing. In other words, the inference model learning unit 140 ends the learning of the inference model. Then, the inference model learning unit 140 stores the learned inference model in the inference model storage unit 150.

[0091] On the other hand, if the iteration does not exceed the specified value and the loss function has not converged (NO in S130), the inference model training unit 140 continues training the inference model. Therefore, the processing flow returns to S120. Then, the inference model training unit 140 acquires another correct tracking weight information (S120) and performs the training process of the inference model (S122 to S128). Then, the training process of the inference model is repeated until the iteration exceeds the specified value or the loss function converges.

[0092] The input data specification unit 160 (FIG. 6) specifies data to be used as input data. Specifically, the input data specification unit 160 may specify components of feature information to be used in training the inference model. The input data specification unit 160 is realized by controlling the interface unit 58. For example, a user can use the input data specification unit 160 to specify which features are to be used in training the inference model. For example, the input data specification unit 160 allows the user to select which components of feature information to use and which components not to use. This allows the inference model to be trained effectively when the user knows in advance which components of feature information are effective for the inference model.

[0093] FIG. 14 is a diagram for explaining a learning method of an inference model according to the first embodiment. FIG. 14 shows a learning method using correct tracking weight information related to the tracked object information A exemplified in FIG. 11. The inference model learning unit 140 acquires correct tracking weight information related to the tracked object information A (S120). Then, the inference model learning unit 140 generates a graph structure G1 indicating the similarity relationship of the tracked object data A1 to A8 included in the correct tracking weight information (S122). The graph structure G1 exemplified in FIG. 14 is shown such that, among the combinations of the tracked object data A1 to A8, combinations whose similarity is equal to or greater than a threshold are connected by lines. For example, focusing on the tracked object data A1, the similarity between the tracked object data A1 and the tracked object data A5 and the similarity between the tracked object data A1 and the tracked object data A6 are equal to or greater than a threshold. Furthermore, when attention is focused on the tracked object data A6, the similarity between the tracked object data A6 and each of the tracked object data A1, A2, A3, A4, A5, and A7 is equal to or greater than the threshold value.

[0094] The inference model learning unit 140 inputs the feature information included in each of the tracked object data A1 to A8 and the graph structure data indicating the graph structure G1 into the inference model as input data (features). As a result, the inference model learning unit 140 infers tracked object data weights corresponding to each of the tracked object data A1 to A8, as indicated by arrow W1 (S124). In the example of FIG. 14, the tracked object data weight for the tracked object data A2 is "0.3". Similarly, the tracked object data weights for the tracked object data A3, A5, A6, and A8 are "0.1", "0.1", "0.4", and "0.1", respectively.

[0095] The inference model learning unit 140 calculates the loss function as described above using the correct weight of the tracked object information A indicated by the arrow W2 and the inferred tracked object data weight indicated by the arrow W1 (S126).Then, the inference model learning unit 140 adjusts the parameters of the inference model by backpropagation based on the calculated loss function (S128).

[0096] As described above, the learning device 100 according to the first embodiment uses the correct tracked object pair information to generate correct weights corresponding to the tracked object data included in the tracked object information. Then, the learning device 100 according to the first embodiment uses data related to the tracked object information as input data and the correct weights generated for the tracked object information as correct data to learn an inference model.

[0097] As a result, as shown in formula (1), in the matching process of a pair of tracked bodies, the weights of the tracked body data included in the tracked body information related to the first tracked body and the tracked body data included in the tracked body information related to the second tracked body can be associated with the similarity between these tracked body data. This makes it possible to increase the accuracy of the tracked body matching score. Therefore, it is possible to reduce the FAR (False Acceptance Rate) and FRR (False Rejection Rate). This makes it possible to improve the accuracy of matching.

[0098] Furthermore, the input data input to the inference model according to the first embodiment is graph-structured data indicating feature information included in each tracked object data of the tracked object information and similarity relationships between the tracked object data. By configuring the input data in this way, the input data can be low-load (small-volume) data, such as text data. Here, in a technology that uses image input data to train a model that infers tracked object features, there is a risk that the processing time will increase in the learning and inference stages of the inference model. In contrast, in the first embodiment, an inference model based on tracking is trained using low-load input data, rather than an inference model of tracked object features, and therefore it is possible to reduce the processing time in the learning and inference stages of the inference model.

[0099] As described above, the learning device 100 according to the first embodiment calculates the similarity between each of the tracked object data included in the tracked object information of one tracked object in each of the plurality of pieces of correct tracked object pair information and each of the tracked object data included in the tracked object information of the other tracked object. Then, the learning device 100 according to the first embodiment generates a correct weight for the tracked object data based on the calculated similarity. This configuration enables more accurate generation of the correct weight.

[0100] As described above, the learning device 100 according to the first embodiment assigns points (weighted points) to the tracked object data based on the calculated similarity, and generates a correct weight for the tracked object data according to the number of assigned points. In this case, the learning device 100 according to the first embodiment assigns points to tracked object data corresponding to the highest similarity among the similarities calculated using the same correct tracked object pair information in the correct tracked object pair information. On the other hand, the learning device 100 according to the first embodiment assigns points to tracked object data corresponding to the lowest similarity among the similarities calculated using the different correct tracked object pair information in the correct tracked object pair information. With this configuration, the correct weight can be generated using both the same correct tracked object pair information and the different correct tracked object pair information, making it possible to generate a correct weight more accurately.

[0101] FIG. 15 is a diagram illustrating a configuration of the matching device 200 according to the first embodiment. The matching device 200 may have, as a hardware configuration, the control unit 52, the storage unit 54, the communication unit 56, and the interface unit 58 shown in FIG. 5. The matching device 200 also has, as components, an inference model storage unit 202, a tracked object information acquisition unit 210, a weight inference unit 220, and a tracked object matching unit 240. Note that the matching device 200 does not need to be configured as a single physical device. In this case, the above-described components may be realized by multiple physically separate devices.

[0102] The inference model storage unit 202 functions as an inference model storage means. The inference model storage unit 202 stores the inference model learned by the learning device 100 as described above. The tracked object information acquisition unit 210 functions as a tracked object information acquisition means. The weight inference unit 220 corresponds to the weight inference unit 22 shown in FIG. 3. The weight inference unit 220 functions as a weight inference means (inference means). The tracked object matching unit 240 corresponds to the tracked object matching unit 24 shown in FIG. 3. The tracked object matching unit 240 functions as a tracked object matching means (matching means).

[0103] The tracked object information acquisition unit 210 acquires tracked object information regarding each of a pair of tracked objects to be compared. Specifically, the tracked object information acquisition unit 210 may acquire tracked object information generated in advance by some method from a database or the like. Alternatively, the tracked object information acquisition unit 210 may acquire tracked object information by tracking the tracked object using images (video) of the tracked object acquired by an imaging device. In this case, as described above, the tracked object information acquisition unit 210 detects the tracked object by performing object detection processing (image processing) on ​​the corresponding tracked object for each frame constituting the video, extracts feature amounts of the detected tracked object, and performs object tracking processing. In this way, the tracked object information acquisition unit 210 acquires tracked object data regarding the tracked object to be compared. Then, the tracked object information acquisition unit 210 acquires tracked object information including one or more tracked object data.

[0104] The weight inference unit 220 uses a trained inference model to infer tracked object data weights corresponding to each of the tracked object data included in the tracked object information relating to the pair of tracked objects to be compared. Hereinafter, this will be explained using a flowchart.

[0105] Fig. 16 is a flowchart showing the processing of the weight inference unit 220 according to the first embodiment. The processing of the flowchart shown in Fig. 16 corresponds to the processing of S22 shown in Fig. 4. The weight inference unit 220 acquires tracked object information of tracked objects to be matched (step S202). Specifically, for example, when tracked objects A and B are to be matched, the weight inference unit 220 acquires tracked object information A related to tracked object A and tracked object information B related to tracked object B.

[0106] The weight inference unit 220 inputs input data related to the tracked object information acquired in S202 into the inference model, and infers tracked object data weights for each of the tracked object data included in the tracked object information related to the input data (step S204). Note that the inference process for the tracked object data weights can be executed independently for each of a pair of tracked objects. That is, the weight inference unit 220 inputs input data related to tracked object information A, and infers tracked object data weights for each of the tracked object data A1 to A8 included in tracked object information A. Furthermore, the weight inference unit 220 inputs input data related to tracked object information B, and infers tracked object data weights for each of the tracked object data B1 to B8 included in tracked object information B.

[0107] The weight inference unit 220 inputs, for example, feature information included in each tracked object data of the tracked object information to the inference model as input data. The weight inference unit 220 may also input the above-mentioned graph structure data to the inference model as input data. That is, the input data may include feature information of each tracked object data and graph structure data. The weight inference unit 220 may generate the graph structure data by the above-mentioned method. Alternatively, the graph structure data may be generated by the tracked object information acquisition unit 210. By using the graph structure data as input data, it is possible to infer the tracked object data weight with high accuracy.

[0108] The weight inference unit 220 generates weighted tracked object information for each of the pair of tracked objects to be matched (step S206). The weighted tracked object information is information that associates the tracked object data included in the tracked object information acquired in S202 with the tracked object data weight inferred in S204. The weighted tracked object information for tracked object A may have, for example, a configuration substantially similar to the correct tracked object weight information illustrated in FIG. 11. However, it should be noted that the weighted tracked object information for tracked object A does not have a "correct weight" but has a "tracked object data weight" obtained by inference.

[0109] The tracked object matching unit 240 matches a pair of tracked objects to be matched, as will be explained below with reference to a flowchart.

[0110] Fig. 17 is a flowchart showing the processing of the tracked object matching unit 240 according to the first embodiment. The processing of the flowchart shown in Fig. 17 corresponds to the processing of S24 shown in Fig. 4. The tracked object matching unit 240 acquires weighted tracked object information of a pair of tracked objects to be matched (step S212). For example, when tracked objects A and B are to be matched, the tracked object matching unit 240 acquires the weighted tracked object information of tracked objects A and B generated in the processing of S206.

[0111] The tracked object matching unit 240 calculates a tracked object matching score (step S214). Specifically, the tracked object matching unit 240 calculates the tracked object matching score using the weighted tracked object information acquired in S214. More specifically, the tracked object matching unit 240 calculates a similarity between the tracked object data included in the tracked object information (weighted tracked object information) related to the first tracked object of the pair of tracked objects and the tracked object data included in the tracked object information (weighted tracked object information) related to the second tracked object. Then, the tracked object matching unit 240 calculates the tracked object matching score by associating the calculated similarity with the tracked object data weight related to the tracked object data corresponding to the similarity.

[0112] The tracked object matching unit 240 calculates the tracked object matching score "Score" using, for example, the above-mentioned formula (1). Here, it is assumed that tracked object A and tracked object B are the objects to be matched. In this case, for example, the tracked object matching unit 240 calculates the similarity between the tracked object data for each of all combinations of the tracked object data in the tracked object information of tracked object A and the tracked object data in the tracked object information of tracked object B. The tracked object matching unit 240 multiplies each of the calculated similarities by two tracked object data weights corresponding to the similarities. Then, the tracked object matching unit 240 calculates the sum of the products obtained by multiplying each similarity by the tracked object data weight. In this way, the tracked object matching unit 240 calculates the tracked object matching score "Score" between tracked object A and tracked object B.

[0113] For example, the tracked object matching unit 240 calculates the similarity f between the tracked object data A1 related to the tracked object A and the tracked object data B1 related to the tracked object B. 1,1 The tracked object matching unit 240 calculates the calculated similarity f 1,1 , the tracked object data weight w1 for the tracked object data A1 A and the tracked body data weight w1 of the tracked body data B1 B Furthermore, the tracked object matching unit 240 multiplies the similarity f 1,2 The tracked object matching unit 240 calculates the calculated similarity f 1,2 , the tracked object data weight w1 for the tracked object data A1 A and tracked body data weight w2 of tracked body data B2 B Similarly, the tracked object matching unit 240 multiplies the similarity f between the tracked object data A1 related to the tracked object A and each of the tracked object data B3 to B8 related to the tracked object B. 1,3 ~f 1,8 The tracked object matching unit 240 calculates the calculated similarity f 1,3 ~f 1,8 Each of them has a tracked object data weight w1 for the tracked object data A1. A and tracked body data weight w3 of tracked body data B3 to B8 B ~w8 BThe tracked object matching unit 240 performs the same process on the tracked object data A2 to A8 related to the tracked object A. Then, the tracked object matching unit 240 calculates the sum of the products of the obtained similarities and the tracked object data weights as the tracked object matching score.

[0114] The tracked object matching unit 240 can determine that the pair of tracked objects to be matched are the "same tracked object" when the tracked object matching score is equal to or greater than a predetermined threshold. On the other hand, the tracked object matching unit 240 can determine that the pair of tracked objects to be matched are "different tracked objects" when the tracked object matching score is less than a predetermined threshold.

[0115] As described above, the matching device 200 according to the first embodiment uses a trained inference model to infer tracked object data weights for a pair of tracked objects to be matched. Then, the matching device 200 according to the first embodiment calculates tracked object matching scores for the pair of tracked objects to be matched using the tracked object data weights inferred as described above. This makes it possible to improve the accuracy of the tracked object matching score, and therefore the accuracy of matching.

[0116] (Embodiment 2) Next, a description will be given of a second embodiment. For clarity of explanation, the following description and drawings have been omitted and simplified as appropriate. In addition, in each drawing, the same elements are given the same reference numerals, and duplicate explanations are omitted as necessary.

[0117] The configuration of the verification system 50 according to the second embodiment is substantially the same as the configuration of the verification system 50 according to the first embodiment shown in Fig. 5, and therefore a description thereof will be omitted. The configuration of the verification device 200 according to the second embodiment is substantially the same as the configuration of the verification device 200 according to the first embodiment shown in Fig. 15, and therefore a description thereof will be omitted. That is, the verification system 50 according to the second embodiment includes a learning device 100A (shown in Fig. 18) corresponding to the learning device 100, and the verification device 200.

[0118] In the first embodiment, the correct tracking body pair information is prepared and stored in advance. In contrast, the learning device 100A according to the second embodiment differs from the first embodiment in that it generates pseudo correct tracking body pair information from the tracking body information and generates correct weights using this pseudo correct tracking body pair information.

[0119] FIG. 18 is a diagram illustrating a configuration of a learning device 100A according to the second embodiment. The learning device 100A may have, as a hardware configuration, the control unit 52, the storage unit 54, the communication unit 56, and the interface unit 58 shown in FIG. 5. The learning device 100A also has, as components, a tracked object information storage unit 102A, a tracked object clustering unit 104A, a tracked object cluster information storage unit 106A, a pseudo-correct tracked object pair information generation unit 108A, and a pseudo-correct tracked object pair information storage unit 110A. As will be described later, the learning device 100A uses these components to generate pseudo-correct tracked object pair information used in generating correct weights.

[0120] Similarly to the learning device 100, the learning device 100A has, as its components, a correct answer weight generation unit 120, a correct answer tracking weight information storage unit 130, an inference model learning unit 140, an inference model storage unit 150, and an input data designation unit 160. The functions of the correct answer weight generation unit 120, the correct answer tracking weight information storage unit 130, the inference model learning unit 140, the inference model storage unit 150, and the input data designation unit 160 are substantially the same as those in the first embodiment, and therefore description thereof will be omitted.

[0121] Note that the learning device 100A does not need to be physically configured as a single device. In this case, the above-mentioned components may be realized by multiple physically separate devices. For example, the tracker information storage unit 102A, the tracker clustering unit 104A, the tracker cluster information storage unit 106A, the pseudo-correct tracker pair information generation unit 108A, and the pseudo-correct tracker pair information storage unit 110A may be realized by a device separate from the other components.

[0122] The tracked entity information storage unit 102A functions as tracked entity information storage means (information storage means). The tracked entity clustering unit 104A functions as tracked entity clustering means (clustering means). The tracked entity cluster information storage unit 106A functions as tracked entity cluster information storage means (information storage means). The pseudo-correct tracked entity pair information generation unit 108A functions as pseudo-correct tracked entity pair information generation means (information generation means). The pseudo-correct tracked entity pair information storage unit 110A functions as pseudo-correct tracked entity pair information storage means (information storage means).

[0123] 19 is a flowchart showing a learning method executed by the learning device 100A according to the second embodiment. The learning device 100A clusters tracked objects (step S2A). The learning device 100A generates pseudo-correct tracked object pair information (step S4A). The learning device 100A generates correct weights (step S12). The learning device 100A learns an inference model (step S14). Details of the processes of S2A and S4A will be described later. Furthermore, S12 and S14 are substantially the same as the processes of S12 and S14 described above, and therefore description thereof will be omitted.

[0124] The tracked object information storage unit 102A stores the tracked object information as described above in advance. The tracked object information storage unit 102A stores a large number of pieces of tracked object information as shown in FIG. 7. Unlike the first embodiment, the tracked object information stored in advance in the tracked object information storage unit 102A is not paired. As will be described later, the plurality of pieces of tracked object information stored in the tracked object information storage unit 102A are clustered by the processing of S2A. That is, the plurality of pieces of tracked object information stored in the tracked object information storage unit 102A are assigned to one or more clusters by the processing of S2A.

[0125] The tracked object clustering unit 104A clusters a plurality of pieces of tracked object information stored in the tracked object information storage unit 102A. Specifically, the tracked object clustering unit 104A clusters the tracked object information related to a plurality of tracked objects that are considered to be identical to one another. Note that the clustered plurality of tracked objects are not necessarily the same tracked objects in reality.

[0126] A set of tracked object information related to multiple tracked objects that are considered to be identical to one another and clustered is called a "cluster (tracked object cluster)." The tracked object cluster information storage unit 106A stores information related to the clusters into which tracked objects are clustered (tracked object cluster information). The tracked object cluster information may indicate the cluster ID (identification information) of each cluster and the tracked object information related to the tracked objects that belong to that cluster. In other words, the tracked object cluster information may indicate the tracked object information related to each tracked object and the cluster ID of the cluster to which the tracked object belongs. Note that the tracked object cluster information may include identification information of the tracked objects (tracked object information) that belong to the corresponding cluster, instead of the tracked object information.

[0127] 20 is a flowchart illustrating the processing of the tracked object clustering unit 104A according to the embodiment 2. The processing of the flowchart illustrated in FIG. 20 corresponds to the processing of S2A illustrated in FIG.

[0128] The tracked object clustering unit 104A determines whether or not there is any tracked object information that has not been assigned to a cluster among the tracked object information stored in the tracked object information storage unit 102A (step S302). Subsequent processing is performed for each piece of tracked object information stored in the tracked object information storage unit 102A, and when there is no tracked object information that has not been assigned to a cluster (NO in S302), the processing flow in FIG. 20 ends.

[0129] If there is tracked object information that has not been assigned to a cluster (YES in S302), the tracked object clustering unit 104A acquires tracked object information about a new tracked object from the tracked object information storage unit 102A (step S304). Here, a "new tracked object" refers to a tracked object that has not been clustered and does not belong to any cluster.

[0130] The tracked object clustering unit 104A refers to the tracked object cluster information storage unit 106A and searches for a similar tracked object whose matching score with the new tracked object (tracked object matching score) is higher than a predetermined threshold Th1 (step S306). The threshold Th1 is a threshold that represents the lower limit of the matching score at which tracked objects are considered to be similar (substantially identical). Specifically, the tracked object clustering unit 104A calculates a matching score between all tracked object information stored in the tracked object cluster information storage unit 106A (i.e., tracked object information of clustered tracked objects) and the tracked object information of the new tracked object. The matching score may be calculated using, for example, the above formula (2). Then, the tracked object clustering unit 104A searches for tracked objects related to tracked object information whose matching score is higher than the threshold Th1 as similar tracked objects. Note that at the stage of processing the initially acquired tracked object information, none of the tracked objects have been clustered, and no tracked object information is stored in the tracked object cluster information storage unit 106A. Therefore, no similar tracked object is searched for.

[0131] The tracker clustering unit 104A determines whether the number of searched similar trackers is equal to or greater than a predetermined threshold Th2 (step S308). The threshold Th2 is a threshold that represents the lower limit of the number of similar trackers that belong to the same cluster. The threshold Th2 is an integer equal to or greater than 1. For example, the threshold Th2=1. If the number of searched similar trackers is not equal to or greater than the threshold Th2 (NO in S308), the tracker clustering unit 104A assigns a new cluster ID to the new tracker (step S310). That is, a new tracker with few (or no) similar trackers stored in the tracker cluster information storage unit 106A is clustered into a cluster with a new cluster ID.

[0132] In this way, the tracked object clustering unit 104A associates a new cluster ID with the tracked object information acquired in S304. As a result, the new tracked object is clustered into the cluster of that cluster ID. Then, the tracked object clustering unit 104A stores the cluster ID of the new tracked object and the corresponding tracked object information as tracked object cluster information in the tracked object cluster information storage unit (step S312). Then, the process returns to S302.

[0133] On the other hand, if the number of the searched similar trackers is equal to or greater than the threshold value Th2 (YES in S308), the tracker clustering unit 104A determines whether the cluster IDs corresponding to the searched similar trackers are all the same (step S320). That is, the tracker clustering unit 104A determines whether the searched similar trackers belong to the same cluster.

[0134] If the cluster IDs of the searched similar trackers are all the same (YES in S320), the tracker clustering unit 104A assigns that cluster ID to the new tracker. As a result, the new tracker is clustered into the cluster with that cluster ID. Then, the tracker clustering unit 104A stores the cluster ID of the new tracker and the corresponding tracker information as tracker cluster information in the tracker cluster information storage unit (S312).

[0135] On the other hand, if the cluster IDs of the searched similar trackers are not all the same (NO in S320), the tracker clustering unit 104A integrates the cluster IDs of the search results and reflects the integrated cluster ID in the tracker cluster information storage unit 106A (step S322).Then, the tracker clustering unit 104A stores the cluster ID of the new tracker and the corresponding tracker information as tracker cluster information in the tracker cluster information storage unit (S312).

[0136] That is, if the cluster IDs of the searched similar trackers are not all the same, the tracker clustering unit 104A determines that all of the trackers belonging to these clusters belong to the same cluster. For example, if the cluster IDs of the searched similar trackers are ID=#1, #2, the tracker clustering unit 104A determines that the trackers belonging to these clusters and the new tracker belong to the same cluster (ID=#3). That is, for example, tracker A and tracker B are similar to each other and belong to the same cluster (ID=#1), and tracker C is not similar to tracker A and tracker B and belongs to a different cluster (ID=#2). In this case, if the new tracker D is similar to trackers A, B, and C, trackers A, B, C, and D will belong to the same cluster (ID=#3).

[0137] FIG. 21 is a diagram for explaining the processing of the tracked object clustering unit 104A according to the second embodiment. FIG. 21 shows an example of clustering tracked objects U1 to U4. First, even if the tracked object clustering unit 104A executes the processing of S306 for tracked object U1, no similar tracked object is searched for in the tracked object cluster information storage unit 106A. This is because nothing is stored in the tracked object cluster information storage unit 106A. Therefore, the tracked object clustering unit 104A assigns a new ID=#1 to tracked object U1 (S310). Then, the tracked object clustering unit 104A associates the tracked object information of tracked object U1 with the cluster ID=#1 and stores them in the tracked object cluster information storage unit 106A (S312).

[0138] Next, when the tracker clustering unit 104A executes the process of S306 for the tracker U2, it searches for the tracker U1 as a similar tracker. At this time, the number of similar trackers found is equal to or greater than the threshold Th2 (=1) (YES in S308), and the cluster IDs of the found similar trackers are all the same (ID=#1) (YES in S320). Therefore, the tracker clustering unit 104A assigns ID=#1, which is the cluster ID, to the tracker U2. Then, the tracker clustering unit 104A stores the tracker information of the tracker U2 and the cluster ID=#1 in association with each other in the tracker cluster information storage unit 106A (S312).

[0139] Next, even if the tracked object clustering unit 104A executes the process of S306 for tracked object U3, because tracked object U3 is not similar to tracked objects U1 and U2, no similar tracked objects are searched for in the tracked object cluster information storage unit 106A. Therefore, the tracked object clustering unit 104A assigns a new ID=#2 to tracked object U3 (S310). Then, the tracked object clustering unit 104A associates the tracked object information of tracked object U3 with the cluster ID=#2 and stores them in the tracked object cluster information storage unit 106A (S312).

[0140] Next, even if the tracked object clustering unit 104A executes the process of S306 for tracked object U4, because tracked object U4 is not similar to tracked objects U1, U2, and U3, no similar tracked objects are searched for in the tracked object cluster information storage unit 106A. Therefore, the tracked object clustering unit 104A assigns a new ID=#3 to tracked object U4 (S310). Then, the tracked object clustering unit 104A associates the tracked object information of tracked object U4 with the cluster ID=#3 and stores them in the tracked object cluster information storage unit 106A (S312).

[0141] In this way, tracked object cluster information indicating that tracked objects U1, U2, U3, and U4 have been clustered into the above-mentioned clusters is stored in tracked object cluster information storage unit 106A. That is, tracked object cluster information regarding the cluster with ID=#1 indicates that tracked objects U1 and U2 belong to the cluster with ID=#1. Furthermore, tracked object cluster information regarding the cluster with ID=#2 indicates that tracked object U3 belongs to the cluster with ID=#2. Furthermore, tracked object cluster information regarding the cluster with ID=#3 indicates that tracked object U4 belongs to the cluster with ID=#3.

[0142] FIG. 22 is a diagram illustrating an example of tracked object information stored in the tracked object information storage unit 102A according to the second embodiment. FIG. 23 is a diagram illustrating a state in which the tracked object information stored in the tracked object information storage unit 102A according to the second embodiment is clustered. In the example of FIG. 22, tracked object information 70A to 70D related to tracked objects A to D is stored in the tracked object information storage unit 102A. Then, by processing of the tracked object clustering unit 104A, tracked object information 70A and 70B related to tracked objects A and B are clustered into cluster #1, which is a collection of tracked objects considered to be the same (similar). Similarly, tracked object information 70C and 70D related to tracked objects C and D are clustered into cluster #2, which is a collection of tracked objects considered to be the same (similar).

[0143] The tracked object cluster information storage unit 106A stores tracked object cluster information showing the state illustrated in Fig. 23. The tracked object cluster information may include tracked object information regarding tracked objects belonging to each cluster. In the example of Fig. 23, the tracked object cluster information regarding cluster #1 may include tracked object information 70A regarding tracked object A and tracked object information 70B regarding tracked object B. The tracked object cluster information regarding cluster #2 may include tracked object information 70C regarding tracked object C and tracked object information 70D regarding tracked object D.

[0144] 23, tracked object information 70A includes tracked object data A1 to A8. Similarly, tracked object information 70B includes tracked object data B1 to B8. Tracked object information 70C includes tracked object data C1 to C8. Tracked object information 70D includes tracked object data D1 to D8.

[0145] The pseudo-correct tracking body pair information generation unit 108A (FIG. 18) generates pseudo-correct tracking body pair information using the tracker cluster information stored in the tracker cluster information storage unit 106A. The pseudo-correct tracking body pair information is pseudo information of the correct tracking body pair information according to the first embodiment. Specifically, the pseudo-correct tracking body pair information generation unit 108A generates pseudo-correct tracking body pair information corresponding to the same correct tracking body pair information or pseudo-correct tracking body pair information corresponding to the different correct tracking body pair information. "Pseudo-correct tracking body pair information corresponding to the same correct tracking body pair information" (pseudo-same correct tracking body pair information) corresponds to a set of tracker information of trackers that are considered to be identical to each other. "Pseudo-correct tracking body pair information corresponding to different correct tracking body pair information" (pseudo-different correct tracking body pair information) corresponds to a set of tracker information of trackers that are considered to be different from each other. The pseudo-correct tracking body pair information storage unit 110A stores the generated pseudo-correct tracking body pair information. Then, the correct weight generation unit 120 uses this pseudo correct tracking body pair information as correct tracking body pair information to generate correct weights in a method substantially similar to the method described above (the method shown in FIG. 10).

[0146] As described above, the same correct tracked body pair information according to the first embodiment is generated using tracked body information relating to the same tracked body for sure. In contrast, the "pseudo-correct tracked body pair information corresponding to the same correct tracked body pair information" can be generated using tracked body information relating to similar tracked bodies (tracked bodies considered to be the same), rather than tracked body information relating to the same tracked body for sure. Also, as described above, the different correct tracked body pair information according to the first embodiment is generated using tracked body information relating to different tracked bodies for sure. In contrast, the "pseudo-correct tracked body pair information corresponding to the different correct tracked body pair information" can be generated using tracked body information relating to dissimilar tracked bodies (tracked bodies considered to be different from each other), rather than tracked body information relating to different tracked bodies for sure.

[0147] Furthermore, the pseudo-correct tracking body pair information generation unit 108A may generate pseudo-correct tracking body pair information corresponding to the same correct tracking body pair information using tracker cluster information including tracker information related to a predetermined number or more of trackers. Furthermore, the pseudo-correct tracking body pair information generation unit 108A may calculate a matching score between each piece of tracker information corresponding to the first tracker cluster information and each piece of tracker information corresponding to second tracker cluster information different from the first tracker cluster information. Then, the pseudo-correct tracking body pair information generation unit 108A may generate pseudo-correct tracking body pair information corresponding to the different correct tracking body pair information using a pair of the first tracker cluster information and the second tracker cluster information such that the maximum value of the matching score is equal to or less than a predetermined threshold. Details will be described later.

[0148] 24 and 25 are flowcharts showing the processing of the pseudo-correct tracking body pair information generation unit 108A according to the second embodiment. FIGS. 24 and 25 correspond to the processing of S4A in FIG. 19. FIG. 24 shows the processing for generating "pseudo-correct tracking body pair information corresponding to same correct tracking body pair information." FIG. 25 shows the processing for generating "pseudo-correct tracking body pair information corresponding to different correct tracking body pair information."

[0149] First, Fig. 24 will be described. The pseudo-correct tracked body pair information generation unit 108A acquires clusters in which the number of tracked bodies belonging to the same cluster is equal to or greater than a predetermined threshold Th3 (step S332). The threshold Th3 is a threshold that represents the lower limit of the number of tracked bodies belonging to the same cluster. The threshold Th3 is an integer equal to or greater than 1. Specifically, the pseudo-correct tracked body pair information generation unit 108A determines whether there is a cluster in which the number of tracked bodies (tracked body information) assigned the same cluster ID is equal to or greater than the threshold Th3. Then, the pseudo-correct tracked body pair information generation unit 108A acquires that cluster.

[0150] The pseudo-correct tracked body pair information generation unit 108A registers all tracked body pairs that can be obtained in the same cluster as same-correct tracked body pairs in the pseudo-correct tracked body pair information storage unit 110A for the acquired cluster (step S334). Specifically, the pseudo-correct tracked body pair information generation unit 108A regards tracked body pairs obtained from all combinations of tracked bodies belonging to the acquired cluster as same-correct tracked body pairs. For example, if the acquired cluster includes tracked bodies A, B, and C, the pseudo-correct tracked body pair information generation unit 108A regards the pair of tracked body A and tracked body B, the pair of tracked body A and tracked body C, and the pair of tracked body B and tracked body C as same-correct tracked body pairs. Then, the pseudo-correct tracked body pair information generation unit 108A generates same-correct tracked body pair information as shown in FIG. 8 using tracked body information related to the tracked bodies that make up the acquired same-correct tracked body pairs. The pseudo-correct tracking body pair information generating unit 108A stores the generated identical correct tracking body pair information as pseudo-correct tracking body pair information in the pseudo-correct tracking body pair information storage unit 110A.

[0151] 26 is a diagram illustrating pseudo-correct tracking entity pair information corresponding to identical correct tracking entity pair information according to embodiment 2. Fig. 26 illustrates pseudo-correct tracking entity pair information corresponding to identical correct tracking entity pair information obtained using cluster #1 and cluster #2 illustrated in Fig. 23.

[0152] For example, let the threshold value Th3 be 2. In the example of FIG. 23, both cluster #1 and cluster #2 contain two pieces of tracked body information. Therefore, the pseudo-correct tracked body pair information generation unit 108A acquires cluster #1 and cluster #2. Then, for cluster #1, the pseudo-correct tracked body pair information generation unit 108A sets the pair of tracked body A and tracked body B as the same correct tracked body pair. Therefore, the pseudo-correct tracked body pair information generation unit 108A generates same correct tracked body pair information including the pair of tracked body information 70A regarding tracked body A and tracked body information 70B regarding tracked body B. Furthermore, for cluster #2, the pseudo-correct tracked body pair information generation unit 108A sets the pair of tracked body C and tracked body D as the same correct tracked body pair. Therefore, the pseudo-correct tracked body pair information generation unit 108A generates identical correct tracked body pair information including a pair of tracked body information 70C regarding tracked body C and tracked body information 70D regarding tracked body D. As a result, the pseudo-correct tracked body pair information generation unit 108A generates pseudo-correct tracked body pair information indicating a pair of tracked body information 70A and tracked body information 70B, and a pair of tracked body information 70C and tracked body information 70D, as exemplified in FIG.

[0153] Next, Fig. 25 will be described. The pseudo-correct tracked body pair information generation unit 108A acquires cluster pairs in which the maximum value of the matching score between tracked bodies across clusters is equal to or less than a threshold Th4 (step S342). The threshold Th4 is a threshold that represents the upper limit of the matching score at which a pair of tracked bodies are determined to be separate tracked bodies. Specifically, the pseudo-correct tracked body pair information generation unit 108A uses the tracked body cluster information stored in the tracked body cluster information storage unit 106A to extract all possible combinations of clusters as cluster pairs.

[0154] Then, the pseudo-correct tracked entity pair information generation unit 108A calculates a matching score between tracked entities across clusters for each of the extracted cluster pairs. Specifically, the pseudo-correct tracked entity pair information generation unit 108A calculates a matching score between each piece of tracked entity information included in the tracked entity cluster information for one cluster of the cluster pair and each piece of tracked entity information included in the tracked entity cluster information for the other cluster. That is, the pseudo-correct tracked entity pair information generation unit 108A calculates a matching score for all combinations of each piece of tracked entity information in the tracked entity cluster information for one cluster and each piece of tracked entity information in the tracked entity cluster information for the other cluster. The matching score may be calculated using, for example, the above formula (2). Note that by performing S306 in FIG. 20 described above, matching scores are calculated for all combinations of tracked entity information stored in the tracked entity information storage unit 102A. Therefore, by storing the matching scores between tracked entities calculated in the processing of S306, it is not necessary to calculate the matching score in the processing of S342.

[0155] For example, suppose that tracked entities A1, A2, and A3 belong to one cluster A of a cluster pair, and tracked entities B1 and B2 belong to the other cluster B. In this case, the pseudo-correct tracked entity pair information generation unit 108A calculates a matching score between tracked entities A1 and B1, and a matching score between tracked entities A1 and B2. Similarly, the pseudo-correct tracked entity pair information generation unit 108A calculates a matching score between tracked entities A2 and B1, and a matching score between tracked entities A2 and B2. Similarly, the pseudo-correct tracked entity pair information generation unit 108A calculates a matching score between tracked entities A3 and B1, and a matching score between tracked entities A3 and B2.

[0156] Then, the pseudo-correct tracking body pair information generation unit 108A determines whether the maximum value of the calculated matching score for each cluster pair is equal to or less than a threshold value Th4. Here, if the maximum value of the matching score is equal to or less than the threshold value Th4, it means that all the tracking bodies belonging to one cluster that constitutes the cluster pair and all the tracking bodies belonging to the other cluster are likely to be different tracking bodies. Therefore, the pseudo-correct tracking body pair information generation unit 108A acquires cluster pairs whose maximum value of the matching score is equal to or less than the threshold value Th4. Then, the pseudo-correct tracking body pair information generation unit 108A uses the acquired cluster pairs to generate different correct tracking body pair information in the next process (S344).

[0157] The pseudo-correct tracked body pair information generation unit 108A registers all possible tracked body pairs between the two clusters of the acquired cluster pair as different correct tracked body pairs in the pseudo-correct tracked body pair information storage unit 110A (step S344). Specifically, the pseudo-correct tracked body pair information generation unit 108A defines all combinations of tracked body pairs, each of which is a tracked body belonging to one cluster of the cluster pair and a tracked body belonging to the other cluster, as different correct tracked body pairs. For example, suppose that tracked bodies A1 and A2 belong to one cluster A of the acquired cluster pair, and tracked bodies B1 and B2 belong to the other cluster B. In this case, the pseudo-correct tracked body pair information generation unit 108A defines the pair of tracked body A1 and tracked body B1, the pair of tracked body A1 and tracked body B2, the pair of tracked body A2 and tracked body B1, and the pair of tracked body A2 and tracked body B2 as different correct tracked body pairs. Then, the pseudo-correct tracking body pair information generation unit 108A uses the tracking body information related to the tracking bodies that make up the obtained separate correct tracking body pair to generate separate correct tracking body pair information as exemplified in Fig. 9. The pseudo-correct tracking body pair information generation unit 108A stores the generated separate correct tracking body pair information in the pseudo-correct tracking body pair information storage unit 110A as pseudo correct tracking body pair information.

[0158] FIG. 27 is a diagram illustrating pseudo-correct tracking entity pair information corresponding to the separate correct tracking entity pair information according to the second embodiment. FIG. 27 illustrates pseudo-correct tracking entity pair information corresponding to the separate correct tracking entity pair information obtained using cluster #1 and cluster #2 illustrated in FIG. 23. The pseudo-correct tracking entity pair information generation unit 108A calculates a matching score between tracking entity information 70A related to cluster #1 and each of tracking entity information 70C and 70D related to cluster #2. The pseudo-correct tracking entity pair information generation unit 108A also calculates a matching score between tracking entity information 70B related to cluster #1 and each of tracking entity information 70C and 70D related to cluster #2. Then, it is assumed that the maximum value of the calculated matching scores is equal to or less than threshold Th4. Therefore, separate correct tracking entity pair information is generated using the cluster pair of cluster #1 and cluster #2.

[0159] The pseudo-correct tracked body pair information generation unit 108A sets the pair of tracked body A belonging to cluster #1 and tracked body C belonging to cluster #2 as a different correct tracked body pair. Therefore, the pseudo-correct tracked body pair information generation unit 108A generates different correct tracked body pair information including tracked body information 70A regarding tracked body A and tracked body information 70C regarding tracked body C.

[0160] Furthermore, the pseudo-correct tracked body pair information generation unit 108A sets the pair of tracked body A belonging to cluster #1 and tracked body D belonging to cluster #2 as a different correct tracked body pair. Therefore, the pseudo-correct tracked body pair information generation unit 108A generates different correct tracked body pair information including tracked body information 70A regarding tracked body A and tracked body information 70D regarding tracked body D.

[0161] Furthermore, the pseudo-correct tracked body pair information generation unit 108A sets the pair of tracked body B belonging to cluster #1 and tracked body C belonging to cluster #2 as a separate correct tracked body pair. Therefore, the pseudo-correct tracked body pair information generation unit 108A generates separate correct tracked body pair information including tracked body information 70B regarding tracked body B and tracked body information 70C regarding tracked body C.

[0162] Furthermore, the pseudo-correct tracked body pair information generation unit 108A sets the pair of tracked body B belonging to cluster #1 and tracked body D belonging to cluster #2 as a different correct tracked body pair. Therefore, the pseudo-correct tracked body pair information generation unit 108A generates different correct tracked body pair information including tracked body information 70B regarding tracked body B and tracked body information 70D regarding tracked body D.

[0163] As a result, the pseudo-correct tracked entity pair information generation unit 108A generates pseudo-correct tracked entity pair information indicating a pair of tracked entity information 70A and tracked entity information 70C, as exemplified in Fig. 27. Similarly, the pseudo-correct tracked entity pair information generation unit 108A generates pseudo-correct tracked entity pair information including a pair of tracked entity information 70D and tracked entity information 70B, a pair of tracked entity information 70A and tracked entity information 70D, and a pair of tracked entity information 70C and tracked entity information 70B.

[0164] As described above, the learning device 100A according to the second embodiment is configured to generate pseudo-correct tracked body pair information using one or more tracked body cluster information obtained by clustering tracked body information about multiple tracked bodies that are considered to be identical to each other. That is, the learning device 100A according to the second embodiment is configured to generate pseudo-correct tracked body pair information that is a set of tracked body information about tracked bodies that are considered to be identical to each other or a set of tracked body information about tracked bodies that are considered to be distinct from each other. The learning device 100A according to the second embodiment is then configured to generate correct weights using this pseudo-correct tracked body pair information as correct tracked body pair information.

[0165] This eliminates the need to prepare correct tracking body pair information in advance, as in the first embodiment. Therefore, self-supervised learning of the inference model can be realized. Therefore, it is possible to reduce the complexity of creating training data (correct tracking body pair information) when training the inference model. Furthermore, the tracking body information that constitutes the pseudo correct tracking body pair information is composed of tracking body data including feature information. This tracking body information does not need to include image data. Therefore, the capacity of the pseudo correct tracking body pair information can be reduced compared to training data that includes image data. Therefore, it is possible to perform low-load self-supervised learning.

[0166] Furthermore, the learning device 100A according to the second embodiment is configured to generate pseudo-correct tracked body pair information corresponding to identical correct tracked body pair information using tracked body cluster information including tracked body information related to a predetermined number or more of tracked bodies. "Tracker cluster information including tracked body information related to a predetermined number or more of tracked bodies" corresponds to a large cluster, i.e., a cluster including many tracked bodies. Here, when the cluster size is small, the tracked bodies belonging to the cluster are more likely to be different from each other than when the cluster size is large. Therefore, by using tracked body cluster information related to a cluster including a predetermined number or more of tracked bodies, it is possible to generate pseudo-correct tracked body pair information corresponding to identical correct tracked body pair information with high accuracy. In other words, it is possible to generate pseudo-correct tracked body pair information including pairs of tracked body information related to tracked bodies that are likely to be the same tracked body.

[0167] Furthermore, the learning device 100A according to the second embodiment is configured to calculate a matching score between each piece of tracked object information corresponding to the first tracked object cluster information and each piece of tracked object information corresponding to the second tracked object cluster information. The learning device 100A according to the second embodiment is configured to generate pseudo-correct tracked object pair information corresponding to different correct tracked object pair information using a pair of the first tracked object cluster information and the second tracked object cluster information such that the maximum matching score is equal to or less than a threshold. Here, a "pair of the first tracked object cluster information and the second tracked object cluster information such that the maximum matching score is equal to or less than a threshold" corresponds to a cluster pair to which tracked objects are likely to belong that are distinct from each other. Therefore, by using the tracked object cluster information of such a cluster pair, it is possible to accurately generate pseudo-correct tracked object pair information corresponding to different correct tracked object pair information. In other words, it is possible to generate pseudo-correct tracked object pair information including a pair of tracked object information related to tracked objects that are likely to be distinct from each other.

[0168] Although the learning device 100A according to the second embodiment generates pseudo-correct tracked body pair information using tracked body information that does not include tracked body data weights, the present invention is not limited to this configuration. The learning device 100A may generate pseudo-correct tracked body pair information using weighted tracked body information generated by the matching device 200. In this case, when weighted tracked body information is generated by the weight inference unit 220 of the matching device 200 for tracked body information related to the tracked body to be matched, the learning device 100A acquires the weighted tracked body information and stores it in the tracked body information storage unit 102A. Then, the learning device 100A may perform clustering of the tracked bodies using the weighted tracked body information (S2A in FIG. 19 ) and generate pseudo-correct tracked body pair information (S4A in FIG. 19 ).

[0169] In this case, a tracked object data weight is added to each tracked object data of the tracked object information included in the pseudo-correct tracked object pair information. Therefore, the tracked object clustering unit 104A may use the above formula (1) when calculating the matching score in the process of S306 in FIG. 20. Similarly, the pseudo-correct tracked object pair information generation unit 108A may use the above formula (1) when calculating the matching score in the process of S342 in FIG. 25. This allows a more accurate matching score to be calculated compared to the case where formula (2) is used, so the processes of S306 and S342 can be performed with high accuracy. Therefore, the possibility that a pair of tracked objects related to the same correct tracked object pair information in the pseudo-correct tracked object pair information are actually the same tracked object increases. Similarly, the possibility that a pair of tracked objects related to the different correct tracked object pair information in the pseudo-correct tracked object pair information are actually different tracked objects increases.

[0170] (Variation) The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit of the present invention. For example, the order of the processes in the above-described flowcharts can be modified as appropriate. Furthermore, one or more of the processes in the above-described flowcharts may be omitted.

[0171] The above-mentioned program includes a set of instructions (or software code) that, when loaded into a computer, causes the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable media or tangible storage media include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disk (DVD), Blu-ray® disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable media or communication media include electrical, optical, acoustic, or other forms of propagated signals.

[0172] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the invention.

[0173] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. (Appendix 1) a correct answer weight generating means for generating a correct answer weight corresponding to correct answer data of a tracked object data weight relating to importance indicating how well the tracked object data represents the characteristics of the tracked object corresponding to the tracked object in the tracked object information, using correct answer tracked object pair information which is a set of the tracked object information of the same tracked object or a set of the tracked object information of different tracked objects, for each of the tracked object data of the tracked object information which includes at least feature amount information indicating the characteristics of the tracked object which is the object to be tracked and includes one or more tracked object data obtained by tracking the tracked object using video; an inference model learning means for learning, by machine learning, an inference model that uses data relating to the tracked object information as input data, the correct answer weight generated for the tracked object information as correct answer data, and outputs a tracked object data weight corresponding to the tracked object data included in the tracked object information; and the correct answer weight generation means generates the tracked body data weight to be used in association with a similarity between tracked body data included in the tracked body information related to a first tracked body of the pair of tracked bodies and tracked body data included in the tracked body information related to a second tracked body when calculating a tracked body matching score, which is a matching score of the pair of tracked bodies, in a matching process of the pair of tracked bodies; Learning device. (Appendix 2) the correct weight generating means generates a correct weight for the tracked body data based on a similarity between each of the tracked body data included in the tracked body information of one of the tracked bodies in each of the plurality of correct tracked body pair information and each of the tracked body data included in the tracked body information of the other tracked body; 2. The learning device of claim 1. (Appendix 3) the correct answer weight generation means assigns points to the tracked object data based on the calculated similarity, and generates a correct answer weight for the tracked object data in accordance with the number of the assigned points. 3. The learning device according to claim 2. (Appendix 4) the correct answer weight generation means assigns points to the tracked body data corresponding to the highest similarity among the similarities calculated using a pair of tracked body information of the same tracked body in the correct tracked body pair information; 4. The learning device according to claim 3. (Appendix 5) the correct weight generation means assigns points to the tracked body data corresponding to the lowest similarity among the similarities calculated using the set of tracked body information of separate tracked bodies in the correct tracked body pair information; 5. The learning device according to claim 3 or 4. (Appendix 6) a pseudo-correct tracked body pair information generating means for generating pseudo-correct tracked body pair information, which is a set of tracked body information of tracked bodies considered to be identical to each other or a set of tracked body information of tracked bodies considered to be distinct from each other, using one or more tracked body cluster information obtained by clustering the tracked body information regarding a plurality of tracked bodies considered to be identical to each other; and the correct weight generation means uses the pseudo correct tracking body pair information as the correct tracking body pair information to generate the correct weight; 6. A learning device according to any one of appendices 1 to 5. (Appendix 7) the pseudo-correct tracked body pair information generating means uses the tracked body cluster information including the tracked body information relating to a predetermined number or more of tracked bodies to generate the pseudo-correct tracked body pair information, which is a set of the tracked body information of tracked bodies that are considered to be identical to each other; 7. The learning device according to claim 6. (Appendix 8) the pseudo-correct tracker pair information generation means generates pseudo-correct tracker pair information, which is a set of tracker information of trackers considered to be separate from each other, using a set of the first tracker cluster information and the second tracker cluster information such that a maximum value of a matching score calculated between each of the tracker information corresponding to the first tracker cluster information and each of the tracker information included in second tracker cluster information different from the first tracker cluster information is equal to or less than a predetermined threshold value; 8. The learning device according to claim 6 or 7. (Appendix 9) an input data designation means for designating the elements of the input data to be input to the inference model; 9. The learning device according to any one of claims 1 to 8, further comprising: (Appendix 10) the inference model learning means learns the inference model using, as input data, at least graph structure data indicating a similarity relationship between the plurality of tracked object data included in the tracked object information; 10. The learning device according to any one of appendices 1 to 9. (Appendix 11) a weight inference means for inferring tracked body data weights corresponding to the tracked body data included in the tracked body information of each of a pair of tracked bodies to be compared, using an inference model that has been trained in advance by machine learning, the inference model being trained to output tracked body data weights corresponding to the tracked body data included in the tracked body information of each of a pair of tracked bodies to be compared, using as input data data relating to tracked body information including at least feature amount information indicating the features of the tracked body, which is an object to be tracked, and including one or more tracked body data obtained by tracking the tracked body using video, and using as correct answer data a correct answer weight corresponding to the tracked body data weight related to the importance indicating how well the tracked body data represents the features of the tracked body corresponding to the tracked body in the tracked body information; a tracker matching means for performing a matching process for the pair of tracked objects by associating a similarity between tracked object data included in the tracked object information related to a first tracked object of the pair of tracked objects and tracked object data included in the tracked object information related to a second tracked object with the inferred tracked object data weight, and calculating a tracked object matching score which is a matching score for the pair of tracked objects; A matching device having the above configuration. (Appendix 12) the weight inference means uses, as input data, graph structure data indicating a similarity relationship between the plurality of pieces of tracked object data included in the tracked object information, and infers the tracked object data weight using the inference model; 12. The verification device of claim 11. (Appendix 13) For each piece of tracked body data in tracked body information that includes at least feature amount information indicating the features of a tracked body, which is an object to be tracked, and includes one or more pieces of tracked body data obtained by tracking the tracked body using video, generate a correct answer weight corresponding to the correct answer data of tracked body data weights, which are related to the importance of how well the tracked body data represents the features of the corresponding tracked body in the tracked body information, using correct answer tracked body pair information, which is a set of tracked body information of the same tracked body or a set of tracked body information of different tracked bodies; learning an inference model by machine learning, which uses data related to the tracked object information as input data, and uses the correct answer weight generated for the tracked object information as correct answer data, and outputs a tracked object data weight corresponding to the tracked object data included in the tracked object information; When calculating a tracked body matching score, which is a matching score of a pair of tracked bodies in a matching process of a pair of tracked bodies, the tracked body data weight is used in association with a similarity between tracked body data included in the tracked body information related to a first tracked body of the pair of tracked bodies and tracked body data included in the tracked body information related to a second tracked body. How to learn. (Appendix 14) generating a correct answer weight for the tracked body data based on a similarity between each of the tracked body data included in the tracked body information of one of the tracked bodies in each of the plurality of correct tracked body pair information and each of the tracked body data included in the tracked body information of the other tracked body; Study method described in Appendix 13. (Appendix 15) assigning points to the tracked object data based on the calculated similarity, and generating a correct answer weight for the tracked object data in accordance with the number of the assigned points; Study methods described in Appendix 14. (Appendix 16) A point is assigned to the tracked body data corresponding to the highest similarity among the similarities calculated using the pair of tracked body information of the same tracked body in the correct tracked body pair information. Study method described in Appendix 15. (Appendix 17) A point is assigned to the tracked body data corresponding to the lowest similarity among the similarities calculated using the set of tracked body information of the separate tracked bodies in the correct tracked body pair information. 17. A learning method as set forth in Appendix 15 or 16. (Appendix 18) Using one or more pieces of tracker cluster information obtained by clustering the tracker information regarding a plurality of trackers that are deemed to be identical to one another, pseudo-correct tracker pair information is generated, which is a set of the tracker information of the trackers that are deemed to be identical to one another or a set of the tracker information of the trackers that are deemed to be different from one another; generating the correct weights using the pseudo correct tracked body pair information as the correct tracked body pair information; 18. A learning method according to any one of appendices 13 to 17. (Appendix 19) generating the pseudo-correct tracked body pair information, which is a set of the tracked body information of tracked bodies that are regarded as identical to each other, using the tracked body cluster information that includes the tracked body information related to a predetermined number or more of tracked bodies; Study methods described in Appendix 18. (Appendix 20) generate pseudo-correct tracker pair information, which is a set of tracker information of trackers considered to be distinct from each other, using a set of the first tracker cluster information and the second tracker cluster information such that a maximum value of a matching score calculated between each of the tracker information corresponding to the first tracker cluster information and each of the tracker information included in second tracker cluster information different from the first tracker cluster information is equal to or less than a predetermined threshold value; 19. A learning method as described in Appendix 18 or 19. (Appendix 21) Specifying elements of the input data to be input into the inference model; 21. A learning method according to any one of appendices 13 to 20. (Appendix 22) The inference model is trained using, as input data, at least graph structure data indicating a similarity relationship between the plurality of pieces of tracked object data included in the tracked object information. 22. A learning method according to any one of appendices 13 to 21. (Appendix 23) an inference model that has been trained in advance by machine learning, which uses as input data data relating to tracked body information including at least feature amount information indicating the characteristics of a tracked body, which is an object to be tracked, and one or more pieces of tracked body data obtained by tracking the tracked body using video, and uses as correct answer data a correct answer weight corresponding to correct answer data of a tracked body data weight related to the importance indicating how well the tracked body data represents the characteristics of the tracked body corresponding to the tracked body in the tracked body information, and infers tracked body data weights corresponding to each of the tracked body data included in the tracked body information for each of a pair of tracked bodies to be matched, using the inference model that has been trained to output tracked body data weights corresponding to the tracked body data included in the tracked body information for each of a pair of tracked bodies to be matched, a matching process of the pair of tracked bodies is performed by associating the similarity between the tracked body data included in the tracked body information related to the first tracked body of the pair of tracked bodies and the tracked body data included in the tracked body information related to the second tracked body with the inferred tracked body data weight, and calculating a tracked body matching score which is a matching score of the pair of tracked bodies; Matching method. (Appendix 24) Inferring the tracked object data weight using the inference model, at least, graph structure data indicating a similarity relationship between the plurality of tracked object data included in the tracked object information as the input data. The matching method described in Appendix 23. (Appendix 25) A non-transitory computer-readable medium storing a program that causes a computer to execute the learning method described in any one of appendices 13 to 22. (Appendix 26) A non-transitory computer-readable medium storing a program that causes a computer to execute the matching method described in Appendix 23 or 24. [Explanation of symbols]

[0174] 10 Learning Device 12 Correct weight generation unit 14 Inference model learning unit 20. Verification device 22 Weight inference unit 24 Tracking object matching unit 50 Matching System 100,100A Learning Device 102A Tracking object information storage unit 104A Tracker Clustering Unit 106A Tracking object cluster information storage unit 108A Pseudo-correct tracking body pair information generation unit 110 Correct Tracking Body Pair Information Storage Unit 110A Pseudo-correct tracking body pair information storage unit 120 Correct weight generation unit 130 Correct tracking weight information storage section 140 Inference Model Learning Unit 150 Inference model storage unit 160 Input data specification section 200 Collation Device 202 Inference model storage unit 210 Tracking object information acquisition unit 220 Weight Inference Unit 240 Tracking Body Matching Unit

Claims

1. a correct answer weight generating means for generating a correct answer weight corresponding to correct answer data of a tracked object data weight relating to importance indicating how well the tracked object data represents the characteristics of the tracked object corresponding to the tracked object in the tracked object information, using correct answer tracked object pair information which is a set of the tracked object information of the same tracked object or a set of the tracked object information of different tracked objects, for each of the tracked object data of the tracked object information which includes at least feature amount information indicating the characteristics of the tracked object which is the object to be tracked and includes one or more tracked object data obtained by tracking the tracked object using video; an inference model learning means for learning, by machine learning, an inference model that uses data relating to the tracked object information as input data, the correct answer weight generated for the tracked object information as correct answer data, and outputs a tracked object data weight corresponding to the tracked object data included in the tracked object information; and the correct answer weight generation means generates the tracked body data weight to be used in association with a similarity between tracked body data included in the tracked body information related to a first tracked body of the pair of tracked bodies and tracked body data included in the tracked body information related to a second tracked body when calculating a tracked body matching score, which is a matching score of the pair of tracked bodies, in a matching process of the pair of tracked bodies; Learning device.

2. the correct weight generating means generates a correct weight for the tracked body data based on a similarity between each of the tracked body data included in the tracked body information of one of the tracked bodies in each of the plurality of correct tracked body pair information and each of the tracked body data included in the tracked body information of the other tracked body; The learning device according to claim 1 .

3. the correct answer weight generation means assigns points to the tracked object data based on the calculated similarity, and generates a correct answer weight for the tracked object data in accordance with the number of the assigned points. The learning device according to claim 2 .

4. the correct answer weight generation means assigns points to the tracked body data corresponding to the highest similarity among the similarities calculated using a pair of tracked body information of the same tracked body in the correct tracked body pair information; The learning device according to claim 3 .

5. the correct weight generation means assigns points to the tracked body data corresponding to the lowest similarity among the similarities calculated using the set of tracked body information of separate tracked bodies in the correct tracked body pair information; The learning device according to claim 3 or 4.

6. a weight inference means for inferring tracked body data weights corresponding to the tracked body data included in the tracked body information of each of a pair of tracked bodies to be compared, using an inference model that has been trained in advance by machine learning, the inference model being trained to output tracked body data weights corresponding to the tracked body data included in the tracked body information of each of a pair of tracked bodies to be compared, using as input data data relating to tracked body information including at least feature amount information indicating the features of the tracked body that is the object to be tracked and including one or more tracked body data obtained by tracking the tracked body using video, and using as correct answer data a correct answer weight corresponding to correct answer data of a tracked body data weight related to the importance indicating how well the tracked body data represents the features of the tracked body that corresponds to the tracked body in the tracked body information; a tracked object matching means for performing a matching process for the pair of tracked objects by associating a similarity between tracked object data included in the tracked object information related to a first tracked object of the pair of tracked objects and tracked object data included in the tracked object information related to a second tracked object with the inferred tracked object data weight and calculating a tracked object matching score which is a matching score for the pair of tracked objects; A matching device having the above configuration.

7. For each piece of tracked body data in tracked body information that includes at least feature amount information indicating the features of a tracked body, which is an object to be tracked, and includes one or more pieces of tracked body data obtained by tracking the tracked body using video, generate a correct answer weight corresponding to the correct answer data of tracked body data weights, which are related to the importance of how well the tracked body data represents the features of the corresponding tracked body in the tracked body information, using correct answer tracked body pair information, which is a set of tracked body information of the same tracked body or a set of tracked body information of different tracked bodies; learning an inference model by machine learning, which uses data related to the tracked object information as input data, and uses the correct answer weight generated for the tracked object information as correct answer data, and outputs a tracked object data weight corresponding to the tracked object data included in the tracked object information; When calculating a tracked body matching score, which is a matching score of a pair of tracked bodies in a matching process of a pair of tracked bodies, the tracked body data weight is used in association with a similarity between tracked body data included in the tracked body information related to a first tracked body of the pair of tracked bodies and tracked body data included in the tracked body information related to a second tracked body. A computer-implemented learning method.

8. an inference model that has been trained in advance by machine learning, the inference model having as input data data relating to tracked body information including at least feature amount information indicating the characteristics of a tracked body, which is an object to be tracked, and including one or more tracked body data obtained by tracking the tracked body using video, and using as correct answer data a correct answer weight corresponding to the correct answer data of a tracked body data weight related to the importance indicating how well the tracked body data represents the characteristics of the tracked body corresponding to the tracked body in the tracked body information, and outputting a tracked body data weight corresponding to the tracked body data included in the tracked body information relating to the input data, inferring tracked body data weights corresponding to each of the tracked body data included in the tracked body information of each of a pair of tracked bodies to be matched; a matching process for the pair of tracked bodies is performed by associating the similarity between tracked body data included in the tracked body information related to a first tracked body of the pair of tracked bodies and tracked body data included in the tracked body information related to a second tracked body with the inferred tracked body data weight, and calculating a tracked body matching score which is a matching score for the pair of tracked bodies; A computer-implemented matching method.

9. a step of generating a correct answer weight corresponding to correct answer data of a tracked object data weight relating to importance indicating how well the tracked object data represents the characteristics of the tracked object corresponding to the tracked object in the tracked object information, using correct answer tracked object pair information which is a set of the tracked object information of the same tracked object or a set of the tracked object information of different tracked objects, for each tracked object data of the tracked object information which includes at least feature amount information indicating the characteristics of the tracked object which is the object to be tracked, and which includes one or more tracked object data obtained by tracking the tracked object using video; a step of learning, by machine learning, an inference model that uses data related to the tracked object information as input data, the correct answer weight generated for the tracked object information as correct answer data, and outputs a tracked object data weight corresponding to the tracked object data included in the tracked object information; on the computer, When calculating a tracked body matching score, which is a matching score of a pair of tracked bodies in a matching process of a pair of tracked bodies, the tracked body data weight is used in association with a similarity between tracked body data included in the tracked body information related to a first tracked body of the pair of tracked bodies and tracked body data included in the tracked body information related to a second tracked body. program.

10. a step of inferring tracked body data weights corresponding to the tracked body data included in the tracked body information of each of a pair of tracked bodies to be compared, using an inference model that has been trained in advance by machine learning, the inference model being trained to output tracked body data weights corresponding to the tracked body data included in the tracked body information of each of a pair of tracked bodies to be compared, using as input data data related to tracked body information including at least feature amount information indicating the features of the tracked body that is the object to be tracked and including one or more tracked body data obtained by tracking the tracked body using video, and using as correct answer data a correct answer weight corresponding to the tracked body data weight related to the importance indicating how well the tracked body data represents the features of the tracked body that corresponds to the tracked body in the tracked body information, a step of performing a matching process for the pair of tracked objects by associating the similarity between tracked object data included in the tracked object information related to a first tracked object of the pair of tracked objects and tracked object data included in the tracked object information related to a second tracked object with the inferred tracked object data weight, and calculating a tracked object matching score which is a matching score for the pair of tracked objects; A program that causes a computer to execute the following.

Images