Identification method, learned model generation method, identification device, learned model generation device, and program

Abstract

To provide an identification technique capable of identifying an identification object into a learning object and an unlearned object and of identifying which learning object the identification object is when the identification object is a learning object.SOLUTION: A person identification device 2 inputs skeleton information on a person 5 to a learned model of n-value classification, calculates variance information on score values of n labels with which different learners are associated, inputs statistical information on a granularity m calculated from m sets of variance information to a learned model of binary classification, and acquires score values for two labels with which a learner and a non-learner are associated respectively. The person 5 is identified as a learner if the learner is associated with the label of the largest score value, and is identified as a non-learner if the non-learner is associated with the label. When the person is identified as a learner, the person 5 is identified as the learner associated with the label having the largest score value among the score values of n labels obtained by the learned model of the n value classification.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a recognition technology for recognizing an object to be recognized from its feature information using AI (Artificial Intelligence). [Background technology]

[0002] Non-Patent Document 1 discloses a person identification technology using AI that uses skeletal information obtained from a monocular camera.

[0003] In this person identification technology, for each learner, skeletal information is acquired from video data of the learner captured by a monocular camera using an existing human pose estimation method, and the acquired skeletal information is used as input data for the learner to perform machine learning on an AI so that the score value of the label associated with the learner is high. This generates a trained model for n-ary classification, where n is the number of learners. Next, the trained model for n-ary classification generated in this manner is used to perform person identification of the identification target person captured by the monocular camera. That is, using an existing human pose estimation method, skeletal information is acquired from video data of the identification target person captured by the monocular camera, and the acquired skeletal information is input into the trained model for n-ary classification to obtain a score value for each of the n labels associated with each learner. The identification target person is then identified as the learner associated with the label with the largest score value. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Tetsuro Toda, Alessandro Moro, Kazunori Umeda, "Person Identification Using Skeleton Information Obtained from a Monocular Camera," Proceedings of the 2018 Japan Society for Precision Engineering Spring Meeting, D69, 2018, pp. 267-268 Summary of the Invention [Problem to be solved by the invention]

[0005] However, according to the person identification technology described in Non-Patent Document 1, even if the features of the person to be identified captured by a monocular camera are not involved in the generation of a trained model for n-ary classification (if they are not provided as input data for machine learning, i.e., if they are untrained), by inputting skeletal information obtained from the video data of this person to the trained model for n-ary classification, a score value for each of the n labels associated with each learner is calculated, and the person to be identified is identified as the learner associated with the label with the largest score value. In other words, there is a problem in that the person to be identified is identified as a different person.

[0006] The present invention has been made in consideration of the above circumstances, and its purpose is to provide a classification technology that can classify a classification target into a learning target and an unlearned target, and, if the classification target is a learning target, can identify which learning target the classification target is. [Means for solving the problem]

[0007] In order to solve the above problems, the present invention achieves the following functions.

[0008] [Trained model generation process] First, for each learning object, feature information of the learning object is used as input data, and machine learning is performed by AI to increase the score value of the label associated with the learning object, thereby generating a trained model for n-value classification (= number of learning objects) that can identify individual learning objects. Next, the feature information of the learning object used as input data for this trained model is input into the trained model for n-value classification generated as above, and variance information indicating the degree of variance of the score value of each of the n labels that are the output values is calculated, and feature information of untrained objects other than the learning object is input, and variance information indicating the degree of variance of the score value of each of the n labels that are the output values is calculated. Then, for each of the learning object and untrained object, statistical information with a granularity of m calculated from m sets of variance information is used as input data, and machine learning is performed by AI to increase the score value of the label associated with it, thereby generating a trained model for binary classification that can identify learning objects and untrained objects.

[0009] [Identification process] First, feature information of the object to be identified is input into a trained model for n-ary classification generated by the trained model generation step, and variance information indicating the degree of variance of the score values of each of the n labels that are the output values is calculated. Then, statistical information of granularity m calculated from the m sets of variance information is input into a trained model for binary classification generated by the trained model generation step, and score values for each of the two labels to which the training object and untrained object are respectively associated are obtained. If the training object is associated with the label with the largest score value of the two labels, the object to be identified is identified as a training object, and if an untrained object is associated with the training object, the object to be identified is identified as an untrained object. If the object to be identified is identified as a training object, the object to be identified is identified as the training object associated with the label with the largest score value of the n labels obtained by inputting the feature information of the object to be identified into the trained model for n-ary classification generated by the trained model generation step.

[0010] For example, one aspect of the present invention is A method for identifying an object to be identified from feature information of the object to be identified using AI, a variance information calculation step of inputting the feature information of the classification target into a trained model for n-value classification that identifies each of a total of n learning targets, and calculating variance information indicating the degree of variance of the score values of the n labels associated with the different learning targets; a statistical information calculation step of calculating statistical information of granularity m from the m sets of variance information calculated for each piece of feature information of the classification target in the variance information calculation step; a first classification step of inputting the statistical information of granularity m calculated by the statistical information calculation step into a trained model of binary classification that distinguishes between the training object and an untrained object other than the training object, obtaining score values for two labels associated with the training object and the untrained object, and identifying the classification object as the training object if the training object is associated with the label having the largest score value of the two labels, and identifying the classification object as the untrained object if the untrained object is associated with the label; a second classification step of, when the object to be classified is classified as the learning object by the first classification step, classifying the object to be classified as the learning object associated with the label having the largest score value among the score values of the n labels obtained by inputting feature information of the object to be classified into the trained model of n-ary classification, The trained model for the n-ary classification is For each of the learning objects, a total of n, feature information of the learning object is used as input data, and the AI is trained to perform machine learning so that the score value of the label associated with the learning object becomes high; The trained model for the binary classification is The feature information of the learning object used as the input data is input into the trained model for n-value classification to calculate variance information indicating the degree of variance of the score values of the n labels that are the output values, and the feature information of the unlearned object is input into the trained model for n-value classification to calculate variance information indicating the degree of variance of the score values of the n labels that are the output values, and statistical information with a granularity of m calculated from the m sets of variance information calculated for the learning object and the unlearned object is used as input data, and the AI is trained to perform machine learning so that the score values of the labels associated with each of them become higher.

[0011] Another aspect of the present invention is A trained model generation method for generating a trained model for identifying an object to be identified from feature information of the object to be identified using AI, a first trained model generation step of generating a trained model for n-valued classification that identifies each of the learning objects by using feature information of the learning object as input data and having the AI perform machine learning so that the score value of the label associated with the learning object becomes high; and a second trained model generation process for generating a trained model for binary classification that can distinguish between the training object and the untrained object by inputting feature information of the training object used as the input data into the trained model for n-ary classification generated by the first trained model generation process, calculating variance information indicating the degree of variance of the score values of n labels that are the output values, and inputting feature information of untrained objects other than the training object into the trained model for n-ary classification, calculating variance information indicating the degree of variance of the score values of n labels that are the output values, and using statistical information with a granularity of m calculated from m sets of variance information calculated for the training object and the untrained object as input data, and having the AI perform machine learning so that the score values of the labels associated with each of them become higher. [Effects of the Invention]

[0012] The inventors have found that there is a significant difference between the degree of dispersion of the score values of n labels obtained by inputting feature information of the learning objects into a trained model for n-ary classification that distinguishes between a total of n learning objects and the degree of dispersion of the score values of n labels obtained by inputting feature information of unlearned objects other than the learning objects. Therefore, a trained model for binary classification that distinguishes between learning objects and unlearned objects is generated using, as input data, variance information indicating the degree of dispersion of the score values of the learning objects and statistical information with a granularity of m calculated from m sets of variance information indicating the degree of dispersion of the score values of the unlearned objects, and the feature information of the identification objects is input into the trained model for n-ary classification. By inputting into this trained model for binary classification, statistical information with a granularity of m calculated from m sets of variance information indicating the degree of dispersion of the score values of the identification objects obtained by inputting the feature information of the identification objects into the trained model for n-ary classification, the identification object is distinguished as being a learning object or an unlearned object. If the identification object is identified as a learning object, the identification object is identified as being one of the learning objects based on the score value of the identification object. This makes it possible to identify whether an object to be identified is a learning object or an unlearned object, even when the object to be identified is a mixture of learning objects and unlearned objects, and if it is a learning object, to identify which learning object it is. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a schematic diagram of a person identification system according to a first embodiment of the present invention. [Figure 2] FIG. 2 is a schematic functional configuration diagram of the trained model generation device 1. [Figure 3] FIG. 3 is a diagram for explaining the principle of generating a trained model for n-ary classification. [Figure 4] FIG. 4 is a diagram for explaining the principle of generating a trained model for binary classification. [Figure 5] FIG. 5 is a flow diagram illustrating the process of generating a trained model for n-value classification performed by the trained model generation device 1. [Figure 6] FIG. 6 is a flow diagram illustrating the process of generating a trained model for binary classification by the trained model generation device 1. [Figure 7] FIG. 7 is a flow diagram for explaining the learner statistical information calculation process S110 shown in FIG. [Figure 8] FIG. 8 is a flow diagram for explaining the non-learner statistical information calculation process S120 shown in FIG. [Figure 9] FIG. 9 is a flow diagram for explaining the machine learning process S130 shown in FIG. [Figure 10] FIG. 10 is a diagram showing a schematic functional configuration of the person identification device 2. As shown in FIG. [Figure 11] FIG. 11 is a diagram for explaining the principle of distinguishing between learners and non-learners. [Figure 12] FIG. 12 is a diagram for explaining the person identification process performed by the person identification device 2. As shown in FIG. [Figure 13] FIG. 13 is a flow diagram illustrating the process of generating a trained model for n-value classification by the trained model generation device 1a. [Figure 14] FIG. 14 is a flow diagram illustrating the process of generating a trained model for binary classification by the trained model generation device 1a. [Figure 15] FIG. 15 is a diagram for explaining the person identification process performed by the person identification device 2a. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0015] [First embodiment] FIG. 1 is a schematic diagram of a person identification system according to the present embodiment.

[0016] As shown in the figure, the person identification system according to this embodiment includes a trained model generation device 1, a person identification device 2, and a camera 3. These devices 1 to 3 are interconnected by a network 4 such as a WAN (Wide Area Network) or a LAN (Local Area Network).

[0017] The camera 3 is installed at a gate at the entrance of a company, for example, where personal identification is required to pass through, and captures an image of a person 5 about to enter the gate.

[0018] The trained model generation device 1 uses an existing human posture estimation method to estimate skeletal information as feature information of each person 5 (learner) to be trained from video (video) data captured by a camera 3. Then, using the learner's skeletal information and the learner's correct label (teacher data [1, 2, , n]), it generates a trained model (prediction function) for n (= number of learning subjects) value classification that identifies each learner through AI machine learning.

[0019] The trained model generation device 1 also inputs the skeletal information of the learner used to construct the trained model into the trained model for n-ary classification generated as described above, and calculates variance information indicating the degree of variance of the score values of each of the n labels, which are the output values. Furthermore, using an existing human pose estimation method, skeletal information of a person 5 (non-learner) other than the learner is estimated from video data of the camera 3 of this person 5 (non-learner), and this skeletal information is input into the trained model for n-ary classification, and calculates variance information indicating the degree of variance of the score values of each of the n labels, which are the output values. Then, for each of the learner and non-learner, statistical information with a granularity of m is calculated from m sets of variance information, and using this statistical information as input data and the correct labels (teacher data [0,1]) for the learner and non-learner, a trained model for binary classification that distinguishes between learners and non-learners is generated through AI machine learning using the correct labels for the learner and non-learner (teacher data [0,1]). During this binary classification training, the statistical information with a granularity of m serves as the basic unit of input data, and training is performed using a large amount of statistical information to improve training accuracy. For example, if 40 sets of statistical information with a granularity of 20 are used for learning, the required skeletal information will be 800 sets.

[0020] The trained model generation device 1 is usually used when introducing a person identification system, and may be isolated from the person identification system after generating the trained model for n-ary classification and the trained model for binary classification described above.

[0021] The person identification device 2 uses a trained model for n-ary classification and a trained model for binary classification generated by the trained model generation device 1 to identify the person 5, who is the person to be identified, from video data of the person 5, who is the person to be identified, photographed by the camera 3.

[0022] First, the person identification device 2 uses an existing human pose estimation method to acquire m sets of skeletal information from video data of the person to be identified captured by the camera 3, inputs this skeletal information into a trained model for n-ary classification, and calculates m sets of variance information indicating the degree of variance of the score values of each of the n labels, which are the output values. Next, m-granularity statistical information is calculated from the m sets of variance information of the person to be identified calculated as above, and inputs this into a trained model for binary classification to obtain score values for each of the two labels associated with the learner and unlearner, respectively. If the learner is associated with the label with the largest score value, the person to be identified is identified as a learner, and if an unlearner is associated with the learner, the person to be identified is identified as an unlearner. During this binary classification, only one m-granularity statistical information is used. Then, if the person to be identified is identified as a learner, the person to be identified is identified as the learner associated with the label with the largest score value among the score values of each of the n labels obtained by inputting the skeletal information of the person to be identified into a trained model for n-ary classification. When identifying a learner using this n-ary classification, m sets of skeletal information are input, and m sets of classification results are output. If the accuracy during learning is high, all classification results will be nearly identical.

[0023] Note that real-time processing is not required for generating a trained model by the trained model generation device 1, so it is sufficient if the feature information (skeleton information) required for batch processing can be obtained from the camera 3. On the other hand, real-time processing is required for identifying (inferring) a person 5 by the person identification device 2, so it will need to be installed at a gate or the like. For a detailed explanation of generating trained models using machine learning and identifying (inferring) using trained models, see "The Shortest Path to Understanding PyTorch & Deep Learning Programming" by Akaishi Masanori, Nikkei BP, September 21, 2021.

[0024] Next, the trained model generation device 1 and the person identification device 2 that constitute the person identification system according to this embodiment will be described.

[0025] It should be noted that the camera 3 can be an existing camera with a network connection function, and therefore a detailed description thereof will be omitted.

[0026] First, the trained model generation device 1 will be described in detail.

[0027] FIG. 2 is a schematic functional configuration diagram of the trained model generation device 1.

[0028] As shown in the figure, the trained model generation device 1 has a network interface unit 10, a man-machine interface unit 11, a camera image acquisition unit 12, a posture estimation processing unit 13, a machine learning unit 14, and a trained model transmission unit 15.

[0029] The network interface unit 10 is an interface for connecting to the person identification device 2 and the camera 3 via the network 4 .

[0030] The man-machine interface unit 11 is an interface for displaying information to the user and accepting various operations from the user, and has a display device such as an LCD (Liquid Crystal Display) and input devices such as a keyboard and a mouse.

[0031] The camera image acquisition unit 12 acquires image (video) data of the person 5 captured by the camera 3 from the camera 3 via the network interface unit 10.

[0032] The posture estimation processing unit 13 extracts multiple frame images from the video (video) data acquired by the camera image acquisition unit 12 (for example, extracts all frame images that make up the video data), and for each extracted frame image, estimates skeletal information including coordinate information for multiple skeletal parts of the person 5 depicted in the frame image using an existing human posture estimation method. An example of an existing human posture estimation method is AI processing using Google's trained model "CoralPoseNet (trademark)." This human posture estimation method can estimate skeletal information including coordinate information for up to 18 parts: the nose, both eyes, both ears, both shoulders, both elbows, both wrists, left and right waist parts, both knees, both ankles, and the neck.

[0033] The machine learning unit 14 includes an n-ary classification trained model generation unit 140 and a binary classification trained model generation unit 141.

[0034] 3, the n-value classification trained model generation unit 140 uses, for each of the 1st to nth learners, skeletal information 142 acquired by the posture estimation processing unit 13 from the video data of the learner acquired by the camera image acquisition unit 12 and the correct label of the learner (teacher data [1, 2, . . . , n]) to train the AI to increase the score of the label associated with the learner. In this way, it generates a trained model 143 of n-value classification (= number of learners) that identifies the 1st to nth learners.

[0035] As shown in Figure 4, the binary classification trained model generation unit 141 inputs m sets of skeletal information 142 for each of the 1st to nth learners used as input data for constructing the trained model 143 into the trained model 143 for n-value classification generated by the n-value classification trained model generation unit 140, calculates m sets of variance information 144 indicating the degree of variance of the score values of each of the n labels, which are the output values, and further calculates statistical information 145 of granularity m from the m sets of variance information 144 for each learner. Furthermore, for each of the first to kth unlearned individuals (persons 5 other than the learner), skeletal information 146 estimated by the posture estimation processing unit 13 from the video data of the unlearned individual acquired by the camera image acquisition unit 12 is input into the trained model 143 for n-ary classification generated by the n-ary classification trained model generation unit 140, and m sets of variance information 147 indicating the degree of variance of the score values of each of the n labels, which are the output values, are calculated. Furthermore, for each unlearned individual, statistical information 148 with a granularity of m is calculated from the m sets of variance information 147. Note that the granularity indicates the amount of information used to calculate the statistical information, and the statistical information with a granularity of m indicates that it is statistical information calculated from the m sets of variance information. Since statistical information is not used as input data in n-ary classification learning, there is no limitation such as a granularity of m. Specifically, in order to improve learning accuracy using skeletal information as input data, for example, 800 images of skeletal information 142, 146 are used for learning. On the other hand, since statistical information 145 and 148 are used as input data for binary classification learning, the basic unit of input is granularity m. To improve learning accuracy, a large amount of statistical information with granularity m is used. For example, if learning is performed using 40 sets of statistical information with granularity 20, the required skeleton information will be 800 sets. Therefore, to facilitate the learning process, it is desirable that the skeleton information used for n-value classification be an integer multiple of m.

[0036] Furthermore, the binary classification trained model generation unit 141 uses the learner's statistical information 145 and the unlearned's statistical information 148 as input data, and uses the correct answer labels (teacher data [0, 1]) of the learner and the unlearned to train the AI to increase the score values of the labels associated with each of them. In this way, a binary classification trained model 149 that distinguishes between learners and unlearners is generated.

[0037] Increasing the granularity m of the statistical information reduces the variation and improves the prediction accuracy of the trained model for binary classification that distinguishes between learners and non-learners. However, this requires more skeletal information, and therefore, when identifying a person in the person identification device 2, it takes a long time to acquire the necessary skeletal information from the video data of the camera 3. For this reason, the granularity m of the statistical information is determined appropriately, taking into consideration the prediction accuracy of the trained model 149 for binary classification and the time required for person identification in the person identification device 2.

[0038] Furthermore, in this embodiment, the shared information 144, 147 of the learner and unlearner is a score addition value, which is the sum of the maximum and minimum values of the score values of n labels, which are output values of the trained model 143 for n-ary classification, and / or a score subtraction value, which is the difference therebetween. The inventors have found that by using the score addition value and / or the score subtraction value as the shared information 144, 147 of the learner and unlearner, it is possible to distinguish between a known person 5 (learner) and an unknown person 5 (unlearner). In particular, by using the shared information of both the score addition value and the score subtraction value, the difference between the shared information 144 of the learner and the shared information 147 of the unlearner becomes clearer, and the prediction accuracy of the trained model 149 for binary classification is further improved.

[0039] Furthermore, in this embodiment, the average value and standard deviation of m sets of variance information 144, 147 calculated for each skeletal information of the person 5 are used as statistical information of granularity m for the learner's statistical information 145 and the unlearned's statistical information 148, respectively. The inventors have found that learners and unlearners can be distinguished by using the average value and standard deviation calculated by statistical processing from the m sets of variance information 144, 147 of each of the learner and unlearner as statistical information of granularity m. In particular, by using both the average value and standard deviation of the score addition values and the average value and standard deviation of the score subtraction values as statistical information 145, 148, the difference between the learner's statistical information 145 and the unlearned's statistical information 148 becomes clearer, and the prediction accuracy of the trained model 149 for binary classification is improved.

[0040] The trained model transmission unit 15 transmits the trained models (trained model 143 of n-value classification for identifying individual learners and trained model 149 of binary classification for identifying learners and non-learners) generated by the machine learning unit 14 to the person identification device 2 in accordance with a trained model request received from the person identification device 2 via the network interface unit 10.

[0041] 2 is realized in hardware using an integrated logic IC such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array), or in software using a computer such as a DSP (Digital Signal Processor). Alternatively, it is realized as a process in a general-purpose computer such as a personal computer (PC) that includes a CPU (Central Processing Unit) / GPU (Graphics Processing Unit), memory, an auxiliary storage device such as a hard disk or flash memory, and a communication interface such as a network interface card (NIC), by the CPU / GPU loading a predetermined program from the auxiliary storage device into the memory and executing it.

[0042] FIG. 5 is a flow diagram illustrating the process of generating a trained model for n-value classification performed by the trained model generation device 1.

[0043] This flow starts when the machine learning unit 14 receives an operation to generate a trained model for n-ary classification from the user via the man-machine interface unit 11.

[0044] In the machine learning unit 14, the n-value classification trained model generation unit 140 sets a counter value n to 1 (S100). Then, when the n-value classification trained model generation unit 140 receives a learner designation operation involving designation of a label associated with the n-th learner from the user via the man-machine interface unit 11 while the n-th learner is positioned in front of a gate where the camera 3 is installed (S101), it instructs the camera image acquisition unit 12 to acquire video data. In response to this, the camera image acquisition unit 12 acquires video data of the n-th learner passing through the gate, captured by the camera 3 (S102).

[0045] Next, posture estimation processing unit 13 acquires an integer multiple of m sets of skeletal information from the video data of the nth learner acquired by camera image acquisition unit 12 (S103). Specifically, an integer multiple of m frame images are extracted from the video data acquired by camera image acquisition unit 12, and for each extracted frame image, an existing human posture estimation method is used to estimate skeletal information including coordinate information of multiple parts of the skeleton of the nth learner shown in the frame image. In this way, an integer multiple of m sets of skeletal information is acquired for the nth learner.

[0046] Then, n-value classification trained model generation unit 140 sets an integer multiple of m sets of skeletal information acquired by posture estimation processing unit 13 from the video data of the n-th learner as input data for the n-th learner.

[0047] Next, when a new learner is positioned in front of the gate where camera 3 is installed, and the n-ary classification trained model generation unit 140 receives a learner designation operation from the user via the man-machine interface unit 11, involving the designation of a label associated with this new learner (YES in S104), the n-ary classification trained model generation unit 140 increments the counter value n by one (n=n+1) (S105) and instructs the camera image acquisition unit 12 to acquire video data. In response to this, the process returns to S102, and the camera image acquisition unit 12 acquires video (video) data of the nth learner passing through the gate, captured by camera 3. Then, the posture estimation processing unit 13 acquires an integer multiple of m sets of skeletal information from the video data of the nth learner acquired by the camera image acquisition unit 12 (S103).

[0048] On the other hand, when the n-value classification trained model generation unit 140 receives an operation to end the generation of the trained model for n-value classification (YES in S106) without receiving a learner designation operation from the user via the man-machine interface unit 11 (NO in S104), it has obtained an integer multiple of m sets of skeletal information for each of the n learners, and therefore performs machine learning by AI (S107) using this skeletal information as input data, adding a correct label (teacher data [1, 2, . . . , n]), and then ends the generation of the trained model 143 for n-value classification by AI machine learning (S108), thereby terminating this flow.

[0049] FIG. 6 is a flow diagram illustrating the process of generating a trained model for binary classification by the trained model generation device 1.

[0050] This flow starts when the machine learning unit 14 receives an operation to generate a trained model for binary classification from the user via the man-machine interface unit 11 after a trained model 143 for n-value classification has been generated according to the flow shown in Figure 5.

[0051] In the machine learning unit 14, the binary classification trained model generation unit 141 inputs m sets of skeletal information of the learner as input data into the trained model 143 for n-value classification generated by the n-value classification trained model generation unit 140, and performs a learner statistical information calculation process described below on the m sets of output values (S110) to calculate statistical information 145 of the learner with a granularity of m. This process is repeated to generate a large number of pieces of statistical information with a granularity of m. Next, the binary classification trained model generation unit 141 inputs m sets of skeletal information of an unlearned person as input data into the trained model 143 for n-value classification generated by the n-value classification trained model generation unit 140, and performs a unlearned person statistical information calculation process described below on the m sets of output values (S120) to calculate statistical information 148 of the unlearned person. This process is repeated to generate a large number of pieces of statistical information with a granularity of m. Then, the binary classification trained model generation unit 141 performs the machine learning process described below (S130) using the statistical information 145 of a large number of learners calculated by the learner statistical information calculation process S110 and the statistical information 148 of a large number of non-learners calculated by the non-learner statistical information calculation process S120, thereby generating a binary classification trained model 149 that distinguishes between learners and non-learners.

[0052] In the flow shown in Figure 6, the learner statistical information calculation process S110 and the non-learner statistical information calculation process S120 are performed in that order, but they may also be performed in the order of the non-learner statistical information calculation process S120 and the learner statistical information calculation process S110.

[0053] FIG. 7 is a flow diagram for explaining the learner statistical information calculation process S110 shown in FIG.

[0054] When a learner is on standby in front of a gate where a camera 3 is installed, and the binary classification trained model generation unit 141 receives a learner designation operation from the user via the man-machine interface unit 11, which designates a label associated with the learner (S111), the binary classification trained model generation unit 141 instructs the camera image acquisition unit 12 to acquire video data. In response to this, the camera image acquisition unit 12 acquires video (video) data of the learner passing through the gate, which is captured by the camera 3 (S112).

[0055] Next, the posture estimation processing unit 13 acquires m sets of skeletal information from the video data of the learner acquired by the camera image acquisition unit 12 (S113). Specifically, m frame images are extracted from the video data acquired by the camera image acquisition unit 12, and for each extracted frame image, an existing human posture estimation method is used to estimate skeletal information including coordinate information of multiple parts of the learner's skeleton shown in the frame image. In this way, m sets of skeletal information are acquired for the learner specified by the user.

[0056] Next, the binary classification trained model generation unit 141 inputs the m sets of skeletal information acquired by the posture estimation processing unit 13 from the learner's video data into the n-value classification trained model 143 generated by the n-value classification trained model generation unit 140, and acquires m sets of output values (score values for each of the n labels) (S114).The binary classification trained model generation unit 141 then calculates m sets of variance information (score addition values and / or score subtraction values) 144 from the m sets of output values (score values for the n labels) (S115).Then, statistical processing is performed on the calculated m sets of variance information 144 to calculate the mean value and standard deviation, and these are used as statistical information 145 of granularity m for this learner (S116).The processes from S113 to S116 are repeated to generate a large number of statistical information 145 of granularity m. For example, if 800 pieces of image data are acquired in S112 and the granularity is set to 20, the process is repeated 40 times to generate 40 pieces of statistical information 145 with a granularity of 20.

[0057] Next, if the binary classification trained model generation unit 141 has received a learner designation operation from the user via the man-machine interface unit 11 for all learners whose skeletal information was used as input data for the n-value classification trained model 143 generated by the n-value classification trained model generation unit 140 (YES in S117), it terminates this flow.

[0058] On the other hand, if there is an undesignated learner for whom a learner designation operation has not been received (NO in S117), the binary classification trained model generation unit 141 waits for a learner designation operation involving the designation of a label associated with the undesignated learner from the user via the man-machine interface unit 11 while the undesignated learner is positioned in front of the gate where the camera 3 is installed (S118). Then, if a learner designation operation for the undesignated learner is received (YES in S118), the unit instructs the camera image acquisition unit 12 to acquire video data. In response to this, the process returns to S112, and the camera image acquisition unit 12 acquires video data of the learner captured by the camera 3. Thereafter, the process from S113 onwards is executed for this learner. On the other hand, if a learner designation operation has not been received (NO in S118) and an end operation has been received from the user (YES in S119), the flow ends.

[0059] FIG. 8 is a flow diagram for explaining the non-learner statistical information calculation process S120 shown in FIG.

[0060] When an unlearned person whose skeletal information has not been used as input data for the trained model 143 is standing by in front of the gate where the camera 3 is installed, the binary classification trained model generation unit 141 receives an operation to designate an unlearned person from the user via the man-machine interface unit 11 (S121), and instructs the camera image acquisition unit 12 to acquire video data. In response to this, the camera image acquisition unit 12 acquires video (video) data of the unlearned person passing through the gate, which is captured by the camera 3 (S122).

[0061] Next, posture estimation processing unit 13 acquires m sets of skeletal information from the video data of the unlearned person acquired by camera image acquisition unit 12 (S123). Specifically, m frame images are extracted from the video data acquired by camera image acquisition unit 12, and for each extracted frame image, an existing human posture estimation method is used to estimate skeletal information including coordinate information of multiple parts of the skeleton of the unlearned person shown in the frame image. In this way, m sets of skeletal information about the unlearned person are acquired.

[0062] Next, the binary classification trained model generation unit 141 inputs the m sets of skeletal information acquired by the posture estimation processing unit 13 from the video data of the unlearned person into the n-value classification trained model 143 generated by the n-value classification trained model generation unit 140, and acquires m sets of output values (score values for each of the n labels) (S124). Then, the binary classification trained model generation unit 141 calculates m sets of variance information (score addition values and / or score subtraction values) 147 from the m sets of output values (score values for the n labels) (S125). Then, the calculated m sets of variance information 147 are statistically processed to calculate the mean value and standard deviation, and these are used as statistical information 148 of granularity m of this unlearned person (S126). The processes from S123 to S126 are repeated to generate a large number of statistical information 148 of granularity m. For example, if 800 pieces of image data are acquired in S122 and the granularity is set to 20, the process is repeated 40 times to generate 40 pieces of statistical information 148 with a granularity of 20.

[0063] Next, when another unlearned person is positioned in front of the gate where camera 3 is installed, and the binary classification trained model generation unit 141 receives an unlearned person designation operation from the user via the man-machine interface unit 11 (YES in S127), it instructs the camera image acquisition unit 12 to acquire image data. In response to this, the process returns to S122, and the camera image acquisition unit 12 acquires image data of the unlearned person captured by camera 3. Thereafter, it executes the processes from S123 onwards for this unlearned person. On the other hand, if the unlearned person designation operation is not received from the user via the man-machine interface unit 11 (NO in S127) but an operation to end acquisition of statistical information on the unlearned person is received (YES in S128), this flow ends.

[0064] FIG. 9 is a flow diagram for explaining the machine learning process S130 shown in FIG.

[0065] The binary classification trained model generation unit 141 sets the correct label (teacher data [0]) for the learner (S131), and then sets the correct label (teacher data [1]) for the unlearned person (S132). Then, the binary classification trained model generation unit 141 uses the large amount of granularity m statistical information 145 for multiple learners calculated by the learner statistical information calculation processing unit S110 shown in FIG. 7 and the large amount of granularity m statistical information 148 for multiple unlearners calculated by the unlearner statistical information calculation processing unit S120 shown in FIG. 8 as input data for the learner and the unlearned person, respectively, and causes the AI to perform machine learning so that the score value of each correct label becomes high (S133). Thereafter, the binary classification trained model generation unit 141 ends the generation of the binary classification trained model 149 by AI machine learning (S134), and ends this flow.

[0066] In the flow shown in FIG. 9, the process for setting a correct label for a learner S131 and the process for setting a correct label for a non-learner S132 are performed in that order, but the process for setting a correct label for a non-learner S132 and the process for setting a correct label for a learner S131 may also be performed in that order.

[0067] Next, the person identification device 2 will be described in detail.

[0068] FIG. 10 is a diagram showing a schematic functional configuration of the person identification device 2. As shown in FIG.

[0069] As shown in the figure, the person identification device 2 has a network interface unit 20, a man-machine interface unit 21, a trained model acquisition unit 22, a camera image acquisition unit 23, a posture estimation processing unit 24, and a person identification unit 25.

[0070] The network interface unit 20 is an interface for connecting to the trained model generation device 1 and the camera 3 via the network 4.

[0071] The man-machine interface unit 21 is an interface for displaying information to the user and accepting various operations from the user, and includes a display device such as an LCD, and input devices such as a keyboard and a mouse.

[0072] The trained model acquisition unit 22 sends a trained model request to the trained model generation device 1 via the network interface unit 20 in accordance with a trained model acquisition operation received from a user via the man-machine interface unit 21, and acquires a trained model 143 for n-value classification and a trained model 149 for binary classification from the trained model generation device 1.

[0073] The camera image acquisition unit 23 monitors the image data (moving image) sent from the camera 3 via the network interface unit 20. If it detects that a person 5 is captured in this image data, it acquires this image data.

[0074] Pose estimation processing unit 24 extracts m frame images from the video data acquired by camera video acquisition unit 23 (for example, extracts all frame images that make up the video data), and for each extracted frame image, estimates skeletal information including coordinate information for multiple parts of the skeleton of person 5 shown in the frame image using an existing human posture estimation method. In this way, m sets of skeletal information corresponding to the number of frame images extracted from the video data are acquired.

[0075] The person identification unit 25 has a learner / non-learner identification unit 250 and a learner identification unit 251 .

[0076] The learner / unlearner identification unit 250 uses the n-value classification learned model 143 and the binary classification learned model 149 acquired by the learned model acquisition unit 22 from the learned model generation device 1 to identify whether the person 5 to be identified is a learner or an unlearner from the m sets of skeletal information acquired by the posture estimation processing unit 24.

[0077] 11, m sets of skeletal information 252 acquired by posture estimation processing unit 24 from video data of the person to be identified acquired by camera image acquisition unit 23 are input into trained model 143 for n-value classification, and m sets of score values 253 of n labels, which are its output values, are acquired. Next, m sets of variance information 254 indicating the degree of variance of score values 253 of n labels are calculated from the m sets of output values, and statistical information 255 of granularity m of the person to be identified is calculated from the m sets of variance information 254 by statistical processing. As described above, in this embodiment, score addition values and / or score subtraction values are used as the variance information, and the average value and standard deviation of the m sets of variance information are used as the statistical information.

[0078] Then, the statistical information 255 of the person to be identified is input into the binary classification trained model 149, and the output values, score values 256 for each of the two labels, are obtained. Then, based on the score values 256 for each label, learner / non-learner identification 257 is performed as follows: That is, if a learner is associated with the label having the largest score value 256, the person to be identified is identified as a learner, and if an non-learner is associated with the label, the person to be identified is identified as an non-learner.

[0079] The learner identifying unit 251 identifies which learner the person to be identified who has been identified as a learner by the learner / non-learner identifying unit 250 is.

[0080] 11, m sets of skeletal information acquired by posture estimation processing unit 24 from video data of the person to be identified acquired by camera image acquisition unit 23 are input into trained model 143 for n-value classification, and learner identification 258 is performed as follows based on score values 253 for each of n labels obtained for each piece of skeletal information: That is, the label (learner) associated with the largest score value among score values 253 for each label is identified as the correct answer.

[0081] The functional configuration of the person identification device 2 shown in Fig. 10 is realized in hardware using an integrated logic IC such as an ASIC or FPGA, or in software using a computer such as a DSP, similar to the trained model generation device 1 shown in Fig. 2. Alternatively, it is realized as a process in a general-purpose computer such as a PC equipped with a CPU / GPU, memory, an auxiliary storage device such as a hard disk or flash memory, and a communication interface such as a NIC, by the CPU / GPU loading a predetermined program from the auxiliary storage device into the memory and executing it.

[0082] FIG. 12 is a diagram for explaining the person identification process performed by the person identification device 2. As shown in FIG.

[0083] This flow starts when the camera image acquisition unit 23 detects the person 5 who is the identification target from the image data in front of the gate sent from the camera 3 via the network interface unit 20.

[0084] First, the camera image acquisition unit 23 acquires image (video) data of a person to be identified passing through the gate, captured by the camera 3 (S200).

[0085] Next, posture estimation processing unit 24 acquires m sets of skeletal information from the video data of the person to be identified acquired by camera video acquisition unit 23 (S201). Specifically, m frame images are extracted from the video data acquired by camera video acquisition unit 23, and for each extracted frame image, an existing human posture estimation method is used to estimate skeletal information including coordinate information of multiple parts of the skeleton of the person to be identified shown in the frame image. In this way, m sets of skeletal information 252 about the person to be identified are acquired.

[0086] Next, the learner / unlearned individual identification unit 250 inputs m sets of skeletal information 252 acquired by the posture estimation processing unit 24 from the video data of the person to be identified into the trained model 143 of n-ary classification acquired from the trained model generation device 1 by the trained model acquisition unit 22, and acquires score values 253 for each of the n labels for each piece of skeletal information of the person to be identified (S202).The learner / unlearned individual identification unit 250 then calculates variance information (score addition values and / or score subtraction values) 254 of the score values 253 for the n labels for each piece of skeletal information of the person to be identified, and acquires m sets of variance information (S203).Then, statistical processing is performed on the m sets of variance information 254 to calculate the average value and standard deviation as statistical information 255 of the granularity m for this person to be identified (S204).

[0087] Next, the learner / unlearner identification unit 250 inputs the statistical information 255 of granularity m of the person to be identified calculated as described above into the trained model 149 for binary classification acquired from the trained model generation device 1 by the trained model acquisition unit 22, and acquires the score values 256 of each of the two labels, which are the output values (S205).

[0088] Then, the learner / unlearner identification unit 250 identifies whether the person to be identified is a learner or an unlearner based on the score values 256 of each of the two labels output from the binary classification trained model 149 (S206): That is, if a learner is associated with the label having the largest score value 256, the person to be identified is identified as a learner, and if an unlearner is associated with the label, the person to be identified is identified as an unlearner (learner / unlearner identification 257).

[0089] If the learner / non-learner identification unit 250 identifies the person to be identified as a non-learner (YES in S207), it displays this identification result on the man-machine interface unit 21 (S209). On the other hand, if the person to be identified is identified as a learner (NO in S207), it outputs the score values 253 of each of the n labels output for each skeleton information from the n-value classification trained model 143 in S202 to the learner identification unit 251. In response to this, the learner identification unit 251 identifies which learner the person to be identified is as follows (S208). That is, it identifies the label (learner) associated with the largest score value 253 as the correct answer (learner identification 258). Then, it displays this identification result on the man-machine interface unit 21 (S209).

[0090] The first embodiment of the present invention has been described above.

[0091] The inventors have found that there is a significant difference between the degree of dispersion of the score values of each of the n labels obtained by inputting the skeleton information of a learner into a trained model 143 for n-ary classification that identifies each of a total of n learners and the degree of dispersion of the score values of each of the n labels obtained by inputting the skeleton information of unlearners other than the learner. Therefore, in this embodiment, a large amount of statistical information 145 with a granularity of m calculated from a large amount of variance information 144 of the learner and a large amount of statistical information with a granularity of m calculated from a large amount of variance information 147 of the unlearner are used as input data to generate a trained model 149 for binary classification that distinguishes between learners and unlearners. Then, m sets of variance information 254 calculated from m sets of output values (score values of n labels) obtained by inputting m sets of skeleton information 252 of the person to be identified into the trained model 143 for n-ary classification is statistically processed to calculate m-granularity statistical information 255, which is then input into the trained model 149 for binary classification to identify whether the person to be identified is a learner or an unlearner. Then, when the person to be identified is identified as a learner, the learner to which the person to be identified belongs is identified based on the m sets of output values (score values 253 of n labels) obtained by inputting the m sets of skeletal information 252 of the person to be identified into the trained model 143 for n-ary classification. This makes it possible to identify whether the person to be identified is a learner or a non-learner even when the people to be identified include a mixture of learners and non-learners, and if the person to be identified is a learner, it is possible to identify which learner this person belongs to.

[0092] Furthermore, the present inventors have found that learners and unlearners can be distinguished by using score addition values and / or score subtraction values as the shared information 144, 147 of the learners and unlearners. These score addition values and score subtraction values can be handled independently. Therefore, in this embodiment, the shared information 144, 147 of the learners and unlearners uses the score addition value, which is the sum of the maximum and minimum values of the score values of n labels, which are output values of the trained model 143 for n-ary classification, and the score subtraction value, which is the difference between them. This can improve the prediction accuracy of the trained model 149 for binary classification.

[0093] Furthermore, in this embodiment, m sets of skeletal information 142 of the learner are input into a trained model 143 for n-ary classification to calculate m sets of variance information 144, and a large number of pieces of statistical information 145 of granularity m are calculated from the large number of m sets of variance information 144. Furthermore, m sets of skeletal information 146 of an unlearned person are input into the trained model 143 for n-ary classification to calculate m sets of variance information 147, and a large number of pieces of statistical information 148 of granularity m are calculated from the large number of m sets of variance information 147. The statistical information 145, 148 are used as input data, and machine learning is performed by an AI to increase the score value of the label associated therewith, thereby generating a trained model 149 for binary classification. When identifying a person to be identified, m sets of skeletal information 252 of the person to be identified are input into the trained model 143 for n-ary classification to calculate m sets of variance information 254, and one piece of statistical information 255 of granularity m is calculated from the m sets of variance information 254. Then, statistical information 255 of the person to be identified is input into a trained model 149 for binary classification to obtain a score value 256 for each of the two labels associated with the learner and the unlearner. If the learner is associated with the label with the largest score value, the person to be identified is identified as a learner, and if an unlearner is associated with the label, the person to be identified is identified as an unlearner. If the person to be identified is identified as a learner, m sets of skeleton information 252 of the person to be identified are input into a trained model 143 for n-value classification, and the label (learner) associated with the largest score value among the score values 253 of the n labels obtained for each skeleton information is identified as the correct answer. In this way, by generating a trained model 149 for binary classification using statistical information 145, 148 of the learner and unlearner as input data and inputting statistical information 255 of the person to be identified into this trained model 149 for binary classification, it is possible to more accurately identify whether the person to be identified is a learner or an unlearner.

[0094] The inventors have also found that learners and non-learners can be distinguished by using the mean value and standard deviation obtained from the variance information 144, 147 calculated for each skeleton information as the statistical information 145, 148 of learners and non-learners, respectively. Therefore, in this embodiment, the mean value and standard deviation obtained from m sets of variance information 144, 147 are used as the statistical information 145, 148 of granularity m of learners and non-learners, respectively.

[0095] [Second embodiment] In this embodiment, the n-ary classification and binary classification shown in the first embodiment are applied in multiple stages, and a person to be identified who has been identified as an unlearned person can be further identified as a learner or an unlearned person using n-ary classification and binary classification. The person identification system according to this embodiment differs from the person identification system according to the first embodiment shown in FIG. 1 in that a trained model generation device 1a is used instead of the trained model generation device 1, and a person identification device 2a is used instead of the person identification device 2. The other configurations are the same as those of the person identification system according to the first embodiment shown in FIG. 1.

[0096] The trained model generation device 1a generates, for each specific learner group, a trained model of n-ary classification that identifies each learner belonging to this group, and also generates, for each specific learner group, a trained model of binary classification that identifies learners and non-learners belonging to this group.

[0097] Like the trained model generation device 1 according to the first embodiment, the trained model generation device 1a is typically used when introducing a person identification system, and is isolated from the person identification system after the trained models for n-value classification and binary classification have been generated.

[0098] The person identification device 2a identifies a person 5 (person to be identified) from video data of the person to be identified captured by a camera 3, using a trained model for n-value classification and a trained model for binary classification generated for each specific learner group by the trained model generation device 1a.

[0099] First, the person identification device 2a acquires m sets of skeletal information of the person to be identified from video data captured by the camera 3 using an existing human posture estimation method, inputs these m sets of skeletal information into one trained model for n-ary classification selected from trained models for n-ary classification generated for specific learner groups (hereinafter referred to as the target n-ary classification model), and calculates variance information indicating the degree of variance of the score values of each of the n labels, which are the output values, for each piece of skeletal information. Next, it performs statistical processing on the m sets of variance information of the person to be identified calculated as above to calculate statistical information of granularity m, and inputs this into a trained model for binary classification (hereinafter referred to as the target binary classification model) that distinguishes between learners and non-learners belonging to the learner group corresponding to the target n-ary classification model, obtains score values for each of the two labels associated with the learner and non-learner, and identifies the person to be identified as a learner if the learner is associated with the label with the largest score value, or as a non-learner if the non-learner is associated with the label. If the target person is identified as a learner, the target person is identified as the learner associated with the label with the largest score among the scores for each of the n labels obtained by inputting the skeletal information of the target person into the target n-ary classification model. On the other hand, if the target person is identified as a non-learner, the target person does not belong to a specific learner group. Therefore, an unselected trained model for n-ary classification is selected as a new target n-ary classification model from the trained models for n-ary classification generated for each specific learner group, and statistical information of the target person with a granularity of m is input into this new target n-ary classification model to perform the above-mentioned process. This process is then repeated until the target person is identified as a learner in a specific group or until all trained models for n-ary classification generated for each learner group are selected as target n-ary classification models.

[0100] Next, the trained model generation device 1a and the person identification device 2a that constitute the person identification system according to this embodiment will be described.

[0101] First, the trained model generation device 1a will be described in detail.

[0102] The trained model generation device 1a according to this embodiment differs from the trained model generation device 1 according to the first embodiment shown in Fig. 2 in that a machine learning unit 14a is provided instead of the machine learning unit 14. The other configurations are the same as those of the trained model generation device 1 according to the first embodiment shown in Fig. 2. Furthermore, the machine learning unit 14a differs from the machine learning unit 14 of the trained model generation device 1 according to the first embodiment shown in Fig. 2 in that an n-value classification trained model generation unit 140a and a binary classification trained model generation unit 141a are provided instead of the n-value classification trained model generation unit 140 and the binary classification trained model generation unit 141.

[0103] For each specific group of n learners (a group with a specific number of learners, n), the n-ary classification trained model generation unit 140a uses m sets of integer multiples of skeletal information 142 acquired by the posture estimation processing unit 13 from the video data of the learner acquired by the camera video acquisition unit 12 as input data for each of the 1st to nth learners, as shown in Figure 3, and causes the AI to perform machine learning so as to increase the score of the label associated with this learner. In this way, for each learner group, it generates a trained model 143 of n-ary classification that identifies the 1st to nth learners belonging to the learner group.

[0104] The binary classification trained model generation unit 141a performs the following processing for each specific group of n learners to generate a binary classification trained model that distinguishes between learners and non-learners belonging to this learner group.

[0105] First, as shown in Figure 4, m sets of skeleton information 142 for each of the 1st to nth learners (the 1st to nth learners used as input data for constructing this trained model 143) belonging to a specific n-number learning group are input into a trained model 143 for n-value classification, and m sets of variance information 144 indicating the degree of variance of the score values of the n labels, which are the output values, are calculated for each learner.Furthermore, statistical processing is performed on the m sets of variance information 144 for each learner to calculate statistical information 145 with granularity m.This process is repeated to obtain a large number of statistical information 145 with granularity m. Furthermore, for each of the 1st to kth unlearned individuals (persons 5 other than learners belonging to the learning group), m sets of skeletal information 146 estimated by the posture estimation processing unit 13 from the video data of the unlearned individual acquired by the camera image acquisition unit 12 are input into the n-value classification trained model 143 generated by the n-value classification trained model generation unit 140a, and m sets of variance information 147 indicating the degree of variance of the score values of the n labels, which are the output values, are calculated. Furthermore, for each unlearned individual, statistical processing is performed on the m sets of variance information 147 to calculate statistical information 148 of granularity m. This process is repeated to obtain a large number of statistical information 148 of granularity m. Then, using the large number of statistical information 145 of granularity m for each learner and the large number of statistical information 148 of granularity m for each unlearner as input data, AI is trained to perform machine learning so that the label with the highest score value matches the correct label using the correct label (teacher data [0, 1]) for the learner and unlearner. This generates a binary classification trained model 149 that distinguishes between learners and non-learners who belong to the specific group of n learners for each specific group of n learners. Note that the same skeleton information for the m sets of non-learners' groups may be reused for calculations for different specific groups.

[0106] The functional configuration of the trained model generation device 1a according to this embodiment is also realized in hardware using an integrated logic IC such as an ASIC or FPGA, or in software using a computer such as a DSP, similar to the trained model generation device 1 according to the first embodiment shown in Fig. 2. Alternatively, in a general-purpose computer such as a PC equipped with a CPU / GPU, memory, an auxiliary storage device such as a hard disk or flash memory, and a communication interface such as a NIC, the CPU / GPU loads a predetermined program from the auxiliary storage device into the memory and executes it, thereby realizing the configuration as a process.

[0107] FIG. 13 is a flow diagram illustrating the process of generating a trained model for n-value classification by the trained model generation device 1a.

[0108] This flow starts when the machine learning unit 14a receives an operation to generate a trained model for n-ary classification from the user via the man-machine interface unit 11.

[0109] In the machine learning unit 14a, when the n-value classification learned model generation unit 140a receives a learner group designation operation from the user via the man-machine interface unit 11, which involves designating a learner group for which an n-value classification learned model is to be generated (S300), it executes S100 to S108 shown in Figure 5 for the learners belonging to this learner group, and generates an n-value classification learned model 143 for individually identifying the learners belonging to this learner group (S301).

[0110] Next, when the n-value classification trained model generation unit 140a receives a learner group designation operation involving the designation of a new learner group from the user via the man-machine interface unit 11 (YES in S302), it returns to S301 and generates an n-value classification trained model 143 for individually identifying learners belonging to this new learner group.

[0111] On the other hand, if the n-value classification trained model generation unit 140a does not accept a learner group designation operation from the user via the man-machine interface unit 11 (NO in S302) but accepts an operation to end the generation of the n-value classification trained model (YES in S303), it terminates this flow.

[0112] FIG. 14 is a flow diagram illustrating the process of generating a trained model for binary classification by the trained model generation device 1a.

[0113] This flow starts when the machine learning unit 14a receives an operation to generate a trained model for binary classification from the user via the man-machine interface unit 11.

[0114] In the machine learning unit 14a, when the binary classification trained model generation unit 141a receives a learner group designation operation from the user via the man-machine interface unit 11, which designates a learner group for which a binary classification trained model is to be generated (S310), the binary classification trained model generation unit 141a executes S110 shown in FIG. 6 for the learners belonging to this learner group to calculate m-granularity statistical information 145 of the learners belonging to this learner group, and also executes S120 shown in FIG. 6 for non-learners other than the learners belonging to this learner group to calculate m-granularity statistical information 148 of the non-learners. Then, using the m-granularity statistical information 145 of the n specific learner groups calculated as above and the m-granularity statistical information 148 of the non-learner groups and the correct label (teaching data [0,1]), the AI is trained to perform machine learning so that the label with the highest score value matches the correct label (S130 in FIG. 6), and generates a binary classification trained model 149 for distinguishing between learners and non-learners belonging to this learner group (S311).

[0115] Next, if the binary classification trained model generation unit 141a has accepted learner group designation operations for all n specific learner groups for which n-value classification trained models 143 have been generated by the n-value classification trained model generation unit 140a (YES in S312), it terminates this flow.

[0116] On the other hand, if there are n learner groups for which a specific learner group designation operation has not been accepted (NO in S312), the binary classification trained model generation unit 141a waits for acceptance of a learner group designation operation involving the designation of an undesignated learner group from the user via the man-machine interface unit 11 (S313). Then, if a learner group designation operation involving the designation of an undesignated learner group is accepted (YES in S313), the process returns to S311 and the binary classification trained model 149 is regenerated to distinguish between learners belonging to this new learner group and non-learners.

[0117] Next, the person identification device 2a will be described in detail.

[0118] The person identification device 2a according to this embodiment differs from the person identification device 2 according to the first embodiment shown in Fig. 10 in that a person identification unit 25a is provided instead of the person identification unit 25. The other configurations are the same as those of the person identification device 2 according to the first embodiment shown in Fig. 10. Also, the person identification unit 25a differs from the person identification unit 25 of the person identification device 2 according to the first embodiment shown in Fig. 10 in that a learner / unlearner identification unit 250a and a learner identification unit 251a are provided instead of the learner / unlearner identification unit 250 and the learner identification unit 251.

[0119] The learner / non-learner identification unit 250a uses, for each specific group of n learners for which the trained model 143 for n-value classification and the trained model 149 for binary classification have been generated by the trained model generation device 1a, the trained model acquisition unit 22 acquires from the trained model generation device 1a the trained model 143 for n-value classification and the trained model 149 for binary classification of this learner group, to identify, from the m sets of skeletal information acquired by the posture estimation processing unit 24, whether the person to be identified is a learner belonging to a specific learner group or a non-learner who does not belong to a specific learner group.

[0120] Specifically, an unselected learner group is selected from a plurality of learner groups for which n specific n-ary classification trained models 143 and binary classification trained models 149 have been generated by the trained model generation device 1, and the n-ary classification trained model 143 and binary classification trained model 149 of this learner group are determined as the target n-ary classification model and the target binary classification model, respectively. Then, as shown in FIG. 11 , m sets of skeletal information 252 acquired by the posture estimation processing unit 24 from video data of the person to be identified acquired by the camera image acquisition unit 23 are input into the target n-ary classification model, m sets of variance information 254 indicating the degree of variance of the score values 253 of each of the n labels, which are the output values, are calculated, and further, m sets of variance information 254 are used to calculate m-granularity statistical information 255 of the person to be identified. Then, the m-granularity statistical information 255 of the person to be identified is input into the target binary classification model, and score values 256 of each of the two labels, which are the output values, are obtained. Then, based on the score value 256 of each label, learner / non-learner identification 257 is performed as follows. That is, if a learner is associated with the label having the largest score value 256, the person to be identified is identified as a learner belonging to the selected learner group, and if a non-learner is associated, the person to be identified is identified as a non-learner who does not belong to a learner group. This process is repeated by the learner identification unit 251a, described below, until the person to be identified is identified as one of the learners in the selected learner group, or until all learner groups have been selected.

[0121] When the learner / non-learner identification unit 250a recognizes that the person to be identified is a learner belonging to a selected learner group, the learner identification unit 251a identifies which learner in this learner group the person to be identified belongs to.

[0122] Specifically, as shown in FIG. 11, m sets of skeletal information acquired by the posture estimation processing unit 24 from the video data of the person to be identified acquired by the camera image acquisition unit 23 are input into the n-value classification model to be implemented, thereby obtaining score values 253 for each of n labels for each skeletal information 252, and identifying the label (learner) associated with the largest score value 253 as the correct answer (learner identification 258).

[0123] 10, the functional configuration of the person identification device 2a according to this embodiment is realized in hardware using an integrated logic IC such as an ASIC or FPGA, or in software using a computer such as a DSP. Alternatively, in a general-purpose computer such as a PC equipped with a CPU / GPU, memory, an auxiliary storage device such as a hard disk or flash memory, and a communication interface such as a NIC, the CPU / GPU loads a predetermined program from the auxiliary storage device onto the memory and executes it, thereby realizing the configuration as a process.

[0124] FIG. 15 is a diagram for explaining the person identification operation by the person identification device 2a.

[0125] This flow starts when the camera image acquisition unit 23 detects the person 5 who is the identification target from the image data in front of the gate sent from the camera 3 via the network interface unit 20.

[0126] First, the camera image acquisition unit 23 acquires image data of the person to be identified passing through the gate, captured by the camera 3 (S400).

[0127] Next, posture estimation processing unit 24 acquires m sets of skeletal information from the video data of the person to be identified acquired by camera image acquisition unit 23 (S401). Specifically, m frame images are extracted from the video data acquired by camera image acquisition unit 23, and for each extracted frame image, an existing human posture estimation method is used to estimate skeletal information including coordinate information of multiple parts of the skeleton of the person to be identified shown in the frame image. In this way, m sets of skeletal information 252 are acquired for the person to be identified.

[0128] Next, the learner / non-learner identification unit 250a selects an unselected learner group from among the multiple learner groups for which the trained model 143 for n-ary classification and the trained model 149 for binary classification have been generated by the trained model generation device 1a (S402). Hereinafter, the selected learner group will be referred to as the selected group. Then, the trained model acquisition unit 22 determines the trained model 143 for n-ary classification and the trained model 149 for binary classification of the selected group, acquired from the trained model generation device 1a, as the n-ary classification model and the binary classification model to be performed, and executes S202 to S206 shown in FIG. 12 using the n-ary classification model and the binary classification model to be performed to identify whether the person to be identified is a learner belonging to the selected group or an unlearner other than a learner belonging to the selected group (S403).

[0129] If the learner / non-learner identification unit 250a identifies the person to be identified as a learner (YES in S404), it outputs the score values 253 of the n labels output from the n-value classification model to be implemented for each skeleton information 252 in S403 to the learner identification unit 251a. In response to this, the learner identification unit 251a identifies the label (learner) associated with the largest score value 253 as the correct answer (S405). Then, the identification result is displayed on the man-machine interface unit 21 (S407).

[0130] On the other hand, if the learner / unlearned person identification unit 250a identifies the person to be identified as an unlearned person (NO in S404), it determines whether or not there are any unselected learner groups among the multiple learner groups for which the trained model 143 for n-ary classification and the trained model 149 for binary classification have been generated by the trained model generation device 1a (S406). If there are any unselected learner groups (YES in S406), the process returns to S402. On the other hand, if there are no unselected learner groups (NO in S406), that is, if all of the multiple learner groups for which the trained model 143 for n-ary classification and the trained model 149 for binary classification have been generated by the trained model generation device 1a have been selected, the identification result indicating that the person to be identified is an unlearned person is displayed on the man-machine interface unit 21 (S407).

[0131] The second embodiment of the present invention has been described above.

[0132] In this embodiment, similar to the first embodiment, the person to be identified is identified using skeletal information of the person. Here, the skeletal information is acquired from video data of the person to be identified (moving) while passing through a gate, captured by a camera 3, using an existing human posture estimation method, so there is less variation between individuals compared to other biometric information such as fingerprints, veins, and irises. For this reason, as the number of learners to be classified increases and the number of classifications "n" in the trained model for n-ary classification becomes larger, the accuracy of machine learning by AI decreases, and high identification accuracy cannot be maintained.

[0133] Therefore, in this embodiment, for each different learner group, a trained model for n-ary classification that identifies individual learners belonging to this group and a trained model for binary classification that identifies learners and non-learners belonging to this group are generated. Then, using the trained model for n-ary classification and the trained model for binary classification for a selected group selected from the multiple learner groups for which the trained models for n-ary classification and the trained model for binary classification have been generated, first, it is determined whether the person to be identified is a learner belonging to the selected group or a non-learner other than a learner based on the skeletal information of the person to be identified. If the person to be identified is a learner, the score value of the trained model for n-ary classification is used to identify which learner belonging to the selected group the person to be identified is. On the other hand, if the person to be identified is a non-learner, this means that the person does not belong to the specific learning group, so a new selected group is selected and the above process is repeated until there are no unselected learner groups. In this way, even if the number of learners to be classified increases, this can be handled by increasing the number of groups for specific learner groups and reducing the number of classifications "n" in each trained model for n-ary classification. As a result, high classification accuracy can be maintained even for a large number of learners.

[0134] Other effects are the same as those of the first embodiment.

[0135] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the present invention.

[0136] For example, in each of the above embodiments, the trained model generation devices 1, 1a and the person identification devices 2, 2a are prepared separately, and the trained model generation devices 1, 1a are typically used when introducing a person identification system, and may be isolated from the person identification system after generating a trained model for n-ary classification and a trained model for binary classification. However, the present invention is not limited to this. The trained model generation devices 1, 1a and the person identification devices 2, 2a may be integrated by, for example, building them on the same computer system.

[0137] In addition, in each of the above embodiments, the skeletal information of the person to be identified is used as feature information of the person to be identified, but the present invention is not limited to this. The present invention can be applied to any problem in which the object to be identified is classified using feature information unique to the object to be identified, such as an image, text, video, or metadata.

[0138] Furthermore, in each of the above embodiments, the person 5 who is the identification target is identified, but the present invention can also be used to identify targets other than the person 5. [Explanation of symbols]

[0139] 1, 1a: Trained model generation device 2, 2a: Person identification device 3: Camera 4: Network 5: People 10, 20: Network interface section 11, 21: Man-machine interface section 12, 23: Camera image acquisition unit 13, 24: Attitude estimation processing unit 14, 14a: Machine learning unit 15: Trained model transmission unit 22: Learned model acquisition unit 25: Person identification unit 140, 140a: n-value classification trained model generation unit 141, 141a: binary classification trained model generation unit 250, 250a: Learner / non-learner identification unit 251, 251a: Learner identification unit

Claims

1. A method for identifying an object to be identified from feature information of the object to be identified using AI (Artificial Intelligence), comprising: a variance information calculation step of inputting the feature information of the classification target into a trained model of n-valued classification that identifies each of a total of n learning targets, and calculating variance information indicating the degree of variance of the score values of n labels associated with the different learning targets; a statistical information calculation step of calculating statistical information of granularity m from the m sets of variance information calculated for each piece of feature information of the classification target in the variance information calculation step; a first classification step of inputting the statistical information of granularity m calculated in the statistical information calculation step as input data into a trained model of binary classification that distinguishes between the training object and an untrained object other than the training object, obtaining score values for two labels associated with the training object and the untrained object, and identifying the classification object as the training object if the training object is associated with the label having the largest score value of the two labels, and identifying the classification object as the untrained object if the untrained object is associated with the label; a second classification step of, when the object to be classified is classified as the learning object by the first classification step, classifying the object to be classified as the learning object associated with the label having the largest score value among the score values of the n labels obtained by inputting feature information of the object to be classified into the trained model for n-ary classification, The trained model for n-ary classification is For each of the learning objects, a total of n, feature information of the learning object is used as input data, and the AI is trained to perform machine learning so that the score value of the label associated with the learning object becomes high, The trained model for binary classification is The feature information of the learning object used as the input data is input into the trained model of n-value classification to calculate variance information indicating the degree of variance of the score values of n labels, which are the output values, and the feature information of the unlearned object is input into the trained model of n-value classification to calculate variance information indicating the degree of variance of the score values of n labels, which are the output values, and statistical information of granularity m calculated from the variance information of m sets calculated for each of the learning object and the unlearned object is used as input data, and the AI is made to perform machine learning so that the score values of the labels associated with each of them become higher.

10. A method for identifying a target object.

2. 2. The identification method according to claim 1, The shared information is The score addition value is the sum of the maximum and minimum score values of n labels obtained by inputting the feature information into the trained model of n-value classification, and the score subtraction value is the difference between them.

10. A method for identifying a target object.

3. 2. The identification method according to claim 1, The statistical information is The mean value and standard deviation of m sets of variance information obtained by inputting m sets of feature information into the trained model for n-value classification are 10. A method for identifying a target object.

4. 2. The identification method according to claim 1, the object to be identified is a person, The characteristic information is Skeleton information obtained from the video data of the object to be identified using a human posture estimation method.

10. A method for identifying a target object.

5. 5. The method of claim 1, further comprising: The dispersion information calculation step includes: When the object to be identified is identified as the unlearned object by the first identification step, inputting feature information of the object to be identified into another trained model that performs n-value classification to identify each of a total of n learning objects, which is different from the trained model of n-value classification that has been used to calculate the variance information of the object to be identified, and recalculating the variance information of the object to be identified; The statistical information calculation step includes: recalculating statistical information of granularity m from the m sets of variance information recalculated in the variance information calculation step; The first identification step includes: When m sets of variance information are recalculated by the variance information calculation step, inputting statistical information of granularity m recalculated by the statistical information calculation step based on the m sets of variance information into a trained model of binary classification that is different from the trained model of binary classification that has been used to previously identify the identification target, and that distinguishes between the training target used as input data for the trained model of n-value classification used to recalculate the m sets of variance information and untrained targets other than the training target, and identifying the identification target as the training target or the untrained target; The second identification step includes: When the object to be identified is identified as the learning object in the first identification step, the object to be identified is identified as the learning object associated with the label having the largest score value among the score values of the n labels obtained by inputting feature information of the object to the trained model of n-ary classification used to calculate the variance information in the variance information calculation step performed immediately before.

10. A method for identifying a target object.

6. A trained model generation method for generating a trained model for identifying an object to be identified from feature information of the object to be identified using AI (Artificial Intelligence), a first trained model generation step of generating a trained model of n-value classification that identifies each of the learning objects by using feature information of the learning object as input data and having the AI train the learning object so that the score value of the label associated with the learning object becomes high; and a second trained model generation step of inputting feature information of the training object used as the input data into the trained model of n-value classification generated by the first trained model generation step to calculate variance information indicating the degree of variance of score values of n labels that are output values thereof, inputting feature information of untrained objects other than the training object into the trained model of n-value classification to calculate variance information indicating the degree of variance of score values of n labels that are output values thereof, and using statistical information of granularity m calculated from m sets of variance information calculated for each of the training object and the untrained object as input data, and having the AI perform machine learning so that the score values of the labels associated with each of the training object and the untrained object become high, thereby generating a trained model of binary classification that can distinguish between the training object and the untrained object. A trained model generation method characterized by:

7. An identification device that identifies an identification target from feature information of the identification target using AI (Artificial Intelligence), a variance information calculation means for inputting the feature information of the classification target into a trained model of n-valued classification for identifying each of a total of n learning targets, and calculating variance information indicating the degree of variance of score values of n labels associated with the different learning targets; a statistical information calculation means for calculating statistical information of granularity m from the m sets of variance information calculated for each piece of feature information of the classification target by the variance information calculation means; a first classification means for inputting the statistical information of granularity m calculated by the statistical information calculation means into a trained model of binary classification that distinguishes between the training object and an untrained object other than the training object, obtaining score values for two labels associated with the training object and the untrained object, and identifying the classification object as the training object if the training object is associated with the label having the largest score value of the two labels, and identifying the classification object as the untrained object if the untrained object is associated with the label; and a second classification means for, when the object to be classified is classified as the learning object by the first classification means, classifying the object to be classified as the learning object associated with the label having the largest score value among the score values of the n labels obtained by inputting feature information of the object to be classified into the trained model of n-valued classification, The trained model for n-ary classification is For each of the learning objects, a total of n, feature information of the learning object is used as input data, and the AI is trained to perform machine learning so that the score value of the label associated with the learning object becomes high, The trained model for binary classification is The feature information of the learning object used as the input data is input into the trained model of n-value classification, and a plurality of pieces of variance information indicating the degree of variance of the score values of each of the n labels that are the output values is calculated. The feature information of the unlearned object is input into the trained model of n-value classification, and a plurality of pieces of variance information indicating the degree of variance of the score values of the n labels that are the output values is calculated. Statistical information of granularity m calculated from m sets of variance information calculated for each of the learning object and the unlearned object is used as input data, and the AI is trained to increase the score values of the labels associated with each of them. An identification device characterized by:

8. A trained model generation device that generates a trained model for identifying an object to be identified from feature information of the object to be identified using AI (Artificial Intelligence), a first trained model generation means for generating a trained model of n-value classification that identifies each of the learning objects by using feature information of the learning object as input data and having the AI perform machine learning so that the score value of the label associated with the learning object becomes high; and and second trained model generation means for inputting feature information of the training object used as the input data into the trained model of n-value classification generated by the first trained model generation means, calculating variance information indicating the degree of variance of score values of n labels that are output values thereof, inputting feature information of untrained objects other than the training object into the trained model of n-value classification, calculating variance information indicating the degree of variance of score values of n labels that are output values thereof, and using statistical information of granularity m calculated from m sets of variance information calculated for each of the training object and the untrained object as input data, thereby causing the AI to perform machine learning so as to increase the score values of the labels associated with each of the training object and the untrained object. A trained model generation device characterized by:

9. A program that causes a computer to function as an identification device that identifies an object to be identified from feature information of the object to be identified using AI (Artificial Intelligence), a variance information calculation means for inputting the feature information of the classification target into a trained model of n-value classification that identifies each of a total of n learning targets, and calculating variance information of the classification target that indicates the degree of variance of score values of n labels associated with the different learning targets; a statistical information calculation means for calculating statistical information of granularity m from the m sets of variance information calculated for each piece of feature information of the classification target by the variance information calculation means; a first classification means for inputting the statistical information of granularity m calculated by the statistical information calculation means into a trained model of binary classification that distinguishes between the training object and an untrained object other than the training object, obtaining score values for two labels associated with the training object and the untrained object, and identifying the classification object as the training object if the training object is associated with the label having the largest score value of the two labels, and identifying the classification object as the untrained object if the untrained object is associated with the label; and when the object to be identified is identified as the learning object by the first identification means, causing the computer to function as second identification means that identifies the object to be identified as the learning object associated with the label having the largest score value among the score values of the n labels obtained by inputting feature information of the object to be identified into the trained model of n-valued classification, The trained model for n-ary classification is For each of the learning objects, a total of n, feature information of the learning object is used as input data, and the AI is trained to perform machine learning so that the score value of the label associated with the learning object becomes high, The trained model for binary classification is The feature information of the learning object used as the input data is input into the trained model of n-value classification, and variance information indicating the degree of variance of the score values of n labels, which are the output values, is calculated. The feature information of the unlearned object is input into the trained model of n-value classification, and variance information indicating the degree of variance of the score values of n labels, which are the output values, is calculated. Statistical information of granularity m calculated from m sets of variance information calculated for each of the learning object and the unlearned object is used as input data, and the AI is trained to increase the score values of the labels associated with each of them. A program characterized by:

10. A program that causes a computer to function as a trained model generation device that generates a trained model for identifying an object to be identified from feature information of the object to be identified using AI (Artificial Intelligence), a first trained model generation means for generating a trained model of n-valued classification for identifying each of the learning objects by using feature information of the learning object as input data and having the AI perform machine learning so as to increase the score value of the label associated with the learning object; and The computer is made to function as second trained model generation means that generates a trained model for binary classification that distinguishes between the training object and the untrained object by inputting feature information of the training object used as the input data into the trained model for n-value classification generated by the first trained model generation means, calculating variance information indicating the degree of variance of the score values of n labels that are output values, and inputting feature information of untrained objects other than the training object into the trained model for n-value classification, calculating variance information indicating the degree of variance of the score values of n labels that are output values, and using statistical information with a granularity of m calculated from m sets of variance information calculated for each of the training object and the untrained object as input data, and having the AI perform machine learning so that the score values of the labels associated with each are increased. A program characterized by:

Images