Learning program, information processing device, and learning method

Abstract

This invention provides a learning program, an information processing device, and a learning method that enable improved inference accuracy regarding the types of gestures performed by a person. [Solution] For each of the multiple joints in a person's hand shown in the image data, multiple first reference points corresponding to the foot of the perpendicular when a perpendicular line is drawn from the position of each joint to the first axis are identified. For each of the multiple first reference points and each of the multiple joints, multiple first distances indicating the distance between each first reference point and the position of each joint are identified. For each of the multiple first reference points and each of the multiple combinations including two of the multiple joints, multiple first angles indicating the angle between the line passing through each first reference point and the position of one of the joints included in each combination, and the line passing through each first reference point and the position of the other joint included in each combination are identified. Training data including the multiple first distances and multiple first angles is generated, and a learning model is generated by training the training data.

Claims

1. For each of the multiple joints in the hand of a person shown in the image data, multiple first reference points corresponding to the foot of the perpendicular when a perpendicular line is drawn from the position of each joint to the first axis are identified. For each of the plurality of first reference points and each of the plurality of joints, a plurality of first distances are identified that indicate the distance between each first reference point and the position of each joint. For each of the plurality of first reference points and each of the plurality of combinations that include two of the plurality of joints, a plurality of first angles are identified that represent the angle between a line passing through each first reference point and the position of one of the joints included in each combination, and a line passing through each first reference point and the position of the other joint included in each combination. Training data is generated that includes the plurality of first distances and the plurality of first angles, A learning model is generated by training it with the aforementioned training data. A learning program characterized by having a computer perform the processing.

2. Furthermore, for each of the aforementioned multiple joints, a plurality of second reference points corresponding to the foot of the perpendicular when a perpendicular line is drawn from the position of each joint to the second axis are identified. For each of the plurality of second reference points and each of the plurality of joints, a plurality of second distances are identified that indicate the distance between each second reference point and the position of each joint. For each of the plurality of second reference points and each of the plurality of combinations, a plurality of second angles are identified that represent the angle between a straight line passing through each second reference point and the position of one joint included in each combination, and a straight line passing through each second reference point and the position of the other joint included in each combination. Let the computer perform the process, In the process of generating the training data, the training data is generated which includes the plurality of first distances, the plurality of first angles, the plurality of second distances, and the plurality of second angles, respectively. The learning program according to feature 1.

3. The first axis is a straight line perpendicular to the second axis. The learning program according to feature 2.

4. In the process of obtaining the first distance, for each of the multiple image data that make up the video data, the multiple first distances corresponding to each image data are identified. In the process of obtaining the first angle, for each of the multiple image data, the multiple first angles corresponding to each image data are identified. In the process of generating the training data, the training data is generated which includes the plurality of first distances and the plurality of first angles for each of the plurality of images. The learning program according to feature 1.

5. Furthermore, the system accepts input of first information indicating the state of the person's hands, In the process of generating the training data, the training data is generated which includes the plurality of first distances, the plurality of first angles, and the first information, respectively. The learning program according to feature 1.

6. Furthermore, for each of the multiple joints in the hand of the person shown in other image data, multiple new reference points corresponding to the foot of the perpendicular when a perpendicular line is drawn from the position of each joint to the first axis are identified. For each of the aforementioned multiple new reference points and for each of the aforementioned multiple joints, a plurality of new distances are identified that indicate the distance between each new reference point and the position of each joint. For each of the aforementioned multiple new reference points and each of the aforementioned multiple combinations, a plurality of new angles are identified that represent the angle between a straight line passing through each new reference point and the position of one joint included in each combination, and a straight line passing through each new reference point and the position of the other joint included in each combination. The system outputs second information indicating the state of the person's hand corresponding to the output value obtained by inputting the aforementioned multiple new distances and the aforementioned multiple new angles into the training data. The learning program according to claim 5, characterized in that it causes a computer to perform the processing.

7. A data generation unit that generates training data including the multiple first distances and the multiple first angles, for each of the multiple joints in a person's hand shown in the image data, identifies multiple first reference points corresponding to the foot of the perpendicular when a perpendicular line is drawn from the position of each joint to a first axis, identifies multiple first distances indicating the distance between each first reference point and each joint, for each of the multiple first reference points and each of the multiple joints, identifies multiple first angles indicating the angle between a line passing through each first reference point and the position of one of the joints included in each combination, and a line passing through each first reference point and the position of the other joint included in each combination, The system includes a model generation unit that generates a learning model by training it with the aforementioned training data. An information processing device characterized by the following:

8. For each of the multiple joints in the hand of a person shown in the image data, multiple first reference points corresponding to the foot of the perpendicular when a perpendicular line is drawn from the position of each joint to the first axis are identified. For each of the plurality of first reference points and each of the plurality of joints, a plurality of first distances are identified that indicate the distance between each first reference point and the position of each joint. For each of the plurality of first reference points and each of the plurality of combinations that include two of the plurality of joints, a plurality of first angles are identified that represent the angle between a line passing through each first reference point and the position of one of the joints included in each combination, and a line passing through each first reference point and the position of the other joint included in each combination. Training data is generated that includes the plurality of first distances and the plurality of first angles, A learning model is generated by training it with the aforementioned training data. A learning method characterized by having a computer perform the processing.

Images