Movement prediction device, movement prediction method, and program

Abstract

A movement prediction device 100 according to the present disclosure comprises: an extraction unit 121 that extracts movement feature data representing a feature of movement from movement data of a person; and a prediction unit 122 that predicts predicted movement feature data, which represents change in the movement feature data, on the basis of exercise data representing details of exercise to be performed by the person and the movement feature data. The extraction unit 121 and the prediction unit 122 can be constructed by machine learning, and can assist with user decision-making using prediction results.

Description

Motion prediction device, motion prediction method, and program

[0001] The present disclosure relates to a motion prediction device, a motion prediction method, and a program.

[0002] Exercise and training may be performed in the hope of improving movement, such as preventing the need for nursing care or frailty. In this case, if the improvement in movement resulting from exercise or training can be predicted in advance, it will motivate the person to continue exercising, etc. For example, in Patent Document 1, the amount of exercise is predicted from walking images for the purpose of health management.

[0003] Japanese Patent Application Laid-Open No. 2018-099267

[0004] However, the technology described in Patent Document 1 only predicts the amount of exercise caused by walking, and is not capable of predicting changes in movement itself due to exercise. This causes a problem in that changes in movement due to exercise cannot be predicted appropriately.

[0005] Therefore, one of the purposes of the present disclosure is to solve the above-mentioned problem of being unable to appropriately predict changes in movement due to exercise.

[0006] A movement prediction device according to an embodiment of the present disclosure includes: an extraction unit that extracts movement feature data representing movement features from movement data of a person; and a prediction unit that predicts predicted movement feature data representing a change in the movement feature data based on movement data representing the content of an exercise performed by the person and the movement feature data. A movement prediction method according to an embodiment of the present disclosure includes: extracting movement feature data representing movement features from the movement data of a person; and predicting predicted movement feature data representing a change in the movement feature data based on movement data representing the content of an exercise performed by the person and the movement feature data. A program according to an embodiment of the present disclosure includes: causing a computer to execute processing to extract movement feature data representing movement features from the movement data of a person; and predicting predicted movement feature data representing a change in the movement feature data based on movement data representing the content of an exercise performed by the person and the movement feature data.

[0007] With the above-described configuration, the present disclosure can appropriately predict changes in motion due to exercise.

[0008] FIG. 1 is a block diagram showing a configuration of an operation prediction device according to the present disclosure. FIG. 2 is a diagram showing processing by an operation prediction device according to the present disclosure. FIG. 3 is a flowchart showing processing operations of an operation prediction device according to the present disclosure. FIG. 4 is a diagram showing processing by an operation prediction device according to the present disclosure. FIG. 5 is a flowchart showing processing operations of an operation prediction device according to the present disclosure. FIG. 6 is a diagram showing a configuration and processing operations of an operation prediction device according to the present disclosure. FIG. 7 is a block diagram showing a hardware configuration of an operation prediction device according to the present disclosure. FIG. 8 is a block diagram showing a configuration of an operation prediction device according to the present disclosure.

[0009] First Embodiment A first embodiment of the present disclosure will be described with reference to the drawings. Note that the drawings may be relevant to any embodiment.

[0010] The movement prediction device 10 in this embodiment predicts a person's possible changes in movement based on movement data of the person and exercise data that the person can perform. For example, the movement prediction device 10 is used in the field of healthcare to predict movements that can be improved by exercises or training that can be performed in the hope of improving the person's movements in order to prevent the need for nursing care or frailty. Examples of the person's movements include walking and standing and sitting movements. The following description will use five standing and sitting movements as an example. However, the movements targeted by the movement prediction device 10 may be any movement, and the movement prediction device 10 is not limited to use in the field of healthcare and may be used to predict changes in any movement in any field.

[0011] The movement prediction device 10 is configured with one or more information processing devices each including a calculation device and a storage device. As shown in FIG. 1 , the movement prediction device 10 includes a human movement feature extraction unit 11, a movement feature extraction unit 12, a movement feature prediction unit 13, a movement generation unit 14, a movement generation learning unit 15, and a movement feature learning unit 16. The functions of the movement feature extraction unit 11, the movement feature extraction unit 12, the movement feature prediction unit 13, the movement generation unit 14, the movement generation learning unit 15, and the movement feature learning unit 16 can be realized by the calculation device executing a program for realizing each function stored in the storage device. Below, each component will be described in detail along with the operation of the movement prediction device 10. Below, we will first describe the inference operation of the movement prediction device 10 to predict a movement, and then describe the learning operation for building the movement prediction device 10 using machine learning to realize the inference operation.

[0012] 2 is a block diagram showing the configuration of the behavior prediction device 10, and illustrates the flow of data during inference operations. FIG. 3 is a flowchart showing the inference operations of the behavior prediction device 10.

[0013] The human motion feature extraction unit 11 and the motion feature extraction unit 12 (extraction unit) first acquire motion data of a person to be predicted (F1 and F2 in FIG. 2). At this time, the motions of the person that can be improved by exercises or training that can be performed in the hope of improving the person's motion for, for example, nursing care prevention or frailty prevention are predicted. The target person is, for example, an elderly person, and motion prediction is performed for nursing care prevention or frailty prevention. The human motion data is, for example, data extracted from video data capturing the person's five standing and sitting movements. A specific example of the motion data is time-series data of the coordinate values ​​(e.g., (x, y, z)) and rotation angles (e.g., Euler angles) of the person's joints extracted from 64 consecutive frames of video data. However, the motion data input to the motion prediction device 10 may be the video data itself, or the human motion feature extraction unit 11 and the motion feature extraction unit 12 may extract motion data such as the above-mentioned joint coordinate values ​​from the video data.

[0014] The human motion feature extraction unit 11 and the motion feature extraction unit 12 then extract vector data that quantifies motion features from the input motion data, such as coordinate values ​​(step S1 in FIG. 3 ). Specifically, the human motion feature extraction unit 11 and the motion feature extraction unit 12 extract, as vector data, motion features, such as motion duration, the time difference between the first and last periodic motion, the joint angles of the back and knees in a standing position, the range of motion of the back and knees during the motion, left-right balance and unsteadiness, and the sequence of joint movement. In this embodiment, the case of extracting the maximum degree of back curvature (hereinafter referred to as "maximum curvature") during standing and sitting motions will be described as an example. The degree of maximum curvature, which is the extracted motion feature, may be a continuous value or a discrete value. In this embodiment, however, it is a discrete value divided into five levels. In other words, the human motion feature extraction unit 11 and the motion feature extraction unit 12 extract vector data corresponding to one of five levels of maximum back curvature during standing and sitting motions. The vector data is, for example, vector data having 256 elements or 512 elements obtained by dimensionally compressing the motion data.

[0015] Here, the human motion feature extraction unit 11 extracts human motion feature vector data (human motion feature data) representing the motion features of each individual person from the motion data. In other words, the human motion feature extraction unit 11 extracts human motion feature vector data so that it becomes similar vector data for any motion of the same person, regardless of the characteristics of the motion itself. In other words, in this embodiment, regardless of the value of the maximum curvature of the input motion data, similar vector data is output as long as the motion is that of the same person. Then, the human motion feature extraction unit 11 outputs the extracted human motion feature vector data to the motion generation unit 14 (F3 in FIG. 2).

[0016] The human motion feature extraction unit 11 is configured with a machine learning model generated in advance by machine learning to extract human motion feature vector data from motion data as described above. For example, the machine learning model constituting the human motion feature extraction unit 11 is generated by machine learning using teacher motion data previously acquired from a person and teacher labels, which are values ​​from 1 to 5 indicating the maximum degree of curvature corresponding to the motion, as teacher data. The machine learning model is generated by comparing the human motion feature vectors extracted from the motion data and the teacher motion data using a loss function, and optimizing the weights of the neural network constituting the machine learning model so that the vector data approximates to each other if the person is the same. However, the human motion feature extraction unit 11 may be a machine learning model generated by any machine learning method and is not necessarily limited to being configured with a machine learning model. It may be configured with any configuration, such as a preset arithmetic formula.

[0017] Furthermore, the motion feature extraction unit 12 extracts motion feature vector data (motion feature data) representing the motion features of the motion itself from the motion data. In other words, the person motion feature extraction unit 11 extracts motion feature vector data so that it becomes similar vector data for any identical motion, regardless of the motion features of the individual person. In other words, in this embodiment, even if the input motion data is from different people, if the maximum curvature value is equivalent, it outputs similar vector data. Then, the motion feature extraction unit 12 outputs the extracted motion feature vector data to the motion feature prediction unit 13 (F4 in FIG. 2).

[0018] The movement feature extraction unit 12 is configured with a machine learning model generated in advance by machine learning to extract movement feature vector data from movement data as described above. For example, the machine learning model constituting the movement feature extraction unit 12 is generated by machine learning using the above-mentioned teacher movement data and teacher labels obtained in advance from multiple people as teacher data, comparing the movement feature vectors extracted from the movement data and the teacher movement data using a loss function, and optimizing the weights of the neural network constituting the machine learning model so that the vector data approximates if the teacher labels are the same. However, the movement feature extraction unit 12 may be a machine learning model generated by any machine learning method and is not necessarily limited to being configured with a machine learning model, and may have any configuration, such as being configured with a preset arithmetic formula.

[0019] Next, the motion feature prediction unit 13 (prediction unit) acquires motion data of the person whose motion is to be predicted (F5 in FIG. 2). The motion data of the person represents the type of exercise the person can perform, such as strength training, aerobic exercise, or balance training. Specific examples of the motion data include vector data that quantifies leg press (load, number of repetitions, frequency), treadmill (speed, time, frequency), and one-legged standing (time, frequency), or binary data (1, 0) that indicates whether the exercise corresponding to the training name was performed.

[0020] The movement feature prediction unit 13 then predicts predicted movement feature data representing a change in the movement of the person being the target of movement prediction, based on the movement feature vector data and the movement data output from the movement feature extraction unit 12 (step S2 in FIG. 3). Specifically, the movement feature prediction unit 13 calculates predicted movement feature vector data, which is vector data representing the changed movement, from vector data obtained by combining the movement feature vector data and the movement vector data, which is the movement data. The movement feature prediction unit 13 then outputs the predicted movement feature vector data to the movement generation unit 14 (F6 in FIG. 2).

[0021] Here, the movement feature prediction unit 13 is configured with a machine learning model generated in advance by machine learning so as to predict predicted movement feature data from movement feature vector data and movement data as described above. For example, the machine learning model constituting the movement feature prediction unit 13 is generated by machine learning using the above-described teacher movement data and teacher labels as teacher data, optimizing the weights of a neural network so that predicted predicted movement feature data matches the teacher labels, or so that predicted movement feature data predicted from movement data and teacher movement data match. However, the movement feature prediction unit 13 may be a machine learning model generated by any machine learning method, and is not necessarily limited to being configured with a machine learning model, and may have any configuration, such as being configured with a preset arithmetic formula.

[0022] Next, the motion generation unit 14 (generation unit) generates predicted motion data, which is data on the predicted motion of the person, based on the predicted motion feature data and the person motion feature data (step S3 in FIG. 3 ). Specifically, the motion generation unit 14 calculates predicted motion data representing the predicted motion after the change from vector data combining the predicted motion feature vector data and the person motion feature vector data. A specific example of the predicted motion data, similar to the motion data described above, is time-series vector data of the coordinate values ​​(e.g., (x, y, z)) and rotation angles (e.g., Euler angles) of the person's joints. The motion generation unit 14 then outputs the generated predicted motion data (F7 in FIG. 2 ). At this time, the motion generation unit 14 generates video data showing the person performing the predicted motion, i.e., standing up and sitting down, from the time-series predicted motion data as described above, and outputs the video data (step S5 in FIG. 3 ).

[0023] Here, the movement generation unit 14 is configured with a machine learning model generated in advance by machine learning so as to generate predicted movement data from predicted movement feature data and human movement feature data as described above. For example, the machine learning model constituting the movement generation unit 14 is generated by machine learning using the above-described teacher movement data and teacher labels as teacher data to optimize the weights of a neural network so that the generated predicted movement data matches the teacher movement data. However, the movement generation unit 14 may be a machine learning model generated by any machine learning method, and is not necessarily limited to being configured with a machine learning model, and may have any configuration, such as being configured with a preset arithmetic formula.

[0024] The above-described movement feature extraction unit 12 (extraction unit) receives the predicted movement data generated by the movement generation unit 14 (F8 in FIG. 2 ) and extracts predicted movement feature data (second predicted movement feature data) representing the movement features of the predicted movement data from the predicted movement data (step S4 in FIG. 3 ). At this time, the movement feature extraction unit 12 extracts movement feature vector data representing the predicted movement features from the vector data, which is the predicted movement data, in the same manner as described above. The movement feature extraction unit 12 then outputs the calculated predicted movement feature data. At this time, the movement feature extraction unit 12 outputs comparison data comparing the movement feature data extracted from the movement data before prediction with the predicted movement feature data (step S5 in FIG. 3 ). For example, the movement feature extraction unit 12 further extracts characteristics related to the person's movements that can be extracted from each of the movement feature data before and after prediction, and outputs these characteristics so that they can be compared before and after prediction. As an example, assume that characteristics that can be extracted from the movement feature data include endurance, muscle strength, flexibility, balance, etc. In this case, for example, as shown in Fig. 4, the characteristics before prediction (dotted line) and the characteristics after prediction (solid line) are displayed for comparison on the same graph. Note that the movement feature extraction unit 12 can also generate and output advice data according to the comparison result between the characteristics before and after prediction, as shown in Fig. 4. For example, the movement feature extraction unit 12 may store advice data according to each change in characteristics in advance, and, as a result of actually predicting the person's movement, select and output advice data according to the change in characteristics.

[0025] As described above, the movement prediction device 10 of this embodiment can appropriately predict subsequent movements from the movement data and exercise data of a person. For example, by using the movement prediction device 10 in the fields of medicine and healthcare, it is possible to predict improvements in movements resulting from exercise or training in order to prevent the need for nursing care or frailty in a target person. In this case, doctors and medical professionals who provide guidance on preventing nursing care or frailty to a target person can refer to the prediction results from the movement prediction device 10 to help them make decisions regarding the content of exercise or training guidance.

[0026] Next, the configuration and learning operation for constructing the above-mentioned behavior prediction device 10 by machine learning will be described. Fig. 5 is a block diagram showing the configuration of the behavior prediction device 10, and illustrates the flow of data during the learning operation. Fig. 6 is a flowchart showing the learning operation of the behavior prediction device 10.

[0027] First, when the action prediction device 10 performs machine learning, teacher data is prepared that corresponds to the action data of the person described above and consists of teacher action data that represents the person's subsequent changes in action, and teacher labels that correspond to such actions.

[0028] As described above, the motion prediction device 10 inputs motion data of a person to the person motion feature extraction unit 11 and the motion feature extraction unit 12 (extraction unit) (F1 and F2 in FIG. 5 ), and extracts motion feature data and motion feature data (step S11 in FIG. 6 ). Subsequently, as described above, the motion prediction device 10 inputs the motion feature data and the movement data to the motion feature prediction unit 13 (prediction unit) (F4 and F5 in FIG. 5 ), and calculates predicted motion feature data (step S12 in FIG. 6 ). Furthermore, as described above, the motion prediction device 10 inputs the person motion feature data and predicted motion feature data to the motion generation unit 14 (generation unit) (F3 and F6 in FIG. 5 ), and generates predicted motion data (step S13 in FIG. 6 ).

[0029] Then, as shown by the dotted line in FIG. 5 , the movement generation learning unit 15 (learning unit) uses the generated predicted movement data and teacher data to perform machine learning to optimize the weights of the neural networks constituting the human movement feature extraction unit 11, the movement feature extraction unit 12, and the movement generation unit 14. At this time, the movement generation learning unit 15 may compare the predicted movement data generated from the movement data with the teacher movement data of the teacher data using a loss function, and perform machine learning to optimize the weights of the neural networks constituting the movement generation unit 14. Alternatively, the movement generation learning unit 15 may compare, for example, the human movement feature data and the movement feature data extracted from the movement data with the human movement feature data and the movement feature data extracted from the teacher movement data of the teacher data using a loss function, and perform machine learning to optimize the weights of the neural networks constituting the human movement feature extraction unit 11 and the movement feature extraction unit 12. However, the movement generation learning unit 15 may use any method for machine learning the neural networks constituting the human movement feature extraction unit 11, the movement feature extraction unit 12, and the movement generation unit 14. Furthermore, the movement generation learning unit 15 is not limited to learning all of the human movement feature extraction unit 11, the movement feature extraction unit 12, and the movement generation unit 14, but may learn only one of them.

[0030] Furthermore, as indicated by the dotted line in FIG. 5 , the movement feature learning unit 16 (learning unit) performs machine learning using the generated predicted movement data and teacher data to optimize the weights of the neural network constituting the movement feature prediction unit 13. At this time, the movement feature learning unit 16 may compare, using a loss function, predicted movement feature data predicted based on movement feature data extracted from the generated predicted movement data with predicted movement feature data predicted based on movement feature data extracted from the teacher movement data, and perform machine learning to optimize the weights of the neural network constituting the movement feature prediction unit 13. Alternatively, the movement feature learning unit 16 may perform machine learning of the movement feature prediction unit 13 by comparing, using a loss function, predicted movement feature data predicted from the above-mentioned movement data with teacher labels of the teacher data. However, the movement feature learning unit 16 may perform machine learning of the neural network constituting the movement feature prediction unit 13 using any method.

[0031] Here, the learning processes by the above-described motion generation learning unit 15 and motion feature learning unit 16 may be performed simultaneously. This allows machine learning to be performed using more information than when each machine learning model is trained independently, and therefore it is expected that the prediction accuracy of each model will be improved.

[0032] Note that the movement prediction device 10 in this embodiment does not necessarily have to include the person movement feature extraction unit 11 described above, as shown in Fig. 7. According to the movement prediction device 10 shown in Fig. 7, during inference, first, the movement feature extraction unit 12 (extraction unit) extracts movement feature data using person movement data as input, and the movement feature prediction unit 13 (prediction unit) calculates predicted movement feature data using the movement feature data and motion data as input. Then, the movement generation unit 14 (generation unit) generates predicted movement data using the person movement feature data as input. Furthermore, the movement feature extraction unit 12 extracts predicted movement feature data from the generated predicted movement data.

[0033] In this case, during the learning operation, the movement generation learning unit 15 and the movement feature learning unit 16 perform machine learning of the movement feature extraction unit 12, the movement generation unit 14, and the movement feature prediction unit 13 using the predicted movement data and teacher data, as described above.

[0034] Second Embodiment Next, a second embodiment of the present disclosure will be described with reference to the drawings. This embodiment shows an outline of the configuration of the action prediction device described in the above embodiment. Note that Figures 8 and 9 are diagrams for explaining the configuration, and these drawings may be relevant to any of the embodiments.

[0035] First, the hardware configuration of the action prediction device 100 will be described with reference to Fig. 8. The action prediction device 100 is configured as a general information processing device, and is equipped with the following hardware configuration, for example: CPU (Central Processing Unit) 101 (arithmetic device) ROM (Read Only Memory) 102 (storage device) RAM (Random Access Memory) 103 (storage device) Programs 104 loaded into RAM 103 Storage device 105 storing programs 104 Drive device 106 for reading and writing data from and to a storage medium 110 external to the information processing device Communication interface 107 for connecting to a communication network 111 external to the information processing device Input / output interface 108 for inputting and outputting data Bus 109 for connecting the various components

[0036] 8 shows an example of the hardware configuration of the information processing device that is the operation prediction device 100, and the hardware configuration of the information processing device is not limited to the above-described case. For example, the information processing device may be configured with a part of the above-described configuration, such as not including the drive device 106. Furthermore, instead of the above-described CPU, the information processing device may use a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating point number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination thereof.

[0037] The CPU 101 acquires and executes the program group 104, thereby constructing and equipping the action prediction device 100 with the extraction unit 121 and prediction unit 122 shown in FIG. 9 . The program group 104 is stored in advance in the storage device 105 or the ROM 102, for example, and is loaded into the RAM 103 and executed by the CPU 101 as needed. The program group 104 may be supplied to the CPU 101 via the communication network 111, or may be stored in advance in the storage medium 110, with the drive device 106 reading out the programs and supplying them to the CPU 101. However, the extraction unit 121 and prediction unit 122 described above may be constructed using dedicated electronic circuits for realizing such means.

[0038] The extraction unit 121 extracts movement feature data representing movement features from the movement data of the person. The prediction unit 122 predicts predicted movement feature data representing changes in the movement feature data based on the movement data representing the content of the exercise performed by the person and the movement feature data.

[0039] With the above-described configuration, the present disclosure can appropriately predict changes in the movement itself due to exercise.

[0040] In addition, at least one of the functions of the extraction unit 121 and the prediction unit 122 described above may be executed by an information processing device installed and connected anywhere on the network, that is, they may be executed by so-called cloud computing.

[0041] The above-described program can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-RWs, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program can also be supplied to a computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable media can be supplied to a computer via wired communication paths such as electric wires and optical fibers, or via wireless communication paths.

[0042] Although the present disclosure has been described above with reference to the above-described embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each of the above-described embodiments can be combined with other embodiments as appropriate.

[0043] <Supplementary Notes> Some or all of the above embodiments can also be described as in the following supplementary notes. Below, an outline of the configurations of a movement prediction device, a movement prediction method, and a program according to the present disclosure will be described. However, the present disclosure is not limited to the following configurations. (Supplementary Note 1) A movement prediction device comprising: an extraction unit that extracts movement feature data representing movement features from movement data of a person; and a prediction unit that predicts predicted movement feature data representing changes in the movement feature data based on movement data representing details of an movement performed by the person and the movement feature data. (Supplementary Note 2) The movement prediction device according to Supplementary Note 1, wherein the extraction unit extracts, as the movement feature data, a plurality of features set according to movements from the movement data. (Supplementary Note 3) The movement prediction device according to Supplementary Note 1, further comprising: a generation unit that generates predicted movement data, which is predicted movement data of the person, based on the predicted movement feature data. (Supplementary Note 4) The movement prediction device according to Supplementary Note 3, wherein the extraction unit extracts second predicted movement feature data representing movement features from the predicted movement data. (Supplementary Note 5) The movement prediction device according to Supplementary Note 1, wherein the extraction unit extracts person movement feature data representing movement features of a person who performed the movement from the movement data, and comprises a generation unit that generates predicted movement data, which is movement data predicted for the person, based on the predicted movement feature data and the person movement feature data. (Supplementary Note 6) The movement prediction device according to Supplementary Note 5, wherein the extraction unit extracts a plurality of features set according to the movement from the movement data as the person movement feature data. (Supplementary Note 7) The movement prediction device according to Supplementary Note 5, wherein the extraction unit extracts second predicted movement feature data representing movement features from the predicted movement data. (Supplementary Note 8) The movement prediction device according to Supplementary Note 5, comprising a learning unit that performs machine learning on at least one of the extraction unit, the prediction unit, and the generation unit using teacher data consisting of teacher movement data of a person and a teacher label corresponding to the teacher movement data of the person, and the predicted movement data.(Supplementary Note 9) A movement prediction method comprising: extracting movement feature data representing movement features from movement data of a person; and predicting predicted movement feature data representing changes in the movement feature data based on the movement feature data and movement data representing the content of the movement performed by the person. (Supplementary Note 10) A movement prediction method according to Supplementary Note 9, wherein predicted movement data, which is predicted movement data of the person, is generated based on the predicted movement feature data. (Supplementary Note 11) A movement prediction method according to Supplementary Note 10, wherein second predicted movement feature data representing movement features is extracted from the predicted movement data. (Supplementary Note 12) A movement prediction method according to Supplementary Note 9, wherein person movement feature data representing movement features of the person who performed the movement is extracted from the movement data, and generated predicted movement data, which is predicted movement data of the person, based on the predicted movement feature data and the person movement feature data. (Supplementary Note 13) A movement prediction method according to Supplementary Note 12, wherein second predicted movement feature data representing movement features is extracted from the predicted movement data. (Supplementary Note 14) The movement prediction method according to Supplementary Note 12, comprising: using teacher data consisting of teacher movement data of a person and teacher labels corresponding to the teacher movement data of the person, and the predicted movement data, machine learning is performed to generate at least one of a model that extracts the movement feature data, a model that extracts the person movement feature data, a model that predicts the predicted movement feature data, and a model that generates the predicted movement data. (Supplementary Note 15) A computer-readable storage medium storing a program that causes a computer to execute processes of: extracting movement feature data that represent movement features from the person's movement data, and predicting predicted movement feature data that represent changes in the movement feature data based on the movement feature data and movement data that represent the content of an exercise performed by the person.

[0044] REFERENCE SIGNS LIST 10 Motion prediction device 11 Human motion feature extraction unit 12 Motion feature extraction unit 13 Motion feature prediction unit 14 Motion generation unit 15 Motion generation learning unit 16 Motion feature learning unit 100 Motion prediction device 101 CPU 102 ROM 103 RAM 104 Program group 105 Storage device 106 Drive device 107 Communication interface 108 Input / output interface 109 Bus 110 Storage medium 111 Communication network 121 Extraction unit 122 Prediction unit

Claims

1. A movement prediction device comprising: an extraction unit that extracts movement feature data representing movement characteristics from a person's movement data; and a prediction unit that predicts predicted movement feature data representing changes in the movement feature data based on movement data representing the content of the movement performed by the person and the movement feature data.

2. A motion prediction device according to claim 1, wherein the extraction unit extracts a plurality of features set according to the motion from the motion data as the motion feature data.

3. A motion prediction device according to claim 1, comprising a generation unit that generates predicted motion data, which is predicted motion data of a person, based on the predicted motion feature data.

4. A motion prediction device according to claim 3, wherein the extraction unit extracts second predicted motion feature data representing features of the motion from the predicted motion data.

5. A motion prediction device as described in claim 1, wherein the extraction unit extracts person motion feature data representing the motion features of the person who performed the motion from the motion data, and the motion prediction device includes a generation unit that generates predicted motion data, which is predicted motion data of the person, based on the predicted motion feature data and the person motion feature data.

6. A motion prediction device according to claim 5, wherein the extraction unit extracts a plurality of features set according to the motion from the motion data as the human motion feature data.

7. A motion prediction device according to claim 5, wherein the extraction unit extracts second predicted motion feature data representing features of the motion from the predicted motion data.

8. A movement prediction device as described in claim 5, comprising a learning unit that uses teacher data consisting of teacher movement data of a person and a teacher label corresponding to the teacher movement data of the person, and the predicted movement data to perform machine learning on at least one of the extraction unit, the prediction unit, and the generation unit.

9. A movement prediction method comprising: extracting movement feature data representing movement characteristics from a person's movement data; and predicting predicted movement feature data representing a change in the movement feature data based on the movement data representing the content of the movement performed by the person and the movement feature data.

10. A motion prediction method according to claim 9, further comprising the step of generating predicted motion data, which is predicted motion data of a person, based on the predicted motion feature data.

11. A motion prediction method according to claim 10, further comprising the step of extracting second predicted motion feature data representing a feature of the motion from the predicted motion data.

12. A movement prediction method as claimed in claim 9, comprising the steps of: extracting person movement feature data representing the movement features of the person who performed the movement from the movement data; and generating predicted movement data, which is predicted movement data of the person, based on the predicted movement feature data and the person movement feature data.

13. A motion prediction method according to claim 12, further comprising the step of extracting second predicted motion feature data representing a feature of the motion from the predicted motion data.

14. A movement prediction method as described in claim 12, comprising: using teacher data consisting of teacher movement data of a person and a teacher label corresponding to the teacher movement data of the person, and the predicted movement data, machine learning at least one of a model that extracts the movement feature data, a model that extracts the person movement feature data, a model that predicts the predicted movement feature data, and a model that generates the predicted movement data.

15. A computer-readable storage medium storing a program that causes a computer to execute a process of extracting movement feature data representing movement characteristics from a person's movement data, and predicting predicted movement feature data representing changes in the movement feature data based on the movement data representing the content of the movement performed by the person and the movement feature data.

Images