Information processing device, information processing method, and information processing system

Abstract

The information processing device 1 is an estimation model created based on first motion data indicating the movement of a first part of the body and second motion data indicating the movement of a second part, which are acquired at the same time by a first device worn on a first part of the body and a second device worn on a second part of the body different from the first part, and has a memory unit 12 that stores the estimation model that outputs motion data indicating the movement of the second part when motion data indicating the movement of the first part is input, a receiving unit 131 that receives actual motion data indicating the movement of the first part of the subject to be analyzed, acquired by a device worn on the first part of the subject, an estimation unit 133 that inputs the actual motion data into the estimation model and acquires estimated motion data indicating the movement of the second part of the subject from the estimation model, and an output unit 134 that outputs the estimated motion data.

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and an information processing system that analyze motion data acquired by a device worn on the body. [Background technology]

[0002] Conventionally, there are known techniques for analyzing the motion state of a human body. Patent Document 1 discloses a technique capable of estimating an index related to the motion state of a human body based on various data collected in a time series manner. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-010562 A Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, people have become highly interested in understanding their own health and physical activity. In order to understand one's own health and physical activity, there is a method of analyzing body movements by attaching sensors and devices to the human body, as described in the above-mentioned Patent Document 1. However, the conventional technology has a problem in that it is necessary to attach sensors and devices to all parts of the body whose movements one wishes to understand.

[0005] Therefore, the present invention has been made in consideration of these points, and aims to reduce the burden on users when they wear sensors and devices on their bodies to analyze their own body movements. [Means for solving the problem]

[0006] An information processing device according to a first aspect of the present invention is an estimation model created based on first motion data indicating movement of a first part of the body and second motion data indicating movement of the second part, which are acquired at the same time by a first device worn on a first part of the body and a second device worn on a second part of the body different from the first part, and the information processing device has a memory unit that stores the estimation model and outputs motion data indicating movement of the second part when motion data indicating the movement of the first part is input, a receiving unit that receives measured motion data indicating the movement of the first part of a subject to be analyzed, which is acquired by a device worn on the first part of the subject, an estimation unit that inputs the measured motion data into the estimation model and acquires estimated motion data indicating movement of the second part of the subject from the estimation model, and an output unit that outputs the estimated motion data.

[0007] The memory unit may store the estimation model for each type of movement, the receiving unit may receive the type of movement, and the estimation unit may input the measured motion data to the estimation model corresponding to the received type of movement and obtain the estimated motion data from the estimation model.

[0008] The memory unit may store the estimation model for each type of movement, and the information processing device may further have an identification unit that identifies the type of movement indicated by the received measured motion data, and the estimation unit may input the measured motion data to the estimation model corresponding to the identified type of movement and obtain the estimated motion data from the estimation model.

[0009] The estimation model may have multiple sets of the first motion data and the second motion data, identify the first motion data that has a similarity to the input measured motion data that is greater than or equal to a predetermined threshold, and output the second motion data associated with the identified first motion data.

[0010] The estimation model is a machine learning model trained by machine learning using multiple sets of the first motion data and the second motion data as teacher data, and may output the most probable motion data as the motion data of the second part corresponding to the input actual motion data.

[0011] The estimation model may be created based on the first motion data, the second motion data, and state data indicating the state of the body when the movement indicated by the first motion data and the movement indicated by the second motion data are performed, and when motion data indicating the movement of the first part is input, it may output motion data indicating the movement of the second part and state estimation data which is the result of estimating the state of the body, and the estimation unit may input the measured motion data to the estimation model and obtain the estimated motion data and the state estimation data from the estimation model, and the output unit may output the estimated motion data and the state estimation data.

[0012] The first location may be a single location on the body and the second location may be one or more locations on the body.

[0013] The first part may be the neck, and the second part may be a part of the body other than the neck.

[0014] The estimation model may be created based on the first motion data, the second motion data, and third motion data indicating the movement of the third part acquired at the same time by the first device, the second device, and a third device worn on a third part different from the first part and the second part, and may be a model that outputs motion data indicating the movement of the second part when the motion data indicating the movement of the first part and the motion data indicating the movement of the third part are input, and the receiving unit may receive first actual motion data indicating the movement of the first part of the subject acquired by a device worn on the first part of the subject, and third actual motion data indicating the movement of the third part of the subject acquired by a device worn on the third part of the subject, and the estimation unit may input the first actual motion data and the third actual motion data to the estimation model, and acquire the estimated motion data indicating the movement of the second part of the subject from the estimation model.

[0015] The estimation unit may obtain, as the estimated motion data indicating the movement of the second part of the subject, motion data that is a weighted average of estimated motion data based on the first actual motion data and estimated motion data based on the third actual motion data, so that the actual motion data of a part of the first actual motion data and the third actual motion data that is closer to the second part is reflected to a higher extent in the estimated motion data indicating the movement of the second part of the subject.

[0016] An information processing method according to a second aspect of the present invention is an estimation model created based on first motion data indicating movement of a first part of the body and second motion data indicating movement of the second part, which are acquired at the same time by a first device worn on a first part of the body and a second device worn on a second part of the body different from the first part, and which is executed by a computer having a memory unit that stores the estimation model and outputs motion data indicating movement of the second part when motion data indicating the movement of the first part is input, the method comprising: a receiving step of receiving actual motion data indicating the movement of the first part of a subject, acquired by a device worn on the first part of the subject, an estimation step of inputting the actual motion data into the estimation model and acquiring estimated motion data indicating the movement of the second part of the subject from the estimation model; and an output step of outputting the estimated motion data.

[0017] An information processing system according to a third aspect of the present invention includes an information processing device and an information terminal. The information processing device has an estimation model created based on first motion data indicating a movement of a first part of the body and second motion data indicating a movement of the second part, which are acquired at the same time by a first device worn on a first part of the body and a second device worn on a second part of the body different from the first part, and the estimation model has a memory unit that stores the estimation model and outputs motion data indicating the movement of the second part when motion data indicating the movement of the first part is input, a receiving unit that receives measured motion data indicating the movement of the first part of a subject to be analyzed, which is acquired by a device worn on the first part of the subject, an estimation unit that inputs the measured motion data into the estimation model and acquires estimated motion data indicating the movement of the second part of the subject from the estimation model, and an output unit that outputs the estimated motion data. The information terminal has a setting receiving unit that receives settings related to the device worn on the first part of the subject, and a terminal communication unit that receives the estimated motion data. Effect of the Invention

[0018] According to the present invention, when a user wears a sensor or device on his / her body to analyze his / her body movements, it is possible to reduce the burden on the user. [Brief description of the drawings]

[0019] [Figure 1] FIG. 2 is a diagram showing a configuration of an information processing system S. [Diagram 2] 1 is a diagram illustrating a configuration of an information processing device 1. [Diagram 3] FIG. 2 is a diagram illustrating an example of a motion data set. [Figure 4] FIG. 1 is a diagram illustrating an example of machine learning data. [Figure 5A] FIG. 11 is a diagram showing an example of measured motion data indicating neck movement. [Figure 5B] FIG. 13 is a diagram showing an example of estimated motion data indicating the rotation angle of the knee estimated using an estimation model from measured motion data indicating the movement of the neck. [Figure 6A] FIG. 13 is a diagram showing an example of measured motion data indicating the extension speed of the right forearm when a user is serving in tennis. [Figure 6B] FIG. 13 is a diagram showing an example of estimated motion data indicating the extension speed of the right forearm estimated using an estimation model from actually measured motion data indicating the movement of the neck when a user is serving in tennis. [Figure 7A] FIG. 13 is a diagram showing an example of measured motion data indicating the internal rotation speed of the right forearm when a user is batting a baseball. [Figure 7B] FIG. 13 is a diagram showing an example of estimated motion data indicating the internal rotation speed of the right forearm, estimated using an estimation model from measured motion data indicating the movement of the neck when a user is batting a baseball. [Figure 8] FIG. 2 is a diagram showing a configuration of an information terminal 2. [Figure 9] 3 is a flowchart showing the flow of processing executed by the information processing device 1. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] [Outline of Information Processing System S] An overview of an information processing system S according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an overview of the operation of the information processing system S. The information processing system S includes an information processing device 1, an information terminal 2, and a device D. The information processing system S may also include other devices such as a server and a terminal.

[0021] The information processing system S is a system for estimating the movement of one or more second parts (e.g., knees) different from the first part of the body of a subject to be analyzed, based on the movement of the first part acquired by a device D attached to the first part (e.g., neck) of the body. The first part is, for example, the position of the skin behind the cervical vertebrae.

[0022] The information processing device 1 is a computer such as a server that performs this estimation. The information processing device 1, the information terminal 2, and the device D are capable of communicating with each other. It is assumed that the information processing device 1 is provided in a location (e.g., on the cloud) different from the location where the user U uses the information terminal 2, but it may be provided in the location where the user U uses the information terminal 2.

[0023] The information terminal 2 is a terminal used by a user U who is exercising. The user U uses the information terminal 2 to set the device D and to obtain an estimation result from the information processing device 1. The information terminal 2 is preferably a portable terminal such as a smartphone or a tablet personal computer, but may be a stationary terminal such as a desktop personal computer.

[0024] The device D is a sensor capable of detecting the movement of a body part, and is, for example, an acceleration sensor, an angular velocity sensor, or a sensor using both sensors. The device D may be a wearable terminal that can be attached to a user U who is exercising. As an example, an adhesive member is attached to one surface of the device D to facilitate adhesion to the skin of the user U. The device D may be directly attached to the skin of the user U, or may be attached over clothing that is in close contact with the user U. The device D may not be attached with an adhesive member and may have a metal part. In this case, the user U can wear the device D in a state where it is in close contact with the user U, for example, by inserting the device D inside the sportswear at the position of the collar on the back side of the neck of the user U in the sportswear and fixing the device D with a magnet from the outside of the sportswear.

[0025] The acceleration sensor measures the rate of change (acceleration) of the motion speed of the user U while he is moving. The acceleration sensor detects, for example, acceleration components (acceleration signals) along each of three mutually orthogonal axial directions and outputs them as acceleration data. The angular velocity sensor measures changes in the motion direction (angular velocity) while the user U is moving. The angular velocity sensor detects, for example, angular velocity components (angular velocity signals) occurring in the rotation direction of the rotational motion along each of the three mutually orthogonal axes and outputs them as angular velocity data.

[0026] In recent years, people have become very interested in understanding their own health and physical activity. In order to understand one's own health and physical activity, there is a method of analyzing body movements by attaching sensors and devices to the human body. However, with conventional technology, there is a problem in that it is necessary to attach sensors and devices to all parts of the body whose movements one wishes to understand.

[0027] Therefore, the information processing system S inputs measured motion data indicating the movement of a first part of the subject's body (e.g., neck) to be analyzed, acquired by a device D attached to the first part of the subject's body, into an estimation model, and acquires estimated motion data indicating the movement of one or more second parts (e.g., knees) different from the first part of the subject's body from the estimation model. In other words, the information processing system S can estimate the movement of a certain part based on the actually measured movement of another part.

[0028] The estimation model needs to be created before the estimation. The estimation model is a model created based on first motion data indicating the movement of a first part of the body and second motion data indicating the movement of a second part, which are acquired at the same timing by a first device attached to a first part of the body and a second device attached to a second part of the body. Hereinafter, an overview of the operation of the processing system S will be described with reference to FIG. 1.

[0029] First, before exercising, the user U performs settings on the information terminal 2 to link (pair) the device D with the information terminal 2, sets the part of the body on which the device D will be worn, registers his / her own profile, sets goals, etc.

[0030] When the user U starts exercising, the device D continuously acquires measured motion data indicating the movement of a first part (e.g., the neck) of the body to which the device D is attached. The device D transmits the acquired measured motion data to the information processing device 1.

[0031] The information processing device 1 inputs the received measured motion data indicating the movement of a first body part to the estimation model, and obtains estimated motion data indicating the movement of a second body part (e.g., a knee) different from the first body part from the estimation model. The information processing device 1 uses at least one of the following two types of estimation models.

[0032] The first estimation model is a program or data for outputting estimated motion data by referring to a motion data set having a plurality of sets of first motion data indicating a movement of a first part and second motion data indicating a movement of a second part. This estimation model identifies first motion data having a similarity to the input actual motion data equal to or greater than a predetermined threshold, and outputs the second motion data associated with the identified first motion data as estimated motion data.

[0033] The second estimation model is a machine learning model that is trained by machine learning using multiple sets of first motion data indicating the movement of a first part and second motion data indicating the movement of a second part as training data. This estimation model outputs, as estimated motion data, motion data that is most likely to be the motion data of the second part corresponding to the input actual motion data.

[0034] Then, the information processing device 1 transmits the acquired estimated motion data indicating the movement of the second body part to the information terminal 2. The information terminal 2 displays the received estimated motion data. By referring to the estimated motion data displayed on the information terminal 2, the user U can understand the movement of the second body part (e.g., the knee) where the device D is not attached, and can use this information for future exercise, rest, etc. The information terminal 2 may also display the user U's physical condition (risk of injury, physical fatigue level, etc.).

[0035] In this way, the information processing system S can estimate the movement of one or more second body parts (e.g., knees) different from the first body part based on the movement of the first body part (e.g., neck) acquired by the device D attached to the first body part of the user U. This eliminates the need for the user U to attach sensors and devices to all body parts whose movement is to be understood, as in the past. As a result, the economic, physical, and time burden on the user U is reduced.

[0036] In particular, the inventor found that it is effective to use the cervical vertebrae as the first part because the movement of the cervical vertebrae of the neck is highly linked to the movement of other bones in a human body in many sports movements. As an example, the correlation between the movement of the neck and the movement of the knee is higher than the correlation between the movement of the right arm and the movement of the knee. Therefore, by using the neck as the first part, the estimation accuracy of the movement of the second part different from the neck is improved. The configurations and operations of the information processing device 1 and the information terminal 2 will be described in detail below.

[0037] [Configuration and operation of information processing device 1] The following describes the configuration and operation of the information processing device 1. Fig. 2 is a diagram showing an example of the configuration of the information processing device 1. As shown in Fig. 2, the information processing device 1 includes a device communication unit 11, a storage unit 12, and a control unit 13.

[0038] The device communication unit 11 is a communication interface for communicating with the information terminal 2 and the device D via a communication network such as the Internet.

[0039] The storage unit 12 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The storage unit 12 stores a program executed by the control unit 13. For example, the storage unit 12 stores an information processing program that causes the control unit 13 to function as a receiving unit 131, an identifying unit 132, an estimating unit 133, and an output unit 134. The storage unit 12 stores an estimation model.

[0040] The estimation model is a model created based on first motion data indicating a movement of a first part of the body and second motion data indicating a movement of a second part, which are acquired at the same time by a first device attached to a first part of the body and a second device attached to a second part of the body different from the first part. When the motion data indicating the movement of the first part is input, the estimation model outputs motion data indicating the movement of the second part.

[0041] The first part is, for example, one part on the body, and the second part is, for example, one or more parts on the body. Specifically, the first part is, for example, the neck, and the second part is, for example, a part on the body other than the neck (for example, an arm or a knee). In other words, when motion data indicating the movement of the neck is input, the estimation model can output motion data indicating the movement of the entire body.

[0042] The first device and the second device are sensors capable of detecting the movement of body parts, such as an acceleration sensor, an angular velocity sensor, or a sensor using both sensors in combination. The first device and the second device may be a conventional device different from device D, or may be device D. The "same timing" may be exactly the same timing, or may be timing within a different predetermined range (for example, within a few milliseconds).

[0043] The motion data is data that indicates the movement of a body part, for example, data that shows a two-dimensional graph in which the horizontal axis represents the number of frames and the vertical axis represents angular velocity (unit: radians per second (rad / s) or degrees per second (deg / s)).

[0044] The estimation model may be a model created based on the first motion data, the second motion data, and condition data indicating the state of the body when the movement indicated by the first motion data and the movement indicated by the second motion data are performed. The state of the body is, for example, the risk of injury and the degree of physical fatigue. The learning condition data used to create the model is, for example, data indicating at least one of the results of observing the body state of a person from whom the first motion data and the second motion data for learning are obtained by wearing the first device and the second device, the results of measuring the body state, or the results of asking the person about the parts of the body that hurt.

[0045] The storage unit 12 may store an estimation model for each type of movement. The type of movement may be, for example, a type of sport (jogging, baseball, tennis, etc.), or a type of movement within the same sport (overthrow and underthrow in the case of baseball pitching, etc.). There are two types of estimation models:

[0046] The first estimation model is a model having a plurality of sets of first motion data indicating the movement of a first part and second motion data indicating the movement of a second part. Hereinafter, this model may be referred to as a motion data set. Fig. 3 is a diagram showing an example of the motion data set. In the motion data set, the first motion data, the second motion data, and state data indicating the state of the body are associated with each type of movement.

[0047] The second estimation model is a machine learning model that is machine-learned using a plurality of sets of first motion data indicating the movement of a first part and second motion data indicating the movement of a second part as training data. FIG. 4 is a diagram showing an example of machine learning data. In the machine learning data, the type of movement is associated with the learned machine learning model. Note that machine learning can be performed using a known method. Examples of known methods include convolutional neural network (CNN), logistic regression, support vector machine (SVM), decision tree, random forest, and the like.

[0048] The control unit 13 is, for example, a CPU (Central Processing Unit). The control unit 13 executes an information processing program stored in the storage unit 12, thereby functioning as a receiving unit 131, an identifying unit 132, an estimating unit 133, and an output unit 134.

[0049] The receiving unit 131 receives measured motion data indicating the movement of a first part of the user U, who is a subject to be analyzed, acquired by the device D attached to a first part of the user U. The receiving unit 131 receives the measured motion data indicating the movement of the first part of the user U from the device D via the device communication unit 11, for example.

[0050] The receiving unit 131 may receive the actual measured motion data from the device D immediately after the device D acquires the actual measured motion data. This allows the user U to refer to the estimated motion data indicating the movement of the second part in real time while exercising, and to improve the movement of the body, etc.

[0051] The receiving unit 131 may receive the type of movement. For example, the receiving unit 131 may receive the type of movement input by the user U to the information terminal 2 from the information terminal 2 via the device communication unit 11. In this way, by the user U directly inputting the type of movement, an incorrect type of movement is not used when estimating estimated motion data, and therefore the accuracy of estimation of the estimated motion data is improved.

[0052] However, some users U want to avoid the trouble of inputting the type of movement, or some users U cannot input the type of movement because they are exercising. Therefore, the identification unit 132 identifies the type of movement indicated by the received actual measured motion data indicating the movement of the first part of the user U.

[0053] The identification unit 132, for example, compares the received measured motion data indicating the movement of the first part of the user U with typical motion data indicating the movement of the first part for each type of motion, and identifies the type of motion with the highest similarity. For example, if the measured motion data indicating the movement of the user U's neck is more similar to typical motion data indicating the movement of the neck when a person is playing tennis than to typical motion data indicating the movement of the neck when a person is jogging or playing baseball, the identification unit 132 identifies "tennis" as the type of motion. In this way, by the identification unit 132 identifying the type of motion, the user U does not need to input the type of motion, and the burden on the user U is reduced.

[0054] The estimation unit 133 inputs the received measured motion data into the estimation model, and acquires, from the estimation model, estimated motion data indicating the movement of the second part of the user U. The estimation unit 133 may acquire the estimated motion data without considering the type of movement, but it is preferable to acquire the estimated motion data by considering the type of movement, as described in the next paragraph.

[0055] The estimation unit 133 may input the measured motion data to an estimation model corresponding to the type of movement received by the receiving unit 131 or identified by the identification unit 132, and acquire estimated motion data from the estimation model. This allows the estimation unit 133 to acquire more appropriate estimated motion data using a more appropriate estimation model. The output of estimated motion data by the estimation model will be described below.

[0056] When the estimation model is a motion data set, the estimation model identifies first motion data having a similarity to the input measured motion data equal to or greater than a predetermined threshold. For example, the estimation model of the motion data set shown in FIG. 3 identifies first motion data that corresponds to a type of motion received by the receiving unit 131 or identified by the identifying unit 132 and satisfies a criterion that the similarity to the input measured motion data is X% or greater. The value of X, which is the predetermined threshold, is determined based on the probability that the motion data is more similar than other motion data among a plurality of motion data, and is, for example, 80%. The estimation model may identify the first motion data that is most similar to the input measured motion data.

[0057] The estimation model then outputs the second motion data associated with the identified first motion data. For example, the estimation model of the motion data set shown in FIG. 3 outputs the second motion data stored in the same row as the identified first motion data.

[0058] When the estimation model is a machine learning model, the estimation model outputs the most probable motion data as the motion data of the second body part corresponding to the input measured motion data. The machine learning model used at this time is, for example, a machine learning model corresponding to the type of movement received by the receiving unit 131 or identified by the identifying unit 132 in the machine learning data shown in FIG.

[0059] In either case where the estimation model is a motion data set or a machine learning model, the estimation model may output ideal motion data indicating ideal motion of the second part together with estimated motion data indicating the movement of the second part. This allows the user U to visually grasp the deviation between the estimated motion data and the ideal motion data, which is useful for improving the movement of the body. Note that the estimation model may display the graph of the estimated motion data and the graph of the ideal motion data superimposed (synchronized), or displayed vertically or horizontally.

[0060] The estimation model may calculate a score based on the value indicated by the estimated motion data. The estimation model may also output, as analysis results, the physical condition (risk of injury, physical fatigue, etc.), the state of exercise (speed pace, balance, pitch slide, power, speed, stamina, aggressiveness, stability, insufficient movements, etc.), and suggestions regarding rest and exercise (care methods, rest methods, practice contents, strengthening exercises, etc.) based on the value indicated by the estimated motion data or the calculated score. The estimation model may digitize these analysis results and output them as a graph or radar chart. By visualizing the analysis results in this way, the user U can easily and quickly grasp what he or she is lacking and what he or she should be careful about in terms of exercise.

[0061] The estimation model may digitize the analysis result by referring to publicly known information. The estimation model may, for example, refer to data in which a value indicated by the motion data or a calculated score is associated with a value related to the analysis result, and output a value indicated by the estimated motion data or a value related to the analysis result associated with the calculated score. As an example, the estimation model may output a score range (e.g., xx points or more) calculated based on the angular velocity indicated by the estimated motion data in association with a value indicating the magnitude of the risk of injury (e.g., xx%). Alternatively, the estimation model may output a value related to the analysis result by inputting a value indicated by the estimated motion data or a calculated score into a machine learning model that has been machine-learned using the value indicated by the motion data or the calculated score and the value related to the analysis result as teacher data.

[0062] Here, specific examples of measured motion data and estimated motion data will be described. FIG. 5A is a diagram showing an example of measured motion data showing neck movement. In FIG. 5A, the horizontal axis indicates the number of frames, and the vertical axis indicates the angular velocity (rad / s). In the case where three mutually orthogonal directions are the x-direction, the y-direction, and the z-direction, in FIG. 5A, the graph with "x" corresponds to the angular velocity in the x-direction, the graph with "y" corresponds to the angular velocity in the y-direction, and the graph with "z" corresponds to the angular velocity in the z-direction. FIG. 5B is a diagram showing an example of estimated motion data showing the rotation angle of the knee estimated using an estimation model from the measured motion data showing neck movement. In FIG. 5B, the horizontal axis indicates the number of frames, and the vertical axis indicates the angular velocity (deg / s). FIG. 5 is a result of measurement at 100hz, so 1 frame takes 1 / 100 seconds, and the total measurement time is about 6 seconds ((1 / 100 seconds) x about 600 frames).

[0063] Also, the result of verifying the reliability of the estimation model according to this embodiment will be described. FIG. 6A is a diagram showing an example of measured motion data showing the extension speed of the right forearm when the user is serving in tennis. FIG. 6B is a diagram showing an example of estimated motion data showing the extension speed of the right forearm estimated using the estimation model from measured motion data showing the movement of the neck when the user is serving in tennis. FIG. 7A is a diagram showing an example of measured motion data showing the internal rotation speed of the right forearm when the user is batting a baseball. FIG. 7B is a diagram showing an example of estimated motion data showing the internal rotation speed of the right forearm estimated using the estimation model from measured motion data showing the movement of the neck when the user is batting a baseball. In these four figures, the horizontal axis indicates the number of frames, and the vertical axis indicates the angular velocity (rad / s). Figures 6 and 7 were measured at 60 Hz, so one frame is 1 / 60th of a second, and the total measurement time is approximately 3 seconds ((1 / 60th of a second) x approximately 180 frames) for Figure 6 and approximately 0.67 seconds ((1 / 60th of a second) x approximately 40 frames) for Figure 7.

[0064] The values ​​and shapes of the graphs in Fig. 6A and Fig. 6B were similar, and the values ​​and shapes of the graphs in Fig. 7A and Fig. 7B were also similar. In other words, the graph of estimated motion data (graph showing estimated values) showing the movement of the arm estimated from the measured motion data showing the movement of the neck using the estimation model was similar in value and shape to the graph of measured motion data (graph showing measured values) showing the movement of the arm acquired at the same timing as the acquisition of the measured motion data showing the movement of the neck. From this, it was confirmed that the estimation model according to this embodiment is highly reliable.

[0065] The above description has been given of the process in which the estimation model outputs estimated motion data, but as described above, there are also cases in which the estimation model is created based on the first motion data, the second motion data, and state data indicating the state of the body when the movements indicated by the first motion data and the movements indicated by the second motion data are performed. In this way, when state data is used to create the estimation model, the estimation unit 133 inputs actual motion data indicating the movement of the first part to the estimation model, and obtains estimated motion data and state estimation data, which is a result of estimating the state of the body, from the estimation model. Below, the output of estimated motion data and state estimation data performed by the estimation model will be described.

[0066] When motion data indicating the movement of a first part is input, the estimation model outputs motion data indicating the movement of a second part and state estimation data that is a result of estimating the state of the body. For example, when the estimation model is a motion data set, the estimation model first identifies first motion data having a similarity to the input measured motion data that is equal to or greater than a predetermined threshold, using the above-mentioned method. Next, the estimation model outputs second motion data associated with the identified first motion data as estimated motion data, and outputs state data associated with the identified first motion data as state estimation data.

[0067] Also, for example, when the estimation model is a machine learning model, the estimation model outputs, as estimated motion data, the most probable motion data of the second part corresponding to the input measured motion data, and outputs, as state estimation data, the most probable state data of the body corresponding to the input measured motion data.

[0068] In this way, the estimation unit 133 estimates the physical condition of the user U in addition to the estimated motion data, so that the user U can improve the amount and manner of exercise according to his / her physical condition. As a result, the user U can exercise in a healthy and effective manner for a long period of time.

[0069] As described above, in this embodiment, the user U basically needs to wear one device D on a first part. However, when it is desired to obtain estimated motion data with higher accuracy, the user U may wear a device D on a third part different from the first part and the second part. The process when the device D is worn on multiple parts will be described below.

[0070] In this case, the estimation model used by the estimation unit 133 is a model created based on the first motion data, the second motion data, and the third motion data indicating the movement of the third part, which are acquired at the same timing by a first device attached to a first part of the body, a second device attached to a second part different from the first part of the body, and a third device attached to a third part different from the first part and the second part. In this case, in the motion data set, the first motion data, the second motion data, the third motion data, and the state data indicating the state of the body are associated with each type of movement.

[0071] Furthermore, when the estimation unit 133 uses a machine learning model, one model created based on the first motion data, the second motion data, and the third motion data is used as the machine learning model. Alternatively, two models, a first model created based on the first motion data and the second motion data, and a third model created based on the second motion data and the third motion data, may be used as the machine learning model.

[0072] The receiving unit 131 receives first measured motion data indicating the movement of a first part of the user U, acquired by a device worn on a first part of the user U, and third measured motion data indicating the movement of a third part of the user U, acquired by a device worn on a third part of the user U.

[0073] The estimation unit 133 inputs the first measured motion data and the third measured motion data to the estimation model, and acquires, from the estimation model, estimated motion data indicating the movement of the second part of the user U. Hereinafter, the output of estimated motion data performed by the estimation model will be described.

[0074] When motion data indicating the movement of a first part and motion data indicating the movement of a third part are input, the estimation model outputs motion data indicating the movement of a second part. When the estimation model is a motion data set, the estimation model identifies first motion data having a similarity to the input first actual motion data equal to or greater than a predetermined threshold, and third motion data having a similarity to the input third actual motion data equal to or greater than a predetermined threshold. Next, the estimation model identifies second motion data associated with the identified first motion data and second motion data associated with the identified third motion data as estimated motion data.

[0075] When the estimation model is one machine learning model, the estimation model specifies, as estimated motion data, the most likely motion data of the second part corresponding to the first measured motion data and the third measured motion data input to the estimation model. In this case, the estimation unit 133 inputs the first measured motion data and the third measured motion data to the machine learning model, and sets the motion data indicating the movement of the second part output from the machine learning model as estimated motion data.

[0076] When the estimation model is two machine learning models, the estimation model identifies estimated motion data based on the most probable motion data of the second part corresponding to the first measured motion data input to the first model, and the most probable motion data of the second part corresponding to the third measured motion data input to the third model.

[0077] When the estimation model is a motion data set or when the estimation model is two machine learning models, the estimation model makes it so that, of the first measured motion data and the third measured motion data, the measured motion data of the part closer to the second part is reflected at a higher rate in the estimated motion data indicating the movement of the second part of the user U. Specifically, the estimation model outputs, for example, motion data obtained by weighting the first estimated motion data based on the first measured motion data and the third estimated motion data based on the third measured motion data as the second estimated motion data of the user U. For example, the estimation model may output, as the second estimated motion data, a value obtained by multiplying a value indicated by the first estimated motion data by the distance from the first part to the second part and a value indicated by the third estimated motion data by the distance from the third part to the second part, and dividing the sum of the distance from the first part to the second part and the distance from the third part to the second part.

[0078] In this way, the estimation model outputs estimated motion data based on multiple actual motion data acquired by the device D worn at multiple locations, so that even if the value of the estimated motion data based on one location deviates slightly from the actual value, the estimated motion data is corrected using the value of the estimated motion data based on another location. As a result, the reliability of the estimated motion data finally output is improved. Furthermore, if the value of the estimated motion data acquired based on each of the multiple actual motion data acquired by the device D worn at multiple locations is weighted according to the distance from the device D to the second part, the reliability of the estimated motion data finally output is further improved.

[0079] The output unit 134 outputs estimated motion data indicating the movement of the second part. For example, the output unit 134 outputs the estimated motion data by transmitting the estimated motion data to the information terminal 2 used by the user U via the device communication unit 11. Alternatively, the output unit 134 may output the estimated motion data to a printer so that it can be printed. Note that, when the estimation unit 133 acquires the state estimation data from the estimation model, the output unit 134 outputs the estimated motion data and the state estimation data.

[0080] In this way, the output unit 134 outputs estimated motion data indicating the movement of the second part, so that the user U can grasp the movement of the second part where the device D is not attached. In other words, the user U can basically grasp the movement of his / her entire body by attaching one device D to the first part. This eliminates the need for the user U to attach sensors and devices to all parts of the body where the user U wants to know the movement, as in the past, thereby reducing the economic, physical, and time burden on the user U.

[0081] As described above, since the movement of the cervical vertebrae in the neck is relatively highly linked to the movement of other bones in a human body, the reliability of the output estimated motion data is increased by using the neck as the first part. In addition, if the first part on which the device D is worn is the neck, the device D can be worn while wearing clothes, making it easy to wear the device D.

[0082] [Configuration and operation of information terminal 2] The following describes the configuration and operation of the information terminal 2. Fig. 8 is a diagram showing an example of the configuration of the information terminal 2. As shown in Fig. 8, the information terminal 2 includes an operation unit 21, a terminal communication unit 22, a display unit 23, a storage unit 24, and a control unit 25.

[0083] The operation unit 21 is a device that receives operations by the user U, and is, for example, a touch panel provided on the display unit 23. The terminal communication unit 22 is a communication interface for communicating with the information processing device 1 and the device D via a communication network such as the Internet. The terminal communication unit 22 receives estimated motion data indicating the movement of the second body part.

[0084] The display unit 23 is configured, for example, with a liquid crystal display or an organic EL (Electro-Luminescence) display, etc. The display unit 23 displays various information according to the control of the display processing unit 251. The display unit 23 displays, for example, a screen for performing various settings related to the device D, a screen for registering information related to the user U, and a screen showing estimated motion data indicating the movement of the second part.

[0085] The storage unit 24 is a storage medium including a ROM, a RAM, etc. The storage unit 24 stores a program executed by the control unit 25. For example, the storage unit 24 stores a program that causes the control unit 25 to function as a display processing unit 251 and a setting receiving unit 252.

[0086] The control unit 25 is, for example, a CPU. The control unit 25 executes a program stored in the storage unit 24 to function as a display processing unit 251 and a setting receiving unit 252.

[0087] The display processing unit 251 displays various types of information on the display unit 23. The display processing unit 251 displays a screen for making various settings related to the device D and a screen for registering information related to the user U on the display unit 23. In addition, the display processing unit 251 receives estimated motion data indicating the movement of the second part from the output unit 134 of the information processing device 1 via the terminal communication unit 22, and then displays the received estimated motion data on the display unit 23.

[0088] The setting reception unit 252 receives settings related to the device D attached to the first part of the user U. The setting reception unit 252 receives various settings related to the device D and an operation for registering information about the user U from the user U, for example, via an operation by the user U from the operation unit 21. The setting reception unit 252 transmits a command to the device D for reflecting the various received settings related to the device D in the device D, via the terminal communication unit 22. The setting reception unit 252 transmits the received information about the user U to the information processing device 1, via the terminal communication unit 22.

[0089] [Processing flow in information processing device 1] 9 is a flowchart showing the flow of processing in the information processing device 1. Hereinafter, the flow of processing in the information processing device 1 will be described with reference to FIG.

[0090] The receiving unit 131 receives measured motion data indicating a movement of a first part of the user U from the device D worn by the user U (S1). Next, the receiving unit 131 determines whether or not a type of movement has been input to the information terminal 2 by the user U (S2). If a type of movement has not been input (S2: NO), the identification unit 132 identifies the type of movement indicated by the received measured motion data (S3). On the other hand, if a type of movement has been input (S2: YES), the process skips S3 and proceeds to S4 below.

[0091] The estimation unit 133 determines whether or not a machine learning model that uses multiple sets of the first motion data and the second motion data as teacher data is used to acquire the estimated motion data (S4). When the estimation unit 133 determines that the machine learning model is used (S4: YES), the machine learning model outputs the most likely motion data as the motion data of the second part corresponding to the measured motion data input to the model. The estimation unit 133 acquires the output motion data as estimated motion data indicating the movement of the second part (S5).

[0092] On the other hand, if the estimation unit 133 determines that the machine learning model is not used (S4: NO), the estimation model having a plurality of sets of first motion data and second motion data identifies first motion data having a similarity to the input actual measured motion data that is equal to or higher than a predetermined threshold, and outputs second motion data associated with the identified first motion data. The estimation unit 133 acquires the output motion data as estimated motion data indicating the movement of the second body part (S6).

[0093] The output unit 134 outputs the estimated motion data indicating the movement of the second body part by transmitting the estimated motion data to the information terminal 2 used by the user U (S7).

[0094] [Effects of information processing device 1] As described above, the information processing device 1 according to this embodiment can estimate the movement of a second body part (e.g., a knee) different from the first body part based on the movement of the first body part (e.g., a neck) acquired by the device D attached to the first body part of the user U. This eliminates the need for the user U to attach sensors and devices to all body parts whose movements the user U wishes to grasp, as in the conventional case, thereby reducing the economic, physical, and time burden on the user U.

[0095] Furthermore, the information processing device 1 according to the present embodiment can estimate the physical condition of the user U (risk of injury and degree of physical fatigue) in addition to the movement of the second part. This allows the user U to improve the amount and manner of exercise according to his / her physical condition. As a result, the user U can perform healthy and effective exercise for a long period of time.

[0096] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, all or part of the device can be configured by distributing or integrating functionally or physically in any unit. In addition, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effect of the new embodiment resulting from the combination combines the effect of the original embodiment. [Explanation of symbols]

[0097] 1. Information processing device 11 Device communication section 12 Storage section 13 Control section 131 Receiving unit 132 Identification Section 133 Estimation Department 134 Output section 2. Information terminal 21 Control section 22 Terminal communication unit 23 Display section 24 Memory section 25 Control Unit 251 Display Processing Unit 252 Settings Reception Section U User D Device S Information Processing System

Claims

1. a storage unit that stores an estimation model for each type of movement, the estimation model being created based on first motion data indicating a movement of a first part of a body and second motion data indicating a movement of the second part, the first motion data being acquired at the same time by a first device attached to a first part of the body and a second device attached to a second part of the body different from the first part; and A receiving unit that receives actual measured motion data indicating a movement of the first part of a subject to be analyzed, the actual measured motion data being acquired by a device attached to the first part of the subject; an identification unit that identifies a type of the movement indicated by the received measured motion data; an estimation unit that inputs the measured motion data into the estimation model corresponding to the identified type of motion, and acquires estimated motion data indicating a motion of the second part of the subject from the estimation model; an output unit for outputting the estimated motion data; An information processing device having the above configuration.

2. a storage unit that stores an estimation model created based on first motion data indicating a movement of a first part of a body and second motion data indicating a movement of the second part, the first motion data being acquired at the same time by a first device attached to a first part of the body and a second device attached to a second part of the body different from the first part, the estimation model outputting motion data indicating the movement of the second part when the motion data indicating the movement of the first part is input; A receiving unit that receives actual measured motion data indicating a movement of the first part of a subject to be analyzed, the actual measured motion data being acquired by a device attached to the first part of the subject; an estimation unit that inputs the measured motion data into the estimation model and acquires estimated motion data indicating a movement of the second part of the subject from the estimation model; an output unit for outputting the estimated motion data; having the estimation model has a plurality of sets of the first motion data and the second motion data, identifies the first motion data having a similarity to the input actual measured motion data equal to or greater than a predetermined threshold, and outputs the second motion data associated with the identified first motion data. Information processing device.

3. a storage unit for storing an estimation model created based on first motion data indicating a movement of a first part of a body and second motion data indicating a movement of the second part, which are acquired at the same time by a first device attached to a first part of the body and a second device attached to a second part of the body different from the first part, and state data indicating a state of the body when the movement indicated by the first motion data and the movement indicated by the second motion data are performed, the estimation model outputting, when the motion data indicating the movement of the first part is input, the motion data indicating the movement of the second part and state estimation data which is a result of estimating the state of the body; A receiving unit that receives actual measured motion data indicating a movement of the first part of a subject to be analyzed, the actual measured motion data being acquired by a device attached to the first part of the subject; an estimation unit that inputs the measured motion data into the estimation model and acquires estimated motion data indicating a movement of the second part of the subject and the state estimation data from the estimation model; an output unit that outputs the estimated motion data and the state estimation data; An information processing device having the above configuration.

4. The first site is a site on a body, and the second site is one or more sites on a body. The information processing device according to claim 1 .

5. The first part is a neck, and the second part is a part of the body other than the neck. The information processing device according to claim 4.

6. a storage unit that stores an estimation model created based on first motion data indicating a movement of a first part of a body, second motion data indicating a movement of the second part, and third motion data indicating a movement of the third part, which are acquired at the same time by a first device attached to a first part of the body, a second device attached to a second part of the body different from the first part, and a third device attached to a third part different from the first part and the second part, the estimation model outputting motion data indicating the movement of the second part when the motion data indicating the movement of the first part and the motion data indicating the movement of the third part are input; A receiving unit that receives first actual motion data indicating the movement of the first part of the subject that is the subject's target for analysis, acquired by a device worn on the first part of the subject, and third actual motion data indicating the movement of the third part of the subject that is acquired by a device worn on the third part of the subject; an estimation unit that inputs the first measured motion data and the third measured motion data to the estimation model and acquires estimated motion data indicating a movement of the second part of the subject from the estimation model; an output unit for outputting the estimated motion data; having the estimation unit acquires, as the estimated motion data indicating the movement of the second part of the subject, motion data obtained by weighting-averaging estimated motion data based on the first actual motion data and estimated motion data based on the third actual motion data, so that actual motion data of a part closer to the second part, out of the first actual motion data and the third actual motion data, is reflected at a higher rate in the estimated motion data indicating the movement of the second part of the subject; Information processing device.

7. The estimation model is created based on first motion data indicating a movement of a first part of a body and second motion data indicating a movement of a second part, the first motion data being acquired at the same time by a first device attached to a first part of the body and a second device attached to a second part of the body different from the first part, and the estimation model is executed by a computer having a storage unit that stores, for each type of movement, the estimation model that outputs motion data indicating the movement of the second part when the motion data indicating the movement of the first part is input. receiving actual motion data indicating a movement of the first part of the subject, the actual motion data being acquired by a device attached to the first part of the subject to be analyzed; an identification step of identifying a type of the movement indicated by the received measured motion data; an estimation step of inputting the measured motion data into the estimation model corresponding to the identified type of motion, and acquiring estimated motion data indicating a motion of the second part of the subject from the estimation model; an output step of outputting the estimated motion data; An information processing method comprising the steps of:

8. An estimation model is created based on first motion data indicating a movement of a first part of a body and second motion data indicating a movement of a second part, the first motion data being acquired at the same time by a first device attached to a first part of the body and a second device attached to a second part of the body different from the first part, the estimation model being executed by a computer having a storage unit that stores the estimation model and outputs motion data indicating the movement of the second part when the motion data indicating the movement of the first part is input. receiving actual motion data indicating a movement of the first part of the subject, the actual motion data being acquired by a device attached to the first part of the subject to be analyzed; an estimation step of inputting the measured motion data into the estimation model and acquiring estimated motion data indicating a movement of the second part of the subject from the estimation model; an output step of outputting the estimated motion data; having the estimation model has a plurality of sets of the first motion data and the second motion data, identifies the first motion data having a similarity to the input actual measured motion data equal to or greater than a predetermined threshold, and outputs the second motion data associated with the identified first motion data. Information processing methods.

9. an estimation model created based on first motion data indicating a movement of a first part of a body and second motion data indicating a movement of a second part, the first motion data being acquired at the same time by a first device attached to a first part of the body and a second device attached to a second part of the body different from the first part, and state data indicating a state of the body when the movement indicated by the first motion data and the movement indicated by the second motion data are performed, the estimation model being created based on first motion data indicating a movement of a first part of a body and second motion data indicating a movement of a second part, the first motion data being acquired at the same time by a first device attached to a first part of the body and a second motion data indicating a movement of the second part, the estimation model being created based on state data indicating a state of the body when the movement indicated by the first motion data and the movement indicated by the second motion data are performed, the estimation model being executed by a computer having a memory unit that stores the estimation model, receiving actual motion data indicating a movement of the first part of the subject, the actual motion data being acquired by a device attached to the first part of the subject to be analyzed; an estimation step of inputting the measured motion data into the estimation model, and acquiring estimated motion data indicating a movement of the second part of the subject and the state estimation data from the estimation model; an output step of outputting the estimated motion data and the state estimation data; An information processing method comprising the steps of:

10. An information processing device and an information terminal are provided, The information processing device includes: a storage unit that stores an estimation model for each type of movement, the estimation model being created based on first motion data indicating a movement of a first part of a body and second motion data indicating a movement of the second part, the first motion data being acquired at the same time by a first device attached to a first part of the body and a second device attached to a second part of the body different from the first part; and A receiving unit that receives actual measured motion data indicating a movement of the first part of a subject to be analyzed, the actual measured motion data being acquired by a device attached to the first part of the subject; an identification unit that identifies a type of the movement indicated by the received measured motion data; an estimation unit that inputs the measured motion data into the estimation model corresponding to the identified type of motion, and acquires estimated motion data indicating a motion of the second part of the subject from the estimation model; an output unit for outputting the estimated motion data; having The information terminal includes: a setting reception unit that receives settings related to a device attached to the first body part of the subject; a terminal communication unit for receiving the estimated motion data; An information processing system having the above configuration.

11. An information processing device and an information terminal are provided, The information processing device includes: a storage unit that stores an estimation model created based on first motion data indicating a movement of a first part of a body and second motion data indicating a movement of the second part, the first motion data being acquired at the same time by a first device attached to a first part of the body and a second device attached to a second part of the body different from the first part, the estimation model outputting motion data indicating the movement of the second part when the motion data indicating the movement of the first part is input; A receiving unit that receives actual measured motion data indicating a movement of the first part of a subject to be analyzed, the actual measured motion data being acquired by a device attached to the first part of the subject; an estimation unit that inputs the measured motion data into the estimation model and acquires estimated motion data indicating a movement of the second part of the subject from the estimation model; an output unit for outputting the estimated motion data; having the estimation model has a plurality of sets of the first motion data and the second motion data, identifies the first motion data having a similarity to the input actual measured motion data equal to or greater than a predetermined threshold, and outputs the second motion data associated with the identified first motion data; The information terminal includes: a setting reception unit that receives settings related to a device attached to the first body part of the subject; a terminal communication unit for receiving the estimated motion data; An information processing system having the above configuration.

12. An information processing device and an information terminal are provided, The information processing device includes: a storage unit for storing an estimation model created based on first motion data indicating a movement of a first part of a body and second motion data indicating a movement of the second part, which are acquired at the same time by a first device attached to a first part of the body and a second device attached to a second part of the body different from the first part, and state data indicating a state of the body when the movement indicated by the first motion data and the movement indicated by the second motion data are performed, the estimation model outputting, when the motion data indicating the movement of the first part is input, the motion data indicating the movement of the second part and state estimation data which is a result of estimating the state of the body; A receiving unit that receives actual measured motion data indicating a movement of the first part of a subject to be analyzed, the actual measured motion data being acquired by a device attached to the first part of the subject; an estimation unit that inputs the measured motion data into the estimation model and acquires estimated motion data indicating a movement of the second part of the subject and the state estimation data from the estimation model; an output unit that outputs the estimated motion data and the state estimation data; having The information terminal includes: a setting reception unit that receives settings related to a device attached to the first body part of the subject; a terminal communication unit for receiving the estimated motion data; An information processing system having the above configuration.