Information processing device, information processing method, and information processing system

Abstract

This information processing device (1) includes: a storage unit (12) that stores an estimation model created on the basis of first motion data and second motion data that are acquired at the same timing by a first device attached to a first portion of a body and a second device attached to a second portion different from the first portion of the body, the first motion data indicating movement of the first portion and the second motion data indicating movement of the second portion, and the estimation model outputting the motion data that indicates the movement of the second portion when the motion data that indicates the movement of the first portion is inputted; a reception unit (131) that receives actual measurement motion data that indicates the movement of the first portion of a subject to be analyzed and that is acquired by the device attached to the first portion of the subject; an estimation unit (133) that inputs the actual measurement motion data to the estimation model and acquires, from the estimation model, estimated motion data that indicates the movement of the second portion of the subject; and an output unit (134) that outputs the estimated motion data.

Description

Information processing device, information processing method, and information processing system

[0001] The present invention relates to an information processing device, an information processing method, and an information processing system that analyze motion data acquired by a device worn on the body.

[0002] Conventionally, techniques for analyzing the motion state of a human body have been known. Patent Document 1 discloses a technique that can estimate an index related to the motion state of a human body based on various data collected in a time series.

[0003] JP 2016-010562 A

[0004] In recent years, people have become highly interested in understanding their own health and exercise status. To understand one's own health and exercise status, there is a method of analyzing body movements by attaching sensors or devices to the human body, as described in the above-mentioned Patent Document 1. However, conventional techniques have a problem in that sensors or devices must be attached to all parts of the body whose movements one wishes to understand.

[0005] Therefore, the present invention has been made in consideration of these points, and aims to reduce the burden on users when they wear sensors or devices on their bodies to analyze their own body movements.

[0006] An information processing device according to a first aspect of the present invention is an estimation model created based on first motion data indicating the movement of a first part of the body and second motion data indicating the movement of the second part, which are acquired at the same time by a first device worn on a first part of the body and a second device worn on a second part of the body different from the first part, and which includes: a memory unit that stores the estimation model, which outputs motion data indicating the movement of the second part when motion data indicating the movement of the first part is input; a receiving unit that receives measured motion data indicating the movement of the first part of a subject to be analyzed, acquired by a device worn on the first part of the subject; an estimation unit that inputs the measured motion data into the estimation model and acquires estimated motion data indicating the movement of the second part of the subject from the estimation model; and an output unit that outputs the estimated motion data.

[0007] The memory unit may store the estimation model for each type of movement, the receiving unit may receive the type of movement, and the estimation unit may input the measured motion data into the estimation model corresponding to the received type of movement and obtain the estimated motion data from the estimation model.

[0008] The memory unit may store the estimation model for each type of movement, and the information processing device may further have an identification unit that identifies the type of movement indicated by the received measured motion data, and the estimation unit may input the measured motion data into the estimation model corresponding to the identified type of movement and obtain the estimated motion data from the estimation model.

[0009] The estimation model may have multiple sets of the first motion data and the second motion data, identify the first motion data that has a similarity to the input measured motion data that is greater than or equal to a predetermined threshold, and output the second motion data associated with the identified first motion data.

[0010] The estimation model is a machine learning model that is machine-learned using multiple sets of the first motion data and the second motion data as training data, and may output the most probable motion data as the motion data of the second part that corresponds to the input measured motion data.

[0011] The estimation model may be created based on the first motion data, the second motion data, and state data indicating the state of the body when the movement indicated by the first motion data and the movement indicated by the second motion data are performed, and when motion data indicating the movement of the first part is input, it may output motion data indicating the movement of the second part and state estimation data that is the result of estimating the state of the body, and the estimation unit may input the measured motion data to the estimation model and obtain the estimated motion data and the state estimation data from the estimation model, and the output unit may output the estimated motion data and the state estimation data.

[0012] The first location may be one location on the body, and the second location may be one or more locations on the body.

[0013] The first part may be the neck, and the second part may be a part of the body other than the neck.

[0014] The estimation model may be created based on the first motion data, the second motion data, and third motion data indicating the movement of the third part, which are acquired at the same time by the first device, the second device, and a third device worn on a third part different from the first part and the second part, and may be a model that outputs motion data indicating the movement of the second part when the motion data indicating the movement of the first part and the motion data indicating the movement of the third part are input, and the receiving unit may receive first measured motion data indicating the movement of the first part of the subject, which is acquired by a device worn on the first part of the subject, and third measured motion data indicating the movement of the third part of the subject, which is acquired by a device worn on the third part of the subject, and the estimation unit may input the first measured motion data and the third measured motion data to the estimation model, and acquire the estimated motion data indicating the movement of the second part of the subject from the estimation model.

[0015] The estimation unit may obtain, as the estimated motion data indicating the movement of the second part of the subject, motion data that is a weighted average of estimated motion data based on the first measured motion data and estimated motion data based on the third measured motion data, so that the measured motion data of a part of the first measured motion data and the third measured motion data that is closer to the second part is reflected to a higher extent in the estimated motion data indicating the movement of the second part of the subject.

[0016] An information processing method according to a second aspect of the present invention is an estimation model created based on first motion data indicating the movement of a first part of the body and second motion data indicating the movement of the second part, which are acquired at the same time by a first device worn on a first part of the body and a second device worn on a second part of the body different from the first part, and the estimation model is executed by a computer having a memory unit that stores the estimation model, which outputs motion data indicating the movement of the second part when motion data indicating the movement of the first part is input, and includes the following steps: receiving measured motion data indicating the movement of the first part of the subject, acquired by a device worn on the first part of the subject; an estimation step that inputs the measured motion data into the estimation model and acquires estimated motion data indicating the movement of the second part of the subject from the estimation model; and an output step that outputs the estimated motion data.

[0017] An information processing system according to a third aspect of the present invention includes an information processing device and an information terminal. The information processing device has an estimation model created based on first motion data indicating the movement of a first part of the body and second motion data indicating the movement of the second part, which are acquired at the same time by a first device worn on a first part of the body and a second device worn on a second part different from the first part of the body, the estimation model having a memory unit that stores the estimation model and outputs motion data indicating the movement of the second part when motion data indicating the movement of the first part is input, a receiving unit that receives measured motion data indicating the movement of the first part of the subject to be analyzed, acquired by a device worn on the first part of the subject, an estimation unit that inputs the measured motion data into the estimation model and acquires estimated motion data indicating the movement of the second part of the subject from the estimation model, and an output unit that outputs the estimated motion data. The information terminal has a setting receiving unit that receives settings related to the device worn on the first part of the subject, and a terminal communication unit that receives the estimated motion data.

[0018] According to the present invention, when a user wears a sensor or device on his or her body to analyze his or her body movements, the burden on the user can be reduced.

[0019] 1 is a diagram showing a configuration of an information processing system S. FIG. 2 is a diagram showing a configuration of an information processing device 1. FIG. 3 is a diagram showing an example of a motion dataset. FIG. 4 is a diagram showing an example of machine learning data. FIG. 5 is a diagram showing an example of measured motion data showing neck movement. FIG. 6 is a diagram showing an example of estimated motion data showing a knee rotation angle estimated using an estimation model from measured motion data showing neck movement. FIG. 7 is a diagram showing an example of measured motion data showing the extension speed of a right forearm when a user is serving in tennis. FIG. 8 is a diagram showing an example of estimated motion data showing the extension speed of a right forearm estimated using an estimation model from measured motion data showing neck movement when a user is serving in tennis. FIG. 9 is a diagram showing an example of measured motion data showing the internal rotation speed of a right forearm when a user is batting a baseball. FIG. 10 is a diagram showing an example of estimated motion data showing the internal rotation speed of a right forearm estimated using an estimation model from measured motion data showing neck movement when a user is batting a baseball. FIG. 11 is a diagram showing a configuration of an information terminal 2. FIG. 12 is a flowchart showing the flow of processing executed by an information processing device 1.

[0020] [Outline of Information Processing System S] An outline of the information processing system S according to this embodiment will be described using Fig. 1. Fig. 1 is a diagram showing an outline of the operation of the information processing system S. The information processing system S includes an information processing device 1, an information terminal 2, and a device D. The information processing system S may also include other equipment such as a server and a terminal.

[0021] The information processing system S is a system for estimating the movement of one or more second body parts (e.g., knees) different from the first body part (e.g., neck) of a subject to be analyzed, based on the movement of the first body part acquired by a device D attached to the first body part (e.g., neck). The first body part may be, for example, a position of the skin behind the cervical vertebrae.

[0022] The information processing device 1 is a computer such as a server that performs this estimation. The information processing device 1, the information terminal 2, and the device D are capable of communicating with each other. The information processing device 1 is assumed to be installed in a location (e.g., on the cloud) different from the location where the user U uses the information terminal 2, but may also be installed in the location where the user U uses the information terminal 2.

[0023] The information terminal 2 is a terminal used by a user U who is exercising. The user U uses the information terminal 2 to set up the device D and to obtain estimation results from the information processing device 1. The information terminal 2 is preferably a portable terminal such as a smartphone or a tablet personal computer, but may also be a stationary terminal such as a desktop personal computer.

[0024] The device D is a sensor that can detect the movement of a body part, and is, for example, an acceleration sensor, an angular velocity sensor, or a sensor that combines both sensors. The device D may be a wearable terminal that can be attached to a user U who is exercising. As an example, one surface of the device D has an adhesive member attached to it to make it easier to adhere to the skin of the user U. The device D may be attached directly to the skin of the user U, or may be attached over clothing that fits tightly to the user U. The device D may not have an adhesive member and may have a metal part. In this case, the user U can wear the device D in close contact with the user U, for example, by inserting the device D inside the sportswear at the collar position on the back of the neck of the user U and fixing the device D with a magnet from the outside of the sportswear.

[0025] The acceleration sensor measures the rate of change (acceleration) in the movement speed of the user U while he / she is exercising. The acceleration sensor detects, for example, acceleration components (acceleration signals) along each of three mutually orthogonal axial directions and outputs the detected acceleration components as acceleration data. The angular velocity sensor measures changes in the movement direction (angular velocity) of the user U while he / she is exercising. The angular velocity sensor detects, for example, angular velocity components (angular velocity signals) occurring in the rotational direction of rotational movement along each of three mutually orthogonal axes and outputs the detected angular velocity data.

[0026] In recent years, people have become increasingly interested in understanding their own health and physical activity. One method for understanding one's own health and physical activity is to attach sensors and devices to the human body to analyze bodily movements, but conventional technology has the problem of requiring sensors and devices to be attached to every part of the body whose movements one wishes to understand.

[0027] Therefore, the information processing system S inputs measured motion data indicating the movement of a first part of the subject's body (e.g., the neck) to be analyzed, acquired by a device D attached to the first part, into an estimation model, and acquires estimated motion data indicating the movement of one or more second parts (e.g., knees) different from the first part of the subject's body from the estimation model. In other words, the information processing system S can estimate the movement of a certain part based on the measured movement of another part.

[0028] The estimation model needs to be created in advance before the estimation. The estimation model is a model created based on first motion data indicating the movement of a first body part and second motion data indicating the movement of a second body part, which are acquired at the same time by a first device attached to a first body part and a second device attached to a second body part. Below, an overview of the operation of the processing system S will be described with reference to FIG. 1.

[0029] First, before exercising, user U uses information terminal 2 to set up linking (pairing) device D with information terminal 2, set the part of the body where device D will be worn, register his or her own profile, set goals, etc.

[0030] When the user U starts exercising, the device D continuously acquires measured motion data indicating the movement of a first part of the body (e.g., the neck) to which the device D is attached. The device D transmits the acquired measured motion data to the information processing device 1.

[0031] The information processing device 1 inputs the received measured motion data indicating the movement of a first body part into an estimation model, and thereby obtains estimated motion data indicating the movement of a second body part (e.g., a knee) different from the first body part from the estimation model. The information processing device 1 uses at least one of the following two types of estimation models:

[0032] The first estimation model is a program or data for outputting estimated motion data by referencing a motion data set having multiple sets of first motion data indicating the movement of a first body part and second motion data indicating the movement of a second body part. This estimation model identifies first motion data that has a similarity to the input measured motion data that is equal to or greater than a predetermined threshold, and outputs the second motion data associated with the identified first motion data as estimated motion data.

[0033] The second estimation model is a machine learning model that uses multiple sets of first motion data indicating the movement of a first body part and second motion data indicating the movement of a second body part as training data, and outputs, as estimated motion data, motion data that is most likely to be the motion data of the second body part corresponding to the input measured motion data.

[0034] The information processing device 1 then transmits the acquired estimated motion data indicating the movement of the second body part to the information terminal 2. The information terminal 2 displays the received estimated motion data. By referring to the estimated motion data displayed on the information terminal 2, the user U can understand how the second body part (e.g., the knee) to which the device D is not attached was moving, and can use this information to plan future exercise, rest, etc. The information terminal 2 may also display the user U's physical condition (risk of injury, degree of physical fatigue, etc.).

[0035] In this way, the information processing system S can estimate the movements of one or more second body parts (e.g., knees) different from the first body part (e.g., neck) based on the movement of the first body part acquired by the device D attached to the first body part of the user U. This eliminates the need for the user U to attach sensors or devices to all body parts whose movements the user U wishes to grasp, as was the case in the past. As a result, the financial, physical, and time burden on the user U is reduced.

[0036] In particular, the inventors have found that using the cervical vertebrae as the first part is effective because, in many sports movements, the movement of the cervical vertebrae in the neck is highly linked to the movement of other bones in the human body. As an example, the correlation between the movement of the neck and the movement of the knee is higher than the correlation between the movement of the right arm and the movement of the knee. Therefore, by using the neck as the first part, the accuracy of estimating the movement of the second part different from the neck is improved. The configurations and operations of the information processing device 1 and the information terminal 2 are described in detail below.

[0037] [Configuration and Operation of Information Processing Device 1] The following describes the configuration and operation of the information processing device 1. Fig. 2 is a diagram showing an example of the configuration of the information processing device 1. As shown in Fig. 2, the information processing device 1 includes a device communication unit 11, a storage unit 12, and a control unit 13.

[0038] The device communication unit 11 is a communication interface for communicating with the information terminal 2 and the device D via a communication network such as the Internet.

[0039] The storage unit 12 is a storage medium including a ROM (Read Only Memory) and a RAM (Random Access Memory), etc. The storage unit 12 stores a program executed by the control unit 13. For example, the storage unit 12 stores an information processing program that causes the control unit 13 to function as a receiving unit 131, an identifying unit 132, an estimating unit 133, and an output unit 134. The storage unit 12 stores an estimation model.

[0040] The estimation model is a model created based on first motion data indicating the movement of a first body part and second motion data indicating the movement of a second body part, which are acquired at the same time by a first device attached to a first body part and a second device attached to a second body part different from the first body part. When the motion data indicating the movement of the first body part is input, the estimation model outputs motion data indicating the movement of the second body part.

[0041] The first part is, for example, one part of the body, and the second part is, for example, one or more parts of the body. Specifically, the first part is, for example, the neck, and the second part is, for example, a part of the body other than the neck (e.g., an arm or a knee). In other words, when motion data indicating neck movement is input, the estimation model can output motion data indicating whole-body movement.

[0042] The first device and the second device are sensors capable of detecting the movement of body parts, such as an acceleration sensor, an angular velocity sensor, or a sensor that combines both sensors. The first device and the second device may be conventional devices different from device D, or may be device D. The "same timing" may be exactly the same timing, or may strictly be timing that falls within a different predetermined range (for example, within a few milliseconds).

[0043] The motion data is data that indicates the movement of body parts, such as data that shows a two-dimensional graph in which the horizontal axis represents the number of frames and the vertical axis represents angular velocity (units: radians per second (rad / s) or degrees per second (deg / s)).

[0044] The estimation model may be a model created based on first motion data, second motion data, and condition data indicating the physical state when the movement indicated by the first motion data and the movement indicated by the second motion data are performed. The physical state may be, for example, the risk of injury, the degree of physical fatigue, etc. The learning condition data used to create the model is, for example, data indicating at least one of the results of observing the physical state of a person from whom the first motion data and second motion data for learning were obtained while wearing the first device and the second device, the results of measuring the physical state of the person, or the results of asking the person about painful parts of their body.

[0045] The storage unit 12 may store an estimation model for each type of movement. The type of movement may be, for example, a type of sport (jogging, baseball, tennis, etc.) or a type of movement within the same sport (overthrow and underthrow in the case of baseball pitching, etc.). There are two types of estimation models:

[0046] The first estimation model is a model having multiple sets of first motion data indicating the movement of a first body part and second motion data indicating the movement of a second body part. Hereinafter, this model may be referred to as a motion data set. Fig. 3 is a diagram showing an example of a motion data set. In the motion data set, the first motion data, the second motion data, and state data indicating the body state are associated with each type of movement.

[0047] The second estimation model is a machine learning model that is trained using multiple sets of first motion data indicating the movement of a first body part and second motion data indicating the movement of a second body part as training data. FIG. 4 is a diagram showing an example of machine learning data. In the machine learning data, the type of movement is associated with the trained machine learning model. Note that machine learning can be performed using a known method. Examples of known methods include convolutional neural networks (CNN), logistic regression, support vector machines (SVM), decision trees, and random forests.

[0048] The control unit 13 is, for example, a CPU (Central Processing Unit). The control unit 13 executes an information processing program stored in the storage unit 12, thereby functioning as a receiving unit 131, an identifying unit 132, an estimating unit 133, and an output unit 134.

[0049] The receiving unit 131 receives measured motion data indicating the movement of the first part of the user U, who is a subject to be analyzed, acquired by the device D attached to the first part of the user U. The receiving unit 131 receives the measured motion data indicating the movement of the first part of the user U, for example, from the device D via the device communication unit 11.

[0050] The receiving unit 131 may receive the measured motion data from the device D immediately after the device D acquires the measured motion data. This allows the user U to refer to the estimated motion data indicating the movement of the second body part in real time while exercising, and improve the movement of the body, for example.

[0051] The receiving unit 131 may receive the type of movement. For example, the receiving unit 131 may receive the type of movement input by the user U to the information terminal 2 from the information terminal 2 via the device communication unit 11. By having the user U input the type of movement directly in this way, an incorrect type of movement will not be used when estimating estimated motion data, thereby improving the accuracy of estimating estimated motion data.

[0052] However, some users U may want to avoid the trouble of inputting the type of movement, or may be unable to input the type of movement because they are currently exercising. Therefore, the identification unit 132 identifies the type of movement indicated by the received measured motion data indicating the movement of the first part of the user U.

[0053] The identification unit 132, for example, compares the received measured motion data indicating the movement of the first part of the user U with typical motion data indicating the movement of the first part for each type of movement, and identifies the type of movement with the highest similarity. For example, if the measured motion data indicating the movement of the user U's neck is more similar to typical motion data indicating the movement of the neck when a person is playing tennis than to typical motion data indicating the movement of the neck when a person is jogging or playing baseball, the identification unit 132 identifies the type of movement as "tennis." By having the identification unit 132 identify the type of movement in this way, the user U does not need to input the type of movement, thereby reducing the burden on the user U.

[0054] The estimation unit 133 inputs the received measured motion data into the estimation model and acquires, from the estimation model, estimated motion data indicating the movement of the second part of the user U. The estimation unit 133 may acquire the estimated motion data without considering the type of movement, but it is preferable to acquire the estimated motion data while considering the type of movement, as will be described in the next paragraph.

[0055] The estimation unit 133 may input the measured motion data to an estimation model corresponding to the type of motion received by the receiving unit 131 or identified by the identification unit 132, and acquire estimated motion data from the estimation model. This allows the estimation unit 133 to acquire more appropriate estimated motion data using a more appropriate estimation model. The output of estimated motion data by the estimation model will be described below.

[0056] When the estimation model is a motion data set, the estimation model identifies first motion data that has a similarity to the input measured motion data that is equal to or greater than a predetermined threshold. For example, the estimation model of the motion data set shown in FIG. 3 identifies first motion data that corresponds to a type of movement received by the receiving unit 131 or identified by the identifying unit 132 and that satisfies a criterion that the similarity to the input measured motion data is X% or greater. The value of X, which is the predetermined threshold, is determined based on the probability that one of the multiple motion data is more similar than other motion data, and is, for example, 80%. The estimation model may identify the first motion data that is most similar to the input measured motion data.

[0057] The estimation model then outputs the second motion data associated with the identified first motion data. For example, the estimation model for the motion data set shown in Figure 3 outputs the second motion data stored in the same row as the identified first motion data.

[0058] When the estimation model is a machine learning model, the estimation model outputs the most probable motion data as the motion data of the second body part corresponding to the input measured motion data. The machine learning model used in this case is, for example, a machine learning model corresponding to the type of movement received by the receiving unit 131 or identified by the identifying unit 132 in the machine learning data shown in FIG.

[0059] In either case where the estimation model is a motion data set or a machine learning model, the estimation model may output ideal motion data indicating ideal motion of the second body part together with estimated motion data indicating the movement of the second body part. This allows the user U to visually grasp the discrepancy between the estimated motion data and the ideal motion data, which can be used to improve the movement of the body. Note that the estimation model may display the graph of the estimated motion data and the graph of the ideal motion data superimposed (synchronized), or displayed vertically or horizontally.

[0060] The estimation model may calculate a score based on values ​​indicated by the estimated motion data. Furthermore, the estimation model may output analysis results such as physical condition (risk of injury, level of physical fatigue, etc.), exercise status (speed, pace, balance, pitch slide, power, speed, stamina, aggressiveness, stability, insufficient movements, etc.), and suggestions regarding rest and exercise (care methods, rest methods, practice content, strengthening exercises, etc.) based on values ​​indicated by the estimated motion data or the calculated score. The estimation model may quantify these analysis results and output them as a graph or radar chart. By visualizing the analysis results in this way, the user U can easily and quickly understand their own exercise deficiencies and areas they need to pay attention to.

[0061] The estimation model may quantify the analysis results by referring to publicly known information. For example, the estimation model may reference data correlating a value indicated by the motion data or a calculated score with a value related to the analysis results, and output a value related to the analysis results associated with the value indicated by the estimated motion data or the calculated score. As an example, the estimation model may output a score range (e.g., XX points or more) calculated based on the angular velocity indicated by the estimated motion data, correlated with a value indicating the level of injury risk (e.g., XX%). Alternatively, the estimation model may output a value related to the analysis results by inputting the value indicated by the estimated motion data or the calculated score into a machine learning model that has been trained using the value indicated by the motion data or the calculated score and the value related to the analysis results as training data.

[0062] Here, specific examples of measured motion data and estimated motion data will be described. FIG. 5A is a diagram showing an example of measured motion data indicating neck movement. In FIG. 5A, the horizontal axis represents the number of frames, and the vertical axis represents angular velocity (rad / s). If the three mutually orthogonal directions are the x-, y-, and z-directions, the graphs marked with "x" in FIG. 5A correspond to angular velocity in the x-direction, the graphs marked with "y" correspond to angular velocity in the y-direction, and the graphs marked with "z" correspond to angular velocity in the z-direction. FIG. 5B is a diagram showing an example of estimated motion data indicating the rotation angle of the knee estimated using an estimation model from the measured motion data indicating neck movement. In FIG. 5B, the horizontal axis represents the number of frames, and the vertical axis represents angular velocity (deg / s). Since FIG. 5 shows the results measured at 100 Hz, each frame takes 1 / 100 seconds, resulting in a total measurement time of approximately 6 seconds ((1 / 100 seconds) × approximately 600 frames).

[0063] Next, the results of verifying the reliability of the estimation model according to this embodiment will be described. Fig. 6A is a diagram showing an example of measured motion data indicating the extension speed of the right forearm when a user is serving in tennis. Fig. 6B is a diagram showing an example of estimated motion data indicating the extension speed of the right forearm estimated using an estimation model from measured motion data indicating the movement of the neck when a user is serving in tennis. Fig. 7A is a diagram showing an example of measured motion data indicating the internal rotation speed of the right forearm when a user is batting a baseball. Fig. 7B is a diagram showing an example of estimated motion data indicating the internal rotation speed of the right forearm estimated using an estimation model from measured motion data indicating the movement of the neck when a user is batting a baseball. In these four figures, the horizontal axis represents the number of frames, and the vertical axis represents angular velocity (rad / s). 6 and 7 are the results of measurements taken at 60 Hz, so that each frame takes 1 / 60 seconds, and the total measurement time is approximately 3 seconds ((1 / 60 seconds) x approximately 180 frames) for FIG. 6 and approximately 0.67 seconds ((1 / 60 seconds) x approximately 40 frames) for FIG. 7.

[0064] The values ​​and shapes of the graphs in Figures 6A and 6B were similar, and the values ​​and shapes of the graphs in Figures 7A and 7B were also similar. That is, the graph of estimated motion data (graph showing estimated values) showing arm movement estimated using the estimation model from measured motion data showing neck movement was similar in value and shape to the graph of measured motion data (graph showing measured values) showing arm movement acquired at the same time as the measured motion data showing neck movement was acquired. This confirmed that the estimation model according to this embodiment is highly reliable.

[0065] The above explanation has been given of the process when the estimation model outputs estimated motion data, but as mentioned above, the estimation model may also be created based on first motion data, second motion data, and state data indicating the body state when the movements indicated by the first motion data and the movements indicated by the second motion data are performed. In this way, when state data is used to create the estimation model, the estimation unit 133 inputs measured motion data indicating the movement of the first part into the estimation model, and obtains estimated motion data and state estimation data, which are the results of estimating the body state, from the estimation model. The output of estimated motion data and state estimation data performed by the estimation model will be described below.

[0066] When motion data indicating the movement of a first body part is input, the estimation model outputs motion data indicating the movement of a second body part and state estimation data resulting from estimating the body state. For example, when the estimation model is a motion data set, the estimation model first identifies first motion data having a similarity to the input measured motion data that is equal to or greater than a predetermined threshold using the above-described method. Next, the estimation model outputs second motion data associated with the identified first motion data as estimated motion data, and also outputs state data associated with the identified first motion data as state estimation data.

[0067] Also, for example, when the estimation model is a machine learning model, the estimation model outputs the most probable motion data as the motion data of the second part corresponding to the input measured motion data as estimated motion data, and outputs the most probable state data as the body state corresponding to the input measured motion data as state estimation data.

[0068] In this way, the estimation unit 133 estimates the physical condition of the user U in addition to the estimated motion data, so that the user U can improve the amount and manner of exercise according to his or her physical condition. As a result, the user U can exercise in a healthy and effective manner over a long period of time.

[0069] As explained above, in this embodiment, the user U basically only needs to wear one device D on a first body part. However, if more accurate estimated motion data is desired, the user U may wear a device D on a third body part different from the first and second body parts. The following describes the process when the user wears the device D on multiple body parts.

[0070] In this case, the estimation model used by the estimation unit 133 is a model created based on first motion data, second motion data, and third motion data indicating the movement of the third part, which are acquired at the same time by a first device attached to a first part of the body, a second device attached to a second part different from the first part of the body, and a third device attached to a third part different from the first and second parts. In this case, in the motion data set, the first motion data, second motion data, third motion data, and status data indicating the state of the body are associated with each type of movement.

[0071] Furthermore, when the estimation unit 133 uses a machine learning model, one model created based on the first motion data, the second motion data, and the third motion data is used as the machine learning model. Alternatively, two machine learning models may be used: a first model created based on the first motion data and the second motion data, and a third model created based on the second motion data and the third motion data.

[0072] The receiving unit 131 receives first measured motion data indicating the movement of a first part of the user U, acquired by a device worn on a first part of the user U, and third measured motion data indicating the movement of a third part of the user U, acquired by a device worn on a third part of the user U.

[0073] The estimation unit 133 inputs the first measured motion data and the third measured motion data into the estimation model, and obtains, from the estimation model, estimated motion data indicating the movement of the second part of the user U. The output of estimated motion data performed by the estimation model will be described below.

[0074] When motion data indicating the movement of a first body part and motion data indicating the movement of a third body part are input, the estimation model outputs motion data indicating the movement of a second body part. If the estimation model is a motion data set, the estimation model identifies first motion data having a similarity to the input first measured motion data equal to or greater than a predetermined threshold and third motion data having a similarity to the input third measured motion data equal to or greater than a predetermined threshold. Next, the estimation model identifies second motion data associated with the identified first motion data and second motion data associated with the identified third motion data as estimated motion data.

[0075] When the estimation model is a single machine learning model, the estimation model identifies, as estimated motion data, motion data of the second body part that is most likely to correspond to the first measured motion data and the third measured motion data input to the estimation model. In this case, the estimation unit 133 inputs the first measured motion data and the third measured motion data to the machine learning model, and determines, as estimated motion data, the motion data indicating the movement of the second body part that is output from the machine learning model.

[0076] When the estimation model is two machine learning models, the estimation model identifies estimated motion data based on the most probable motion data of the second part corresponding to the first measured motion data input to the first model, and the most probable motion data of the second part corresponding to the third measured motion data input to the third model.

[0077] When the estimation model is a motion data set or when the estimation model is two machine learning models, the estimation model adjusts the first measured motion data or the third measured motion data so that the measured motion data of a part closer to the second part is reflected at a higher rate in the estimated motion data indicating the movement of the second part of the user U. Specifically, the estimation model outputs, for example, motion data that is a weighted average of first estimated motion data based on the first measured motion data and third estimated motion data based on the third measured motion data, as the second estimated motion data of the user U. For example, the estimation model may output, as the second estimated motion data, a value obtained by multiplying a value indicated by the first estimated motion data by the distance from the first part to the second part and a value obtained by multiplying a value indicated by the third estimated motion data by the distance from the third part to the second part, and then dividing the sum of the distance from the first part to the second part and the distance from the third part to the second part.

[0078] In this way, the estimation model outputs estimated motion data based on multiple pieces of actual motion data acquired by devices D worn at multiple locations. For example, even if the value of estimated motion data based on a certain location deviates slightly from the actual measurement value, the estimated motion data can be corrected using the value of estimated motion data based on another location. As a result, the reliability of the estimated motion data that is finally output is increased. Furthermore, if the values ​​of estimated motion data acquired based on the multiple pieces of actual motion data acquired by devices D worn at multiple locations are weighted according to the distance from the device D to the second part, the reliability of the estimated motion data that is finally output is further increased.

[0079] The output unit 134 outputs estimated motion data indicating the movement of the second body part. The output unit 134 outputs the estimated motion data, for example, by transmitting the estimated motion data to the information terminal 2 used by the user U via the device communication unit 11. Alternatively, the output unit 134 may output the estimated motion data to a printer so that it can be printed. Note that, when the estimation unit 133 acquires the state estimation data from the estimation model, the output unit 134 outputs the estimated motion data and the state estimation data.

[0080] In this way, the output unit 134 outputs estimated motion data indicating the movement of the second body part, allowing the user U to grasp the movement of the second body part where the device D is not attached. In other words, the user U can basically grasp the movement of his or her entire body by attaching one device D to the first body part. This eliminates the need for the user U to attach sensors or devices to all body parts whose movements the user U wants to know, as was the case in the past, thereby reducing the financial, physical, and time burden on the user U.

[0081] As described above, because the movement of the cervical vertebrae in the neck is relatively closely linked to the movement of other bones in a human body, using the neck as the first part increases the reliability of the output estimated motion data. Furthermore, if the first part on which device D is worn is the neck, device D can be worn while wearing clothing, making it easy to wear device D.

[0082] [Configuration and Operation of Information Terminal 2] The following describes the configuration and operation of the information terminal 2. Fig. 8 is a diagram showing an example of the configuration of the information terminal 2. As shown in Fig. 8, the information terminal 2 includes an operation unit 21, a terminal communication unit 22, a display unit 23, a storage unit 24, and a control unit 25.

[0083] The operation unit 21 is a device that receives operations from the user U, and is, for example, a touch panel provided on the display unit 23. The device communication unit 22 is a communication interface for communicating with the information processing device 1 and the device D via a communication network such as the Internet. The device communication unit 22 receives estimated motion data that indicates the movement of the second body part.

[0084] The display unit 23 is configured by, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, etc. The display unit 23 displays various information according to the control of the display processing unit 251. The display unit 23 displays, for example, a screen for making various settings related to the device D, a screen for registering information related to the user U, and a screen showing estimated motion data indicating the movement of the second region.

[0085] The storage unit 24 is a storage medium including a ROM, a RAM, etc. The storage unit 24 stores a program executed by the control unit 25. For example, the storage unit 24 stores a program that causes the control unit 25 to function as a display processing unit 251 and a setting receiving unit 252.

[0086] The control unit 25 is, for example, a CPU. The control unit 25 executes the programs stored in the storage unit 24 to function as a display processing unit 251 and a setting receiving unit 252.

[0087] The display processing unit 251 displays various types of information on the display unit 23. The display processing unit 251 displays, on the display unit 23, a screen for making various settings related to the device D and a screen for registering information related to the user U. Furthermore, the display processing unit 251 receives estimated motion data indicating the movement of the second part from the output unit 134 of the information processing device 1 via the device communication unit 22, and then displays the received estimated motion data on the display unit 23.

[0088] The setting acceptance unit 252 accepts settings related to the device D attached to the first part of the user U. The setting acceptance unit 252 accepts various settings related to the device D and operations for registering information about the user U from the user U, for example, via an operation by the user U on the operation unit 21. The setting acceptance unit 252 transmits to the device D, via the device communication unit 22, a command for reflecting the various accepted settings related to the device D in the device D. The setting acceptance unit 252 transmits the accepted information about the user U to the information processing device 1 via the device communication unit 22.

[0089] [Processing Flow in Information Processing Device 1] Fig. 9 is a flowchart showing the processing flow in the information processing device 1. The processing flow in the information processing device 1 will be described below with reference to Fig. 9 .

[0090] The receiving unit 131 receives measured motion data indicating the movement of a first part of the user U from the device D worn by the user U (S1). Next, the receiving unit 131 determines whether the type of movement has been input to the information terminal 2 by the user U (S2). If the type of movement has not been input (S2: NO), the identifying unit 132 identifies the type of movement indicated by the received measured motion data (S3). On the other hand, if the type of movement has been input (S2: YES), the process skips S3 and proceeds to S4 below.

[0091] The estimation unit 133 determines whether a machine learning model that uses multiple sets of first motion data and second motion data as training data will be used to acquire the estimated motion data (S4). If the estimation unit 133 determines that a machine learning model will be used (S4: YES), the machine learning model outputs the most likely motion data as the motion data of the second body part that corresponds to the measured motion data input to the model. The estimation unit 133 acquires the output motion data as estimated motion data that indicates the movement of the second body part (S5).

[0092] On the other hand, if the estimation unit 133 determines that a machine learning model is not to be used (S4: NO), the estimation model, which has multiple sets of first motion data and second motion data, identifies first motion data that has a similarity to the input measured motion data that is equal to or greater than a predetermined threshold, and outputs second motion data associated with the identified first motion data. The estimation unit 133 acquires the output motion data as estimated motion data that indicates the movement of the second body part (S6).

[0093] The output unit 134 outputs the estimated motion data indicating the movement of the second body part by transmitting the estimated motion data to the information terminal 2 used by the user U (S7).

[0094] [Effects of Information Processing Device 1] As described above, the information processing device 1 according to the present embodiment can estimate the movement of a second body part (e.g., knee) different from the first body part (e.g., neck) based on the movement of the first body part acquired by the device D attached to the first body part of the user U. This eliminates the need for the user U to attach sensors or devices to all body parts of the user U whose movements it is desired to grasp, as in the past, thereby reducing the financial, physical, and time burden on the user U.

[0095] Furthermore, the information processing device 1 according to the present embodiment can estimate the physical condition of the user U (risk of injury and level of physical fatigue) in addition to the movement of the second body part. This allows the user U to improve the amount and method of exercise according to their own physical condition. As a result, the user U can exercise in a healthy and effective manner over a long period of time.

[0096] The present invention has been described above using embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, all or part of the device can be configured by functionally or physically distributing or integrating in any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments resulting from the combination also have the effects of the original embodiments.

[0097] REFERENCE SIGNS LIST 1 Information processing device 11 Device communication unit 12 Storage unit 13 Control unit 131 Receiving unit 132 Identification unit 133 Estimation unit 134 Output unit 2 Information terminal 21 Operation unit 22 Terminal communication unit 23 Display unit 24 Storage unit 25 Control unit 251 Display processing unit 252 Setting reception unit U User D Device S Information processing system

Claims

1. A storage unit that stores an estimation model created based on first motion data indicating the movement of a first part of the body and second motion data indicating the movement of a second part of the body, which are acquired at the same timing by a first device worn on the first part of the body and a second device worn on a second part of the body different from the first part of the body in the body, and that outputs motion data indicating the movement of the second part of the body when motion data indicating the movement of the first part of the body is input; a receiving unit that receives measured motion data indicating the movement of the first part of the body, which is acquired by a device worn on the first part of the body of a subject to be analyzed; an estimating unit that inputs the measured motion data into the estimation model and acquires estimated motion data indicating the movement of the second part of the body from the estimation model; and an output unit that outputs the estimated motion data.

2. The information processing apparatus according to claim 1, wherein the storage unit stores the estimation model for each type of movement, the receiving unit receives the type of movement, and the estimating unit inputs the measured motion data into the estimation model corresponding to the received type of movement and acquires the estimated motion data from the estimation model.

3. The information processing apparatus according to claim 1, further comprising an identifying unit that identifies the type of movement indicated by the received measured motion data, wherein the storage unit stores the estimation model for each type of movement, and the estimating unit inputs the measured motion data into the estimation model corresponding to the identified type of movement and acquires the estimated motion data from the estimation model.

4. The information processing apparatus according to claim 1, wherein the estimation model has a plurality of sets of the first motion data and the second motion data, identifies the first motion data having a similarity equal to or higher than a predetermined threshold with the input measured motion data, and outputs the second motion data associated with the identified first motion data.

5. The estimation model is a machine learning model obtained by machine learning using a plurality of sets of the first motion data and the second motion data as teacher data, and outputs the most probable motion data as the motion data of the second part corresponding to the input measured motion data. The information processing apparatus according to claim 1.

6. The estimation model is created based on the first motion data, the second motion data, and state data indicating the state of the body when the motion indicated by the first motion data and the motion indicated by the second motion data are performed. When motion data indicating the motion of the first part is input, it is a model that outputs state estimation data, which is the result of estimating the motion data indicating the motion of the second part and the state of the body. The estimation unit inputs the measured motion data into the estimation model and obtains the estimated motion data and the state estimation data from the estimation model. The output unit outputs the estimated motion data and the state estimation data. The information processing apparatus according to claim 1.

7. The first part is one part of the body, and the second part is one or more parts of the body. The information processing apparatus according to claim 1.

8. The first part is the neck, and the second part is a part of the body other than the neck. The information processing apparatus according to claim 7.

9. The model for estimation is created based on the first motion data, the second motion data, and the third motion data indicating the movement of the third part, which are acquired at the same timing by the first device, the second device, and a third device attached to a third part different from the first part and the second part. When the motion data indicating the movement of the first part and the motion data indicating the movement of the third part are input, the model outputs the motion data indicating the movement of the second part. The receiving unit receives the first measured motion data indicating the movement of the first part of the subject, which is acquired by the device attached to the first part of the subject, and the third measured motion data indicating the movement of the third part of the subject, which is acquired by the device attached to the third part of the subject. The estimating unit inputs the first measured motion data and the third measured motion data into the model for estimation, and acquires the estimated motion data indicating the movement of the second part of the subject from the model for estimation. The information processing apparatus according to claim 1.

10. The estimating unit weights and averages the estimated motion data based on the first measured motion data and the estimated motion data based on the third measured motion data so that the measured motion data of the part closer to the second part among the first measured motion data and the third measured motion data is reflected at a higher ratio in the estimated motion data indicating the movement of the second part of the subject, and acquires the weighted average motion data as the estimated motion data indicating the movement of the second part of the subject. The information processing apparatus according to claim 9.

11. A computer having a storage unit that stores an estimation model created based on first motion data indicating the movement of a first part of the body and second motion data indicating the movement of a second part of the body, which are acquired at the same timing by a first device worn on the first part of the body and a second device worn on a second part of the body different from the first part of the body in the body, and that outputs motion data indicating the movement of the second part of the body when motion data indicating the movement of the first part of the body is input. A receiving step of receiving measured motion data indicating the movement of the first part of the subject, which is acquired by a device worn on the first part of the subject to be analyzed; An estimating step of inputting the measured motion data into the estimation model and acquiring estimated motion data indicating the movement of the second part of the subject from the estimation model; An output step of outputting the estimated motion data; An information processing method having.

12. An information processing system including an information processing device and an information terminal, The information processing device includes A storage unit that stores an estimation model created based on first motion data indicating the movement of a first part of the body and second motion data indicating the movement of a second part of the body, which are acquired at the same timing by a first device worn on the first part of the body and a second device worn on a second part of the body different from the first part of the body in the body, and that outputs motion data indicating the movement of the second part of the body when motion data indicating the movement of the first part of the body is input; A receiving unit that receives measured motion data indicating the movement of the first part of the subject, which is acquired by a device worn on the first part of the subject to be analyzed; An estimating unit that inputs the measured motion data into the estimation model and acquires estimated motion data indicating the movement of the second part of the subject from the estimation model; An output unit that outputs the estimated motion data; The information terminal includes A setting reception unit that receives settings related to a device worn on the first part of the subject; A terminal communication unit that receives the estimated motion data; An information processing system having.

Images