Information processing device, information processing method, and information processing system

Abstract

To reduce a load on a user in analyzing a motion of his / her own body by mounting a sensor and a device on the body.SOLUTION: An information processing device 1 includes: a storage section 12 for storing an estimation model which is generated based on first motion data indicating a motion of a first part and second motion data indicating a motion of a second part acquired at the same timing by a first device mounted on the first part of a body and a second device mounted on the second part different from the first part of the body and also which outputs the motion data indicating the motion of the second part when the motion data indicating the motion of the first part is input; a reception section 131 for receiving actual measurement motion data which is acquired by the device mounted on the first part of an object person and indicates the motion of the first part of the object person; an estimation section 133 for inputting the actual measurement motion data to the estimation model so as to acquire, from the estimation model, estimation motion data indicating the motion of the second part of the object person; and an output section 134 for outputting the estimation motion data.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and an information processing system that analyze motion data acquired by a device worn on the body. [Background technology]

[0002] Conventionally, techniques for analyzing the motion state of a human body have been known. Patent Document 1 discloses a technique that can estimate an index related to the motion state of a human body based on various data collected in a time series. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-010562 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, people have become highly interested in understanding their own health and exercise status. To understand one's own health and exercise status, there is a method of analyzing body movements by attaching sensors and devices to the human body, as described in the above-mentioned Patent Document 1. However, conventional techniques have a problem in that sensors and devices must be attached to all parts of the body whose movements one wishes to understand.

[0005] Therefore, the present invention has been made in consideration of these points, and aims to reduce the burden on users when they wear sensors or devices on their bodies to analyze their own body movements. [Means for solving the problem]

[0006] An information processing device according to a first aspect of the present invention is an estimation model created based on first motion data indicating the movement of a first part of the body and second motion data indicating the movement of the second part, which are acquired at the same time by a first device worn on a first part of the body and a second device worn on a second part of the body different from the first part, and which has the following features: a memory unit that stores the estimation model, which outputs motion data indicating the movement of the second part when motion data indicating the movement of the first part is input; a receiving unit that receives measured motion data indicating the movement of the first part of a subject to be analyzed, acquired by a device worn on the first part of the subject; an estimation unit that inputs the measured motion data into the estimation model and acquires estimated motion data indicating the movement of the second part of the subject from the estimation model; and an output unit that outputs the estimated motion data.

[0007] The memory unit may store the estimation model for each type of movement, the receiving unit may receive the type of movement, and the estimation unit may input the measured motion data into the estimation model corresponding to the received type of movement and obtain the estimated motion data from the estimation model.

[0008] The memory unit may store the estimation model for each type of movement, and the information processing device may further have an identification unit that identifies the type of movement indicated by the received measured motion data, and the estimation unit may input the measured motion data into the estimation model corresponding to the identified type of movement and obtain the estimated motion data from the estimation model.

[0009] The estimation model may have multiple sets of the first motion data and the second motion data, identify the first motion data that has a similarity to the input measured motion data that is greater than or equal to a predetermined threshold, and output the second motion data associated with the identified first motion data.

[0010] The estimation model is a machine learning model obtained by machine learning using a plurality of sets of the first motion data and the second motion data as teacher data, and may output the most probable motion data as the motion data of the second part corresponding to the input measured motion data.

[0011] The estimation model is created based on the first motion data, the second motion data, and state data indicating the state of the body when the motion indicated by the first motion data and the motion indicated by the second motion data are performed. When motion data indicating the motion of the first part is input, it may be a model that outputs state estimation data, which is the result of estimating the motion data indicating the motion of the second part and the state of the body. The estimation unit may input the measured motion data into the estimation model and obtain the estimated motion data and the state estimation data from the estimation model. The output unit may output the estimated motion data and the state estimation data.

[0012] The first part may be one part of the body, and the second part may be one or more parts of the body.

[0013] The first part may be the neck, and the second part may be a part of the body other than the neck.

[0014] The estimation model is created based on the first motion data, the second motion data, and the third motion data indicating the movement of the third part, which are acquired at the same timing by the first device, the second device, and a third device attached to a third part different from the first part and the second part. The model may output motion data indicating the movement of the second part when motion data indicating the movement of the first part and motion data indicating the movement of the third part are input. The receiving unit may receive first measured motion data indicating the movement of the first part of the subject, which is acquired by a device attached to the first part of the subject, and third measured motion data indicating the movement of the third part of the subject, which is acquired by a device attached to the third part of the subject. The estimating unit may input the first measured motion data and the third measured motion data into the estimation model and obtain the estimated motion data indicating the movement of the second part of the subject from the estimation model.

[0015] The estimating unit may obtain, as the estimated motion data indicating the movement of the second part of the subject, motion data obtained by weighted averaging the estimated motion data based on the first measured motion data and the estimated motion data based on the third measured motion data, such that the measured motion data of the part closer to the second part among the first measured motion data and the third measured motion data is reflected at a higher ratio in the estimated motion data indicating the movement of the second part of the subject.

[0016] The information processing method according to the second aspect of the present invention is an estimation model created based on first motion data indicating the movement of a first part of the body and second motion data indicating the movement of a second part of the body, which are acquired by a first device worn on the first part of the body and a second device worn on a second part of the body different from the first part of the body at the same timing. When motion data indicating the movement of the first part is input, a computer having a storage unit that stores an estimation model for outputting motion data indicating the movement of the second part executes a reception step of receiving measured motion data indicating the movement of the first part of the subject acquired by the device worn on the first part of the subject to be analyzed, an estimation step of inputting the measured motion data into the estimation model and acquiring estimated motion data indicating the movement of the second part of the subject from the estimation model, and an output step of outputting the estimated motion data.

[0017] The information processing system according to the third aspect of the present invention includes an information processing device and an information terminal. The information processing device includes a storage unit that stores an estimation model created based on first motion data indicating the movement of a first part of the body and second motion data indicating the movement of a second part of the body, which are acquired by a first device worn on the first part of the body and a second device worn on a second part of the body different from the first part of the body at the same timing. When motion data indicating the movement of the first part is input, the estimation model outputs motion data indicating the movement of the second part, a reception unit that receives measured motion data indicating the movement of the first part of the subject acquired by the device worn on the first part of the subject to be analyzed, an estimation unit that inputs the measured motion data into the estimation model and acquires estimated motion data indicating the movement of the second part of the subject from the estimation model, and an output unit that outputs the estimated motion data. The information terminal includes a setting reception unit that receives settings related to the device worn on the first part of the subject and a terminal communication unit that receives the estimated motion data.

Advantages of the Invention

[0018] According to the present invention, when a user wears a sensor or device on his or her body to analyze his or her body movements, the burden on the user can be reduced. [Brief explanation of the drawings]

[0019]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5A

Figure 5B

Figure 6A

Figure 6B

Figure 7A

Figure 7B

Figure 8

Figure 9

[0020] [Outline of Information Processing System S] An overview of an information processing system S according to this embodiment will be described using Fig. 1. Fig. 1 is a diagram showing an overview of the operation of the information processing system S. The information processing system S includes an information processing device 1, an information terminal 2, and a device D. The information processing system S may also include other equipment such as a server and a terminal.

[0021] The information processing system S is a system for estimating the movement of one or more second body parts (e.g., knees) different from the first body part (e.g., neck) of a subject to be analyzed, based on the movement of the first body part acquired by a device D attached to the first body part (e.g., neck). The first body part is, for example, a position of the skin behind the cervical vertebrae.

[0022] The information processing device 1 is a computer such as a server that performs this estimation. The information processing device 1, the information terminal 2, and the device D are capable of communicating with each other. It is assumed that the information processing device 1 is installed in a location (for example, on the cloud) different from the location where the user U uses the information terminal 2, but it may also be installed in the location where the user U uses the information terminal 2.

[0023] The information terminal 2 is a terminal used by a user U who is exercising. The user U uses the information terminal 2 to set up the device D and to obtain estimation results from the information processing device 1. The information terminal 2 is preferably a portable terminal such as a smartphone or a tablet personal computer, but may also be a stationary terminal such as a desktop personal computer.

[0024] Device D is a sensor capable of detecting the movement of a body part, such as an acceleration sensor, an angular velocity sensor, or a sensor that combines both sensors. Device D may be a wearable terminal that can be worn by the exercising user U. As an example, one surface of Device D is provided with an adhesive member to facilitate close contact with the skin of user U. Device D may be directly attached to the skin of user U, or may be attached from above the clothing that is in close contact with user U. Even if Device D is not provided with an adhesive member, Device D may have a metal part. In this case, user U can attach Device D in a state of being in close contact with user U, for example, by inserting Device D inside the sports clothing at the position of the collar edge on the back side of the neck in the sports clothing and fixing Device D with a magnet from the outside of the sports clothing.

[0025] The acceleration sensor measures the rate of change (acceleration) of the operating speed during the movement of user U. The acceleration sensor, for example, detects acceleration components (acceleration signals) along each of three mutually orthogonal axes and outputs them as acceleration data. The angular velocity sensor measures the change in the operating direction (angular velocity) during the movement of user U. The angular velocity sensor, for example, detects angular velocity components (angular velocity signals) generated in the rotational direction of the rotational movement along each of three mutually orthogonal axes and outputs them as angular velocity data.

[0026] In recent years, people have shown a high level of interest in understanding their own health and exercise conditions. To understand their own health and exercise conditions, there is a method of analyzing body movements by attaching sensors and devices to the human body. However, in the conventional technology, there has been a problem that it is necessary to attach sensors and devices to all parts where the movement is to be grasped.

[0027] Therefore, the information processing system S inputs measured motion data indicating the movement of a first part of the subject's body (e.g., the neck) to be analyzed, acquired by a device D attached to the first part, into an estimation model, and acquires estimated motion data indicating the movement of one or more second parts (e.g., knees) different from the first part of the subject's body from the estimation model. In other words, the information processing system S can estimate the movement of a certain part based on the measured movement of another part.

[0028] The estimation model needs to be created in advance before the estimation. The estimation model is a model created based on first motion data indicating the movement of a first body part and second motion data indicating the movement of a second body part, which are acquired at the same time by a first device worn on a first body part and a second device worn on a second body part. Below, an overview of the operation of the processing system S will be described with reference to Figure 1.

[0029] First, before exercising, user U uses information terminal 2 to set up linking (pairing) device D with information terminal 2, set the part of the body where device D will be worn, register his / her own profile, set goals, etc.

[0030] When the user U starts exercising, the device D constantly acquires measured motion data indicating the movement of a first part of the body (for example, the neck) to which the device D is attached. The device D transmits the acquired measured motion data to the information processing device 1.

[0031] The information processing device 1 inputs the received measured motion data indicating the movement of a first body part into an estimation model, and thereby obtains estimated motion data indicating the movement of a second body part (e.g., a knee) different from the first body part from the estimation model. The information processing device 1 uses at least one of the following two types of estimation models:

[0032] The first estimation model is a program or data for outputting estimated motion data by referring to a motion data set having a plurality of sets of first motion data indicating the movement of the first part and second motion data indicating the movement of the second part. This estimation model identifies first motion data having a similarity equal to or higher than a predetermined threshold with the input measured motion data, and outputs the second motion data associated with the identified first motion data as the estimated motion data.

[0033] The second estimation model is a machine learning model that has been machine-learned using a plurality of sets of first motion data indicating the movement of the first part and second motion data indicating the movement of the second part as teacher data. This estimation model outputs, as the estimated motion data, the most probable motion data as the motion data of the second part corresponding to the input measured motion data.

[0034] Then, the information processing apparatus 1 transmits the estimated motion data indicating the movement of the second part of the acquired body to the information terminal 2. The information terminal 2 displays the received estimated motion data. By referring to the estimated motion data displayed on the information terminal 2, the user U can grasp how the second part (for example, the knee) not wearing the device D has moved, and can utilize it for future exercise, rest, etc. Note that the information terminal 2 may also display the state of the user U's body (such as the risk of injury and the degree of physical fatigue).

[0035] In this way, the information processing system S can estimate the movement of one or more second parts (for example, the knee) different from the first part in the body based on the movement of the first part acquired by the device D worn on the first part (for example, the neck) of the user U's body. As a result, the user U does not need to wear sensors or devices on all parts where movement is to be grasped as in the conventional case. As a result, the economic, physical, and time burdens on the user U are reduced.

[0036] In particular, the inventors have found that using the cervical vertebrae as the first part is effective because, in many sports movements, the movement of the cervical vertebrae in the neck is highly linked to the movement of other bones in the human body. For example, the correlation between the movement of the neck and the movement of the knee is higher than the correlation between the movement of the right arm and the movement of the knee. Therefore, by using the neck as the first part, the accuracy of estimating the movement of the second part different from the neck is improved. The configurations and operations of the information processing device 1 and the information terminal 2 will be described in detail below.

[0037] [Configuration and Operation of Information Processing Device 1] The following describes the configuration and operation of the information processing device 1. Fig. 2 is a diagram showing an example of the configuration of the information processing device 1. As shown in Fig. 2, the information processing device 1 includes a device communication unit 11, a storage unit 12, and a control unit 13.

[0038] The device communication unit 11 is a communication interface for communicating with the information terminal 2 and the device D via a communication network such as the Internet.

[0039] The storage unit 12 is a storage medium including a ROM (Read Only Memory) and a RAM (Random Access Memory), etc. The storage unit 12 stores a program executed by the control unit 13. For example, the storage unit 12 stores an information processing program that causes the control unit 13 to function as a receiving unit 131, an identifying unit 132, an estimating unit 133, and an output unit 134. The storage unit 12 stores an estimation model.

[0040] The estimation model is a model created based on first motion data indicating the movement of a first body part and second motion data indicating the movement of a second body part, which are acquired at the same time by a first device attached to a first body part and a second device attached to a second body part different from the first body part. When the motion data indicating the movement of the first body part is input, the estimation model outputs motion data indicating the movement of the second body part.

[0041] The first part is, for example, a single part on the body, and the second part is, for example, one or more parts on the body. Specifically, the first part is, for example, the neck, and the second part is, for example, parts on the body other than the neck (e.g., arms, knees). That is, when motion data indicating the movement of the neck is input, for example, the estimation model can output motion data indicating the movement of the whole body.

[0042] The first device and the second device are sensors that can detect the movement of body parts, such as, for example, an acceleration sensor, an angular velocity sensor, or a sensor that combines both sensors. The first device and the second device may be conventional devices different from device D, or may be device D. "The same timing" may be exactly the same timing, or may be timing within a predetermined range that is strictly different (e.g., within a few milliseconds).

[0043] Motion data is data indicating the movement of body parts. The motion data is, for example, data indicating a two-dimensional graph where the horizontal axis is the number of frames and the vertical axis is the angular velocity (unit: radians per second (rad / s) or degrees per second (deg / s)).

[0044] The estimation model may be a model created based on the first motion data, the second motion data, and state data indicating the state of the body when the movements indicated by the first motion data and the second motion data are performed. The state of the body is, for example, the risk of injury and the degree of body fatigue, etc. The state data for learning used to create the model is, for example, data indicating at least any one of the result of observing the state of the body, the result of measuring the state of the body, or the result of hearing from the person about the painful part of the body, targeting a person for whom the first motion data and the second motion data for learning are acquired by wearing the first device and the second device.

[0045] The storage unit 12 may store an estimation model for each type of movement. The type of movement may be, for example, a type of sport (jogging, baseball, tennis, etc.) or a type of movement within the same sport (overthrow and underthrow in the case of baseball pitching, etc.). There are two types of estimation models:

[0046] The first estimation model is a model having a plurality of sets of first motion data indicating the movement of a first body part and second motion data indicating the movement of a second body part. Hereinafter, this model may be referred to as a motion data set. Fig. 3 is a diagram showing an example of a motion data set. In the motion data set, the first motion data, the second motion data, and state data indicating the body state are associated with each type of movement.

[0047] The second estimation model is a machine learning model that is trained by machine learning using multiple sets of first motion data indicating the movement of a first body part and second motion data indicating the movement of a second body part as training data. FIG. 4 is a diagram showing an example of machine learning data. In the machine learning data, the type of movement is associated with the trained machine learning model. Note that machine learning can be performed using a known method. Examples of known methods include convolutional neural networks (CNNs), logistic regression, support vector machines (SVMs), decision trees, and random forests.

[0048] The control unit 13 is, for example, a CPU (Central Processing Unit). The control unit 13 executes an information processing program stored in the storage unit 12, thereby functioning as a receiving unit 131, an identifying unit 132, an estimating unit 133, and an output unit 134.

[0049] The receiving unit 131 receives measured motion data indicating the movement of the first part of the user U, which is obtained by the device D worn on the first part of the user U who is the subject of analysis. For example, the receiving unit 131 receives the measured motion data indicating the movement of the first part of the user U from the device D via the device communication unit 11.

[0050] The receiving unit 131 may receive the measured motion data from the device D immediately after the device D obtains the measured motion data. Thereby, the user U can refer to the estimated motion data indicating the movement of the second part in real time while exercising, and can improve the movement of the body and the like.

[0051] The receiving unit 131 may receive the type of movement. For example, the receiving unit 131 may receive the type of movement input by the user U to the information terminal 2 from the information terminal 2 via the device communication unit 11. In this way, since the user U directly inputs the type of movement, the incorrect type of movement is not used in the estimation of the estimated motion data, so the accuracy of the estimation of the estimated motion data is improved.

[0052] However, among the users U, there are users U who want to save the trouble of inputting the type of movement, or users U who cannot input the type of movement because they are exercising in the first place. Therefore, the identification unit 132 identifies the type of movement indicated by the measured motion data indicating the movement of the first part of the received user U.

[0053] The identification unit 132 compares, for example, the measured motion data indicating the movement of the first part of the received user U with the typical motion data indicating the movement of the first part for each type of movement, and identifies the type of movement with the highest similarity. For example, if the measured motion data indicating the movement of the neck of user U has a higher similarity with the typical motion data indicating the movement of the neck when a person is playing tennis than with the typical motion data indicating the movement of the neck when a person is jogging or playing baseball, the identification unit 132 identifies "tennis" as the type of movement. In this way, since the identification unit 132 identifies the type of movement, user U does not need to input the type of movement, thus reducing the burden on user U.

[0054] The estimation unit 133 inputs the received measured motion data into the estimation model and obtains the estimated motion data indicating the movement of the second part of user U from the estimation model. The estimation unit 133 may obtain the estimated motion data without considering the type of movement, but as will be described in the next paragraph, it is preferable to obtain the estimated motion data considering the type of movement.

[0055] The estimation unit 133 may input the measured motion data into the estimation model corresponding to the type of movement received by the reception unit 131 or identified by the identification unit 132, and obtain the estimated motion data from the estimation model. Thereby, the estimation unit 133 can obtain more appropriate estimated motion data using a more appropriate estimation model. Hereinafter, the output of the estimated motion data performed by the estimation model will be described.

[0056] When the estimation model is a motion data set, the estimation model identifies first motion data having a similarity equal to or greater than a predetermined threshold with the input measured motion data. For example, the estimation model of the motion data set shown in FIG. 3 corresponds to the type of motion received by the receiving unit 131 or identified by the identifying unit 132, and identifies first motion data satisfying the criterion that the similarity with the input measured motion data is X% or more. The value of X, which is the predetermined threshold, is determined based on, for example, the probability of being more similar than other motion data among a plurality of motion data, and is, for example, 80%. The estimation model may identify the first motion data most similar to the input measured motion data.

[0057] Subsequently, the estimation model outputs second motion data associated with the identified first motion data. For example, the estimation model of the motion data set shown in FIG. 3 outputs the second motion data stored in the same row as the identified first motion data.

[0058] When the estimation model is a machine learning model, the estimation model outputs the most probable motion data as the motion data of the second part corresponding to the input measured motion data. The machine learning model used at this time is, for example, the machine learning model corresponding to the type of motion received by the receiving unit 131 or identified by the identifying unit 132 in the machine learning data shown in FIG. 4.

[0059] In either case where the estimation model is a motion dataset or a machine learning model, the estimation model may output ideal motion data indicating the ideal movement of the second part together with the estimated motion data indicating the movement of the second part. Thereby, the user U can visually grasp the deviation between the estimated motion data and the ideal motion data and utilize it for improving the body movement. Note that the estimation model may display the graph of the estimated motion data and the graph of the ideal motion data by overlapping (synchronizing) them, or may display them vertically or horizontally.

[0060] The estimation model may calculate a score based on the value indicated by the estimated motion data. Further, the estimation model may output, as an analysis result, the state of the body (risk of injury, degree of body fatigue, etc.), the state of the motion (speed pace, balance, pitch slide, power, speed, stamina, positivity, sense of stability, lack of movement, etc.), and proposals regarding rest and exercise (care method, rest method, practice content, strengthening exercise, etc.) based on the value indicated by the estimated motion data or the calculated score. The estimation model may quantify these analysis results and output them as a graph or a radar chart. By visualizing the analysis results in this way, the user U can easily and quickly grasp what is lacking and what should be paid attention to regarding the exercise.

[0061] The estimation model may quantify the analysis results by referring to publicly known information. For example, the estimation model may reference data correlating a value indicated by the motion data or a calculated score with a value related to the analysis results, and output a value related to the analysis results associated with the value indicated by the estimated motion data or the calculated score. As an example, the estimation model may output a score range (e.g., XX points or more) calculated based on the angular velocity indicated by the estimated motion data, correlated with a value indicating the level of injury risk (e.g., XX%). Alternatively, the estimation model may output a value related to the analysis results by inputting the value indicated by the estimated motion data or the calculated score into a machine learning model that has been trained using the value indicated by the motion data or the calculated score and the value related to the analysis results as training data.

[0062] Here, specific examples of measured motion data and estimated motion data will be described. FIG. 5A is a diagram showing an example of measured motion data indicating neck movement. In FIG. 5A, the horizontal axis represents the number of frames, and the vertical axis represents angular velocity (rad / s). If the three mutually orthogonal directions are the x-, y-, and z-directions, the graphs marked with "x" in FIG. 5A correspond to angular velocity in the x-direction, the graphs marked with "y" correspond to angular velocity in the y-direction, and the graphs marked with "z" correspond to angular velocity in the z-direction. FIG. 5B is a diagram showing an example of estimated motion data indicating the rotation angle of the knee estimated using an estimation model from the measured motion data indicating neck movement. In FIG. 5B, the horizontal axis represents the number of frames, and the vertical axis represents angular velocity (deg / s). Since FIG. 5 shows the results of measurements at 100 Hz, each frame takes 1 / 100 seconds, and the total measurement time is approximately 6 seconds ((1 / 100 seconds) × approximately 600 frames).

[0063] Next, the results of verifying the reliability of the estimation model according to this embodiment will be described. FIG. 6A is a diagram showing an example of measured motion data indicating the extension speed of the right forearm when the user is serving in tennis. FIG. 6B is a diagram showing an example of estimated motion data indicating the extension speed of the right forearm estimated using the estimation model from the measured motion data indicating the movement of the neck when the user is serving in tennis. FIG. 7A is a diagram showing an example of measured motion data indicating the internal rotation speed of the right forearm when the user is batting in baseball. FIG. 7B is a diagram showing an example of estimated motion data indicating the internal rotation speed of the right forearm estimated using the estimation model from the measured motion data indicating the movement of the neck when the user is batting in baseball. In these four diagrams, the horizontal axis indicates the number of frames, and the vertical axis indicates the angular velocity (rad / s). Since FIGS. 6 and 7 are the results measured at 60 hz, each frame is 1 / 60 second, and the total measurement time is approximately 3 seconds ((1 / 60 second) × approximately 180 frames) for FIG. 6 and approximately 0.67 seconds ((1 / 60 second) × approximately 40 frames) for FIG. 7.

[0064] The values and shapes of the graphs in FIG. 6A and the graph in FIG. 6B were similar, and the values and shapes of the graphs in FIG. 7A and the graph in FIG. 7B were also similar. That is, the graph of the estimated motion data indicating the movement of the arm estimated using the estimation model from the measured motion data indicating the movement of the neck (the graph showing the estimated values) was similar in value and shape to the graph of the measured motion data indicating the movement of the arm obtained at the same timing as the acquisition of the measured motion data indicating the movement of the neck (the graph showing the measured values). From this, it was confirmed that the estimation model according to this embodiment is highly reliable.

[0065] The above explanation has been given of the process when the estimation model outputs estimated motion data, but as mentioned above, there are also cases where the estimation model is created based on first motion data, second motion data, and state data indicating the body state when the movement indicated by the first motion data and the movement indicated by the second motion data are performed. In this way, when state data is used to create the estimation model, the estimation unit 133 inputs measured motion data indicating the movement of the first part into the estimation model, and obtains estimated motion data and state estimation data, which are the results of estimating the body state, from the estimation model. The output of estimated motion data and state estimation data performed by the estimation model will be described below.

[0066] When motion data indicating the movement of a first body part is input, the estimation model outputs motion data indicating the movement of a second body part and state estimation data that is the result of estimating the body state. For example, when the estimation model is a motion data set, the estimation model first identifies first motion data that has a similarity to the input measured motion data that is equal to or greater than a predetermined threshold using the above-mentioned method. Next, the estimation model outputs second motion data associated with the identified first motion data as estimated motion data, and outputs state data associated with the identified first motion data as state estimation data.

[0067] Also, for example, when the estimation model is a machine learning model, the estimation model outputs the most probable motion data as the motion data of the second part corresponding to the input measured motion data as estimated motion data, and outputs the most probable state data as the body state corresponding to the input measured motion data as state estimation data.

[0068] In this way, by the estimation unit 133 estimating not only the estimated motion data but also the state of the user U's body, the user U can improve the amount and manner of exercise according to the state of their own body. As a result, the user U can perform healthy and effective exercise over a long period of time.

[0069] As described so far, in this embodiment, basically, the user U only needs to wear one device D on the first part. However, when it is desired to obtain more accurate estimated motion data, the user U may wear the device D on a third part different from the first part and the second part. Hereinafter, the processing when the device D is worn at a plurality of locations will be described.

[0070] The estimation model used by the estimation unit 133 in this case is a model created based on the first motion data, the second motion data, and the third motion data indicating the movement of the third part, which are acquired at the same timing by the first device worn on the first part of the body, the second device worn on the second part different from the first part on the body, and the third device worn on the third part different from the first part and the second part. In this case, in the motion data set, for each type of movement, the first motion data, the second motion data, the third motion data, and the state data indicating the state of the body are associated with each other.

[0071] Also, when the estimation unit 133 uses a machine learning model, as the machine learning model, one model created based on the first motion data, the second motion data, and the third motion data is used. Alternatively, as the machine learning model, two models, a first model created based on the first motion data and the second motion data, and a third model created based on the second motion data and the third motion data, may be used.

[0072] The receiving unit 131 receives first measured motion data indicating the movement of a first part of the user U, acquired by a device worn on a first part of the user U, and third measured motion data indicating the movement of a third part of the user U, acquired by a device worn on a third part of the user U.

[0073] The estimation unit 133 inputs the first measured motion data and the third measured motion data into the estimation model, and acquires, from the estimation model, estimated motion data indicating the movement of the second part of the user U. The output of estimated motion data performed by the estimation model will be described below.

[0074] When motion data indicating the movement of a first body part and motion data indicating the movement of a third body part are input, the estimation model outputs motion data indicating the movement of a second body part. If the estimation model is a motion data set, the estimation model identifies first motion data having a similarity to the input first measured motion data equal to or greater than a predetermined threshold and third motion data having a similarity to the input third measured motion data equal to or greater than a predetermined threshold. Next, the estimation model identifies second motion data associated with the identified first motion data and second motion data associated with the identified third motion data as estimated motion data.

[0075] When the estimation model is one machine learning model, the estimation model identifies, as estimated motion data, the most likely motion data of the second body part corresponding to the first measured motion data and the third measured motion data input to the estimation model. In this case, the estimation unit 133 inputs the first measured motion data and the third measured motion data to the machine learning model, and determines, as estimated motion data, the motion data indicating the movement of the second body part output from the machine learning model.

[0076] When the estimation model is two machine learning models, the estimation model identifies estimated motion data based on the most probable motion data of the second part corresponding to the first measured motion data input to the first model, and the most probable motion data of the second part corresponding to the third measured motion data input to the third model.

[0077] When the estimation model is a motion data set or when the estimation model is two machine learning models, the estimation model adjusts the first measured motion data or the third measured motion data so that the measured motion data of a body part closer to the second body part is reflected at a higher rate in the estimated motion data indicating the movement of the second body part of the user U. Specifically, the estimation model outputs, for example, motion data that is a weighted average of first estimated motion data based on the first measured motion data and third estimated motion data based on the third measured motion data as the second estimated motion data of the user U. For example, the estimation model may output, as the second estimated motion data, a value obtained by multiplying a value indicated by the first estimated motion data by the distance from the first body part to the second body part and a value obtained by multiplying a value indicated by the third estimated motion data by the distance from the third body part to the second body part, and dividing the sum of the distance from the first body part to the second body part and the distance from the third body part to the second body part.

[0078] In this way, the estimation model outputs estimated motion data based on multiple pieces of actual motion data acquired by devices D worn at multiple locations. For example, even if the value of estimated motion data based on one location deviates slightly from the actual measurement value, the estimated motion data is corrected using the value of estimated motion data based on another location. As a result, the reliability of the estimated motion data that is finally output is increased. Furthermore, if the estimated motion data values acquired based on the multiple pieces of actual motion data acquired by devices D worn at multiple locations are weighted according to the distance from device D to the second body part, the reliability of the estimated motion data that is finally output is further increased.

[0079] The output unit 134 outputs estimated motion data indicating the movement of the second body part. The output unit 134 outputs the estimated motion data, for example, by transmitting the estimated motion data to the information terminal 2 used by the user U via the device communication unit 11. Alternatively, the output unit 134 may output the estimated motion data to a printer so that it can be printed. Note that, when the estimation unit 133 acquires the state estimation data from the estimation model, the output unit 134 outputs the estimated motion data and the state estimation data.

[0080] In this way, the output unit 134 outputs estimated motion data indicating the movement of the second body part, so that the user U can grasp the movement of the second body part where the device D is not attached. In other words, the user U can basically grasp the movement of his or her entire body by attaching one device D to the first body part. This eliminates the need for the user U to attach sensors or devices to all body parts where the user U wants to know the movement, as in the past, thereby reducing the financial, physical, and time burden on the user U.

[0081] As described above, the movement of the cervical vertebrae in the neck is relatively closely linked to the movement of other bones in a human body, so using the neck as the first part increases the reliability of the output estimated motion data. Also, if the first part to which device D is attached is the neck, device D can be attached while wearing clothing, making it easy to wear device D.

[0082] [Configuration and operation of information terminal 2] The following describes the configuration and operation of the information terminal 2. Fig. 8 is a diagram showing an example of the configuration of the information terminal 2. As shown in Fig. 8, the information terminal 2 includes an operation unit 21, a terminal communication unit 22, a display unit 23, a storage unit 24, and a control unit 25.

[0083] The operation unit 21 is a device that receives operations from the user U, and is, for example, a touch panel provided on the display unit 23. The terminal communication unit 22 is a communication interface for communicating with the information processing apparatus 1 and the device D via a communication network such as the Internet. The terminal communication unit 22 receives estimated motion data indicating the movement of the second part.

[0084] The display unit 23 is constituted by, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 23 displays various kinds of information according to the control of the display processing unit 251. The display unit 23 displays, for example, a screen for making various settings regarding the device D, a screen for registering information regarding the user U, and a screen showing the estimated motion data indicating the movement of the second part.

[0085] The storage unit 24 is a storage medium including a ROM and a RAM. The storage unit 24 stores programs executed by the control unit 25. For example, the storage unit 24 stores a program that causes the control unit 25 to function as the display processing unit 251 and the setting reception unit 252.

[0086] The control unit 25 is, for example, a CPU. The control unit 25 functions as the display processing unit 251 and the setting reception unit 252 by executing the programs stored in the storage unit 24.

[0087] The display processing unit 251 causes the display unit 23 to display various kinds of information. The display processing unit 251 causes the display unit 23 to display a screen for making various settings regarding the device D and a screen for registering information regarding the user U. Further, the display processing unit 251 receives the estimated motion data indicating the movement of the second part from the output unit 134 of the information processing apparatus 1 via the terminal communication unit 22, and then causes the display unit 23 to display the received estimated motion data.

[0088] The setting reception unit 252 receives settings related to the device D worn on the first part of the user U. The setting reception unit 252 receives, for example, various settings related to the device D and operations for registering information related to the user U from the user U via operations by the user U from the operation unit 21. The setting reception unit 252 transmits, via the terminal communication unit 22, an instruction for reflecting the various received settings related to the device D to the device D. The setting reception unit 252 transmits the information related to the received user U to the information processing apparatus 1 via the terminal communication unit 22.

[0089] [Flow of processing in the information processing apparatus 1] FIG. 9 is a flowchart showing the flow of processing in the information processing apparatus 1. Hereinafter, with reference to FIG. 9, the flow of processing in the information processing apparatus 1 will be described.

[0090] The reception unit 131 receives, from the device D worn on the first part of the user U, measured motion data indicating the motion of the first part of the user U (S1). Subsequently, the reception unit 131 determines whether or not the type of motion has been input to the information terminal 2 by the user U (S2). If the type of motion has not been input (S2: NO), the identification unit 132 identifies the type of motion indicated by the received measured motion data (S3). On the other hand, if the type of motion has been input (S2: YES), the process skips S3 and proceeds to S4 below.

[0091] When acquiring the estimated motion data, the estimation unit 133 determines whether or not a machine learning model that has been machine-learned using a plurality of sets of first motion data and second motion data as teacher data is used (S4). When the estimation unit 133 determines that the machine learning model is used (S4: YES), the machine learning model outputs the most probable motion data as the motion data of the second part corresponding to the measured motion data input to the model. The estimation unit 133 acquires the output motion data as estimated motion data indicating the motion of the second part (S5).

[0092] On the other hand, if the estimation unit 133 determines that the machine learning model is not used (S4: NO), the estimation model, which has multiple sets of first motion data and second motion data, identifies first motion data that has a similarity to the input measured motion data that is equal to or greater than a predetermined threshold, and outputs second motion data associated with the identified first motion data. The estimation unit 133 acquires the output motion data as estimated motion data that indicates the movement of the second body part (S6).

[0093] The output unit 134 outputs the estimated motion data indicating the movement of the second body part by transmitting the estimated motion data to the information terminal 2 used by the user U (S7).

[0094] [Effects of information processing device 1] As described above, the information processing device 1 according to this embodiment can estimate the movement of a second body part (for example, a knee) different from the first body part (for example, a neck) based on the movement of the first body part acquired by the device D attached to the first body part of the user U. This eliminates the need for the user U to attach sensors and devices to all body parts of the user U whose movements it is desired to grasp, as in the past, thereby reducing the financial, physical, and time burden on the user U.

[0095] Furthermore, the information processing device 1 according to this embodiment can estimate the physical condition of the user U (risk of injury and level of physical fatigue) in addition to the movement of the second body part. This allows the user U to improve the amount and method of exercise according to his or her physical condition. As a result, the user U can exercise in a healthy and effective manner over the long term.

[0096] As described above, the present invention has been described using embodiments. However, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist. For example, all or part of the device can be configured by being functionally or physically distributed and integrated in any unit. Also, new embodiments resulting from any combination of a plurality of embodiments are included in the embodiments of the present invention. The effects of the new embodiments resulting from the combination have the effects of the original embodiments combined.

Explanation of Signs

[0097] 1 Information processing apparatus 11 Device communication section 12 Storage section 13 Control section 131 Reception section 132 Identification section 133 Estimation section 134 Output section 2 Information terminal 21 Operation section 22 Terminal communication section 23 Display section 24 Storage section 25 Control section 251 Display processing section 252 Setting reception section U User D Device S Information processing system

Claims

1. A storage unit that stores an estimation model created based on first motion data indicating the movement of the first part and second motion data indicating the movement of the second part, which are acquired at the same timing by a first device attached to the first part of the body and a second device attached to a second part different from the first part in the body, and that outputs motion data indicating the movement of the second part when motion data indicating the movement of the first part is input; A receiving unit that receives measured motion data indicating the movement of the first part of the subject, which is acquired by a device attached to the first part of the subject to be analyzed; An estimating unit that inputs the measured motion data into the estimation model and acquires estimated motion data indicating the movement of the second part of the subject from the estimation model; An output unit that outputs the estimated motion data; An information processing apparatus having the above.

2. The storage unit stores the estimation model for each type of movement; The receiving unit receives the type of movement; The estimating unit inputs the measured motion data into the estimation model corresponding to the received type of movement and acquires the estimated motion data from the estimation model. The information processing apparatus according to Claim 1. The information processing apparatus according to Claim 1.

3. The storage unit stores the estimation model for each type of movement; The information processing apparatus further includes an identifying unit that identifies the type of movement indicated by the received measured motion data; The estimating unit inputs the measured motion data into the estimation model corresponding to the identified type of movement and acquires the estimated motion data from the estimation model. The information processing apparatus according to Claim 1. The information processing apparatus according to Claim 1.

4. The estimation model has a plurality of sets of the first motion data and the second motion data, identifies the first motion data having a similarity equal to or higher than a predetermined threshold with the input measured motion data, and outputs the second motion data associated with the identified first motion data. The information processing apparatus according to Claim 1. The information processing apparatus according to Claim 1.

5. The estimation model is a machine learning model obtained by performing machine learning using a plurality of sets of the first motion data and the second motion data as teacher data, and outputs the most probable motion data as the motion data of the second part corresponding to the input measured motion data. The information processing apparatus according to claim 1.

6. The estimation model is created based on the first motion data, the second motion data, and state data indicating the state of the body when the motion indicated by the first motion data and the motion indicated by the second motion data are performed. When motion data indicating the motion of the first part is input, it is a model that outputs state estimation data that is the result of estimating the motion data indicating the motion of the second part and the state of the body. The estimation unit inputs the measured motion data into the estimation model, and acquires the estimated motion data and the state estimation data from the estimation model. The output unit outputs the estimated motion data and the state estimation data. The information processing apparatus according to claim 1.

7. The first part is one part of the body, and the second part is one or more parts of the body. The information processing apparatus according to claim 1.

8. The first part is the neck, and the second part is a part of the body other than the neck. The information processing apparatus according to claim 7.

9. The estimation model is created based on the first motion data, the second motion data, and third motion data indicating the motion of a third part, which are acquired at the same timing by the first device, the second device, and a third device attached to a third part different from the first part and the second part. When motion data indicating the motion of the first part and motion data indicating the motion of the third part are input, it is a model that outputs motion data indicating the motion of the second part. The receiving unit receives first measured motion data indicating the motion of the first part of the subject, which is acquired by a device attached to the first part of the subject, and third measured motion data indicating the motion of the third part of the subject, which is acquired by a device attached to the third part of the subject. The estimation unit inputs the first measured motion data and the third measured motion data into the estimation model, and obtains the estimated motion data indicating the movement of the second part of the subject from the estimation model. The information processing apparatus according to claim 1.

10. Among the first measured motion data and the third measured motion data, the estimation unit weights and averages the estimated motion data based on the first measured motion data and the estimated motion data based on the third measured motion data so that the measured motion data of the part closer to the second part is reflected at a higher rate in the estimated motion data indicating the movement of the second part of the subject, and obtains the motion data as the estimated motion data indicating the movement of the second part of the subject. The information processing apparatus according to claim 9.

11. A storage unit that stores an estimation model created based on first motion data indicating the movement of a first part and second motion data indicating the movement of a second part, which are acquired at the same timing by a first device attached to a first part of the body and a second device attached to a second part different from the first part of the body, and that outputs motion data indicating the movement of the second part when motion data indicating the movement of the first part is input. The computer having the storage unit executes the following steps. A receiving step of receiving measured motion data indicating the movement of the first part of the subject, which is acquired by a device attached to the first part of the subject to be analyzed. An estimating step of inputting the measured motion data into the estimation model and obtaining estimated motion data indicating the movement of the second part of the subject from the estimation model. An output step of outputting the estimated motion data. An information processing method having the above steps.

12. An information processing apparatus and an information terminal are provided. The information processing apparatus is as follows. A storage unit that stores an estimation model created based on first motion data indicating the movement of the first part and second motion data indicating the movement of the second part, which are acquired at the same timing by a first device worn on the first part of the body and a second device worn on a second part different from the first part in the body, and that outputs motion data indicating the movement of the second part when motion data indicating the movement of the first part is input. A receiving unit that receives measured motion data indicating the movement of the first part of the subject, which is acquired by a device worn on the first part of the subject to be analyzed. An estimation unit that inputs the measured motion data into the estimation model and acquires estimated motion data indicating the movement of the second part of the subject from the estimation model. An output unit that outputs the estimated motion data. It has: The information terminal has a setting reception unit that receives settings related to the device worn on the first part of the subject, and a terminal communication unit that receives the estimated motion data. An information processing system.

Images