MOTION ANALYSIS DEVICE, MOTION ANALYSIS METHOD, MOTION ANALYSIS PROGRAM, AND MOTION ANALYSIS SYSTEM

Abstract

To provide a motion analyzing device capable of presenting factors to be paid attention during motion.SOLUTION: A motion analyzing device is provided with: a trained model 19b trained to accept input of time-series information relating to a motion performed in order to achieve a specific objective, and to output result information relating to a result obtained through the motion; a receiving unit 12 for receiving target time-series information relating to a motion to be analyzed; an extraction unit 13 for extracting, on the basis of the result information estimated by inputting the received target time-series information into the trained model 19b, description information indicating a factor contributing to the estimation of the result information for the motion to be analyzed; and an output unit 14 for outputting information relating to an analysis result for the motion to be analyzed, on the basis of the extracted description information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a motion analysis device, a motion analysis method, a motion analysis program, and a motion analysis system. [Background technology]

[0002] In training sites for various sports and production sites such as factories, the abilities of athletes and the skills of workers are often evaluated by analyzing the movements of athletes and workers. For example, Patent Document 1 below discloses an evaluation system that objectively evaluates the movements of athletes during exercise.

[0003] This evaluation system uses classification machine learning to identify plays and performances to be evaluated from the motion data that represents a series of movements of the player, and then uses regression machine learning to evaluate the degree of completion of the identified plays and performances. This allows players and coaches to recognize how complete a player's play or performance is. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-68516 A Summary of the Invention [Problem to be solved by the invention]

[0005] By the way, when playing sports, you often feel like you did well but don't know why. For example, when you are playing darts and the dart hits the center of the board, it is difficult to understand how and when you moved which joints to make the dart hit the center of the board.

[0006] The evaluation system of Patent Document 1 can present an evaluation of a play, but cannot present the factors that made it successful.

[0007] Therefore, the present invention provides a motion analysis device, a motion analysis method, a motion analysis program, and a motion analysis system that are capable of presenting factors that should be considered when performing a motion. [Means for solving the problem]

[0008] A motion analysis device according to one aspect of the present disclosure includes a trained model that is trained to receive time series information regarding a motion performed to achieve a specific purpose as input and to output result information regarding a result resulting from the motion, a receiving unit that receives target time series information regarding the motion to be analyzed, an extraction unit that extracts explanatory information indicating factors that contribute to the estimation of the result information for the motion to be analyzed based on result information estimated by inputting the received target time series information into the trained model, and an output unit that outputs information regarding the analysis result for the motion to be analyzed based on the extracted explanatory information.

[0009] According to this aspect, it is possible to input time-series information about the action to be analyzed into a trained model, and based on result information estimated by the trained model, extract information from the time-series information that explains which part of the action to be analyzed affects the result and how, and to output an analysis result for the action to be analyzed based on the extracted information.

[0010] In the above aspect, the receiving unit may further receive target result information regarding results caused by the action to be analyzed, and the extraction unit may extract explanatory information indicating factors that contribute to the estimation of the target result information in the action to be analyzed based on result information estimated by inputting the received target time series information into the learned model.

[0011] According to this aspect, when the accuracy of the trained model is low, it is possible to extract information that explains which part of the action being analyzed affects the results produced by the action being analyzed and how.

[0012] In the above aspect, the explanatory information may include information indicating a timing between the start and end of the analysis target motion that contributed to the estimation and a body part that contributed to the estimation.

[0013] According to this aspect, the user can objectively recognize the body parts and movement timing that should be taken into consideration when performing the action to be analyzed.

[0014] In the above aspect, the time-series information may be skeletal data, and the body parts may be joints.

[0015] According to this embodiment, when performing the action to be analyzed, it becomes possible to objectively recognize the joints and timing of the movement that should be taken into consideration while looking at the skeletal data.

[0016] A motion analysis method according to another aspect of the present disclosure is a method executed by a processor, and includes training a model to receive time series information regarding a motion performed to achieve a specific purpose as input and output result information regarding a result resulting from the motion, thereby creating a trained model; receiving target time series information regarding the motion to be analyzed; inputting the received target time series information into the trained model and based on the result information estimated, extracting explanatory information indicating factors that contribute to the estimation of the result information in the motion to be analyzed; and outputting information regarding the analysis result for the motion to be analyzed based on the extracted explanatory information.

[0017] According to this aspect, it is possible to input time-series information about the action to be analyzed into a trained model, and based on result information estimated by the trained model, extract information from the time-series information that explains which part of the action to be analyzed affects the result and how, and to output an analysis result for the action to be analyzed based on the extracted information.

[0018] A motion analysis program according to another aspect of the present disclosure causes a computer to function as a trained model that is trained to receive time-series information regarding a motion performed to achieve a specific purpose as input and to output result information regarding a result resulting from the motion; a receiving unit that receives target time-series information regarding the motion to be analyzed; an extracting unit that extracts explanatory information indicating factors that contribute to the estimation of the result information in the motion to be analyzed based on result information estimated by inputting the received target time-series information into the trained model; and an output unit that outputs information regarding the analysis result for the motion to be analyzed based on the extracted explanatory information.

[0019] According to this aspect, it is possible to input time-series information about the action to be analyzed into a trained model, and based on result information estimated by the trained model, extract information from the time-series information that explains which part of the action to be analyzed affects the result and how, and to output an analysis result for the action to be analyzed based on the extracted information.

[0020] A motion analysis system according to another aspect of the present disclosure is a motion analysis system including a sensor terminal and a motion analysis device, wherein the sensor terminal includes a detection unit that detects a motion performed to achieve a specific purpose and outputs time series information related to the motion, and a transmission unit that transmits the output time series information as target time series information related to the motion to be analyzed, and the motion analysis device includes a trained model that has been trained to receive the time series information and output result information related to a result caused by the motion, a receiving unit that receives the target time series information, an extraction unit that extracts explanatory information indicating factors that contribute to the estimation of the result information in the motion to be analyzed based on result information estimated by inputting the received target time series information into the trained model, and an output unit that outputs information related to the analysis result for the motion to be analyzed based on the extracted explanatory information.

[0021] According to this aspect, it is possible to input time-series information about the action to be analyzed into a trained model, and based on result information estimated by the trained model, extract information from the time-series information that explains which part of the action to be analyzed affects the result and how, and to output an analysis result for the action to be analyzed based on the extracted information. Effect of the Invention

[0022] According to the present invention, it is possible to provide a motion analysis device, a motion analysis method, a motion analysis program, and a motion analysis system that are capable of presenting factors that should be considered when performing a motion. [Brief description of the drawings]

[0023] [Figure 1] 1 is a diagram illustrating a configuration of a motion analysis system according to an embodiment of the present invention. [Diagram 2] FIG. 13 is a diagram showing an example of a screen for inputting subject result information. [Diagram 3] FIG. 13 is a diagram showing an example of an image displayed on a display of a sensor terminal based on an analysis result. [Figure 4] 1 is a diagram illustrating an example of a hardware configuration of a motion analysis device according to an embodiment of the present invention. [Diagram 5] 11 is a flowchart of a learning data storage process executed by the motion analysis device according to the present embodiment. [Figure 6] 10 is a flowchart of a model learning process executed by the motion analysis device according to the present embodiment. [Figure 7] 1 is a flowchart of a motion analysis process executed by the motion analysis device according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] Hereinafter, an embodiment according to one aspect of the present invention (hereinafter, referred to as "the present embodiment") will be described with reference to the drawings. In each drawing, the same reference numerals denote the same or similar configurations.

[0025] §1 Examples of application First, an example of a scene to which the present invention is applied will be described with reference to Fig. 1. In the motion analysis system 100 according to this embodiment, for example, the motion of user A is captured by a camera of a sensor terminal 20 such as a smartphone, and motion data (an example of time-series information) extracted from the captured video and the results caused by the motion of user A are transmitted to the motion analysis device 10 as the analysis target motion and the analysis target result. Then, based on the result information estimated by inputting the analysis target motion into the learned model 19b, the motion analysis device 10 extracts explanatory information explaining what factors contributed to the analysis target result, and outputs the explanatory information to the sensor terminal 20 as the result of analyzing the motion of user A.

[0026] Here, the motion analysis device 10 may extract explanatory information that explains what factors contributed to the result that was estimated based on the result output by inputting the motion to be analyzed into the learned model 19b.

[0027] The above-mentioned actions of user A include, for example, the action of throwing a dart towards a dart board when playing darts, the action of putting a golf ball towards a cup in putt-putt golf, and the action of hitting a table tennis ball back along the intended course in table tennis.

[0028] For example, when throwing a dart at a dartboard, the results of the above-mentioned action of user A may include the dart hitting the center of the dartboard, falling above the center of the dartboard, falling below the center of the dartboard, falling to the right of the center of the dartboard, falling to the left of the center of the dartboard, or missing the dartboard.

[0029] The trained model 19b is an AI model trained to receive motion data indicating a user's action as input and output result information related to the result of the action. For example, the trained model 19b receives motion data indicating an action of throwing a dart toward a board as input and outputs result information indicating whether the dart has been stuck in the center of the board. Details of the trained model 19b will be described later.

[0030] The explanatory information includes, for example, information indicating a timing between the start and end of the analysis target motion, and a body part. Specific explanations are given below.

[0031] If the content of the analysis result is that the dart hits the center of the board, the explanatory information may include information indicating the timing "from 3 seconds to 4 seconds after the start of the motion data" and information indicating the body part "right elbow." In this case, it is understood that "the reason the dart hits the center of the board is due to the movement of the right elbow from 3 seconds to 4 seconds after the start of the motion data." In other words, the user can learn that by being conscious of the movement of the right elbow from 3 seconds to 4 seconds after the start of the motion data, the possibility of the dart hitting the center can be increased.

[0032] In this way, according to the motion analysis apparatus 10 of this embodiment, it is possible to present to the user factors that the user should be aware of when making a motion.

[0033] §2 Configuration Example [Function configuration] Next, an example of the functional configuration of a motion analysis system 100 according to this embodiment will be described with reference to Fig. 1. The motion analysis system 100 includes a sensor terminal 20 and a motion analysis device 10. The sensor terminal 20 and the motion analysis device 10 are connected via a communication network N.

[0034] <Sensor terminal 20> The sensor terminal 20 is, for example, a smartphone with a camera, which captures a scene in which the user A is performing a motion with the camera, extracts motion data from the captured video, and transmits the motion analysis device 10.

[0035] The motion data includes, for example, skeletal data indicating the movement of the user's skeleton, coordinate value data indicating the user's movement, and sensor data indicating the user's movement, but what is transmitted to the motion analysis device 10 is not limited to motion data. Any time-series information indicating the user's movement may be transmitted, and such time-series information includes not only motion data but also videos capturing the user's movement.

[0036] The skeletal data can be detected, for example, by a skeletal detection application installed on a smartphone. The coordinate value data can be measured, for example, by a motion capture device provided in place of the sensor terminal 20. The sensor data can be measured, for example, by an acceleration sensor or a gyro sensor provided in place of the sensor terminal 20. It is preferable that the acceleration sensor or the gyro sensor is worn by the user. The video can be captured, for example, by a smartphone camera. When using a video, the motion analysis device 10 that receives the video may detect motion data such as the skeletal data.

[0037] In this embodiment, a case will be described in which skeletal data is used as motion data, as an example.

[0038] The sensor terminal 20 has, as its functional configuration, for example, a detection unit, an input reception unit, and a transmission unit. The detection unit detects the movement of the user A, and outputs motion data indicating the movement of the user A's skeleton.

[0039] The input receiving unit receives an input of result information regarding the result caused by the user A's action.

[0040] The transmission unit transmits the motion data output from the detection unit and the result information input to the input reception unit to motion analysis device 10. The motion data transmitted to motion analysis device 10 is transmitted as motion data related to the motion to be analyzed (hereinafter referred to as "target motion data"). The result information transmitted to motion analysis device 10 is transmitted as result information related to the result caused by the motion to be analyzed (hereinafter referred to as "target result information").

[0041] FIG. 2 illustrates an example of a screen for inputting target result information. The screen in FIG. 2 is a screen for inputting the result of throwing a dart at a dartboard. The display of the sensor terminal 20 has a field Fa for inputting the coordinates of the location where the dart landed, and a field Fb for inputting a label ID for identifying the result of throwing the dart. The coordinates can be expressed, for example, by XY coordinates with the center of the dartboard as the origin. The label ID can be set, for example, according to the position or area on the dartboard. The position or area on the dartboard can be, for example, the center of the dartboard, above the center, below the center, to the right of the center, left of the center, etc.

[0042] <Motion analysis device 10> 1 has, as functional components, for example, a learning unit 11, a receiving unit 12, an extracting unit 13, an output unit 14, and a memory unit 19. The memory unit 19 stores, for example, learning data 19a and a trained model 19b. Each functional component will be described below.

[0043] <Learning Data 19a> The learning data 19a includes motion data, which is time-series information about actions taken to achieve a specific purpose, and result information about the results of the actions. The motion data and the result information are stored in association with each other.

[0044] Examples of actions performed to achieve a specific purpose include the action of throwing a dart at a dart board when playing darts, the action of putting a golf ball towards a cup in golf, and the action of hitting a table tennis ball along the intended course in table tennis.

[0045] A plurality of pieces of result information can be set for each of the above actions. After performing an action, the user inputs a corresponding piece of result information from among the plurality of pieces of result information set to the sensor terminal 20. Examples of the plurality of pieces of result information set for each action are shown below.

[0046] If the action is the action of throwing a dart at a dartboard, the result information can be set to indicate that the dart hit the center of the dartboard, fell above the center of the dartboard, fell below the center of the dartboard, fell to the right of the center of the dartboard, fell to the left of the center of the dartboard, or missed the dartboard.

[0047] If the action is putting a golf ball towards a cup, the result information can be set to indicate that the ball went in, that the ball did not reach the cup, that the ball rolled further than the cup, etc.

[0048] When the action is the action of hitting a table tennis ball back along the intended course, the result information can be set as, for example, whether the ball was hit along the intended course, whether it was above the intended course, whether it was below the intended course, whether it was to the right of the intended course, or whether it was to the left of the intended course.

[0049] <Trained model 19b> The trained model 19b is an AI model trained using the training data 19a, and is created by the training unit 11 described below.

[0050] <Study Section 11> The learning unit 11 creates a learned model 19b by training an AI model so that the motion data of the learning data 19a is input and the result information of the learning data 19a is output. For example, GCN (Graph Convolutional Networks) and ST-GCN (Spatial Temporal GCN) can be used as a method for training the AI ​​model.

[0051] For example, when motion data showing the action of throwing a dart at a dartboard is input to the trained model 19b, result information showing whether or not the dart hit the center of the dartboard can be output from the trained model 19b.

[0052] <Receiving section 12> The receiving unit 12 receives the target motion data and the target result information from the sensor terminal 20 .

[0053] <Extract part 13> The extraction unit 13 extracts explanatory information indicating factors that contribute to the estimation of the target result information from the target motion data, based on the result information estimated by inputting the target motion data to the trained model 19b.

[0054] The result information estimated by inputting the target motion data into the trained model 19b may include, for example, multiple results set as the results of the action and the probability of each result. Specifically, a case where "the dart hit the center of the board" and "the dart did not hit the center of the board" are set as the results of throwing a dart will be described as an example.

[0055] In this case, the estimated result information may include, for example, information that the probability that the ball "hit the center of the board" was 40% and the probability that the ball "did not hit the center of the board" was 60%.

[0056] Here, if the accuracy of the trained model 19b is 100% or close to 100%, the result output by inputting the target motion data will match or approximately match the result of the target result information. In this case, the extraction unit 13 may extract explanatory information corresponding to the result based on the result output from the trained model 19b.

[0057] On the other hand, when the accuracy of the trained model 19b is low, there will be cases where the result output from the trained model 19b differs from the actual result. Assuming such a case, the extraction unit 13 may extract explanatory information that explains what factors contributed under the conditions estimated to result in the target result information, which is the actual result. For example, when the result of actually throwing a dart is "sticking to the center of the board," the result information estimated by inputting the target motion data into the trained model 19b is that the probability of "sticking to the center of the board" is 40% and the probability of "not sticking to the center of the board" is 60%. In this case, the extraction unit 13 extracts explanatory information that explains what factors contributed under the conditions estimated by the trained model 19b to be "sticking to the center of the board."

[0058] The explanatory information may include, for example, information indicating any timing between the start and end of the target motion data that contributed to the estimation of the result, and information indicating the body parts that contributed to the estimation of the result. The explanatory information may also include a score (importance of movement) indicating how important each body part is to the movement of the target motion data.

[0059] The above timing may be, for example, the time (for example, seconds) that has elapsed since the start of the target motion data, or the number of frames that have elapsed since the start of the target motion data.

[0060] The body parts can be specified by joints such as shoulders, elbows, wrists, etc. In this case, the importance of the movement is calculated for each joint.

[0061] As a method for extracting explanatory information, any visualization method based on explainable AI, such as Class Activation Mapping (CAM), Grad-CAM (Gradient-weighted CAM), Grad-CAM++, Attention Branch Network (ABN), and Human-in-the-loop ABN (HITL-ABN), can be used.

[0062] <Output section 14> The output unit 14 outputs information on the analysis result for the analysis target motion based on the explanatory information extracted by the extraction unit 13.

[0063] The information regarding the analysis results will be explained below using a specific example.

[0064] (1) If the result information is "the ball did not penetrate the center of the board" and the explanation information includes information indicating the timing, "from 3 seconds after the start of the target motion data to 4 seconds after the start of the target motion data," and information indicating the body part, "right elbow," then:

[0065] The analysis results include information indicating that the reason the ball "did not hit the center of the board" is due to "the movement of the right elbow between 3 and 4 seconds after the start of the motion data."

[0066] (2) If the result information is "The ball was struck in the center of the board" and the explanatory information includes information indicating the timing, "from 3 seconds after the start of the target motion data to 4 seconds after the start of the target motion data," and information indicating the body parts, "right elbow" and "right wrist":

[0067] The analysis results include information indicating that the reason the ball "hit the center of the board" was due to "the movement of the right elbow and right wrist between 3 and 4 seconds after the start of the motion data."

[0068] The sensor terminal 20 displays the skeletal data superimposed on the movement of the user captured by the camera, based on the information on the analysis result output from the output unit 14. This will be specifically described below.

[0069] FIG. 3 shows a scene of an image displayed on the display of the sensor terminal 20 based on the information related to the analysis result in (2) above.

[0070] On the display of the sensor terminal 20, skeletal data Mb based on information related to the analysis result is displayed superimposed on an image Ma of the user repeatedly throwing a dart.

[0071] The joints in the skeleton data Mb can be displayed in different colors according to the importance of the movement, for example. For example, the colors may be changed in the order of red, white, and blue as the importance of the movement increases.

[0072] In the case of the information on the analysis results of (2) above, the "movement of the right elbow and right wrist from 3 seconds to 4 seconds after the start" is important. Therefore, in the skeletal data Mb from 3 seconds to 4 seconds after the start, the joint parts of the right elbow and right wrist are displayed in blue, and the other joint parts are displayed in red.

[0073] By providing such footage, users will be able to objectively recognize which joints and timing of movements they should be aware of when throwing a dart.

[0074] In addition, when displaying the skeletal data based on the information on the analysis result, it is not limited to displaying it superimposed on the user's movement, but the skeletal data may be displayed around the user. In addition, it is not limited to displaying the skeletal data according to the user's movement captured by the camera. For example, a video used for extracting the motion data may be saved, and when the video is played, the skeletal data may be displayed according to the user's movement in the video. In addition, only the importance of the movement may be displayed according to the user's movement. Furthermore, for example, a message such as "The movement of the right elbow and right wrist from 3 seconds after the start to 4 seconds after is important" may be displayed.

[0075] [Hardware configuration] Next, an example of the hardware configuration of the motion analysis device 10 according to the present embodiment will be described with reference to FIG. 4. The motion analysis device 10 has a CPU (Central Processing Unit) 10a corresponding to the arithmetic unit, a RAM (Random Access Memory) 10b corresponding to the storage unit 19, a ROM (Read only Memory) 10c corresponding to the storage unit 19, a communication device 10d, an input device 10e corresponding to the receiving unit 12, and an output device 10f corresponding to the output unit 14. These components are connected to each other via a bus so that they can transmit and receive data to each other. Note that, although the present embodiment describes a case where the motion analysis device 10 is configured by one computer, the motion analysis device 10 may be realized using multiple computers.

[0076] The CPU 10a executes programs stored in the RAM 10b or the ROM 10c, and functions as a control unit that performs calculations and processing of data. The CPU 10a receives various input data from the input device 10e or the communication device 10d, and outputs the results of calculations on the input data from the output device 10f or stores them in the RAM 10b or the ROM 10c. The CPU 10a in this embodiment executes a program (motion analysis program) that outputs an analysis result that explains factors that contribute to a result estimated from the movement of the motion data. The calculation device is not limited to a CPU, and may be another processor such as a GPU (Graphics Processing Unit).

[0077] The RAM 10b is made up of, for example, a semiconductor memory device, and stores rewritable data.The ROM 10c is made up of, for example, a semiconductor memory device, and stores readable but non-rewritable data.

[0078] The communication device 10d is an interface that connects the motion analysis device 10 to an external device. The communication device 10d is connected to the sensor terminal 20 via a communication network such as a LAN (Local Area Network) or the Internet, and receives motion data from the sensor terminal 20.

[0079] The input device 10e is an interface that accepts data input from a user, and can include, for example, a keyboard, a mouse, and a touch panel.

[0080] The output device 10f is an interface that outputs the results of calculations performed by the CPU 10a, etc. The output device 10f may be an interface that visually displays information. In this case, the output device 10f may be configured, for example, with an LCD (Liquid Crystal Display).

[0081] The motion analysis program may be provided by being stored in a computer-readable storage medium such as RAM 10b or ROM 10c, or may be provided via a communication network connected by communication device 10d. In motion analysis device 10, CPU 10a executes the motion analysis program, thereby realizing the operations of learning unit 11, receiving unit 12, extracting unit 13, and output unit 14 shown in FIG. 1. Note that these physical configurations are merely examples, and do not necessarily have to be independent configurations. For example, motion analysis device 10 may include an LSI (Large-Scale Integration) in which CPU 10a is integrated with RAM 10b and ROM 10c.

[0082] §3 Example of operation An example of a learning data storage process executed by the motion analysis apparatus 10 according to the present embodiment will be described with reference to FIG.

[0083] First, the receiving unit 12 of the motion analysis device 10 receives, from the sensor terminal 20, motion data of a motion performed by a user to achieve a specific purpose (step S101).

[0084] Next, the receiving unit 12 of the motion analysis device 10 receives, from the sensor terminal 20, result information corresponding to the motion data received in the above step S101 (step S102).

[0085] Next, storage unit 19 of motion analysis device 10 associates the motion data received in step S101 with the result information received in step S102, and stores them as learning data 19a (step S103). Then, the learning data storage process ends.

[0086] An example of a model learning process executed by the motion analysis apparatus 10 according to the present embodiment will be described with reference to FIG.

[0087] First, learning unit 11 of motion analysis device 10 trains an AI model so that motion data of learning data 19a is input and result information of learning data 19a is output (step S201).

[0088] Next, storage unit 19 of motion analysis device 10 stores trained model 19b created by the training in step S201 (step S202), and the model training process ends.

[0089] An example of the motion analysis process executed by the motion analysis device 10 according to this embodiment will be described with reference to FIG.

[0090] First, the receiving unit 12 of the motion analysis device 10 receives the target motion data from the sensor terminal 20 (step S301).

[0091] Next, the receiving unit 12 of the motion analysis device 10 receives target result information corresponding to the target motion data from the sensor terminal 20 (step S302).

[0092] Next, the extraction unit 13 of the motion analysis device 10 extracts explanatory information indicating factors that contribute to the estimation of the target result information received in step S302 from the target motion data based on the result information estimated by inputting the target motion data received in step S301 into the learned model 19b (step S303).

[0093] Next, output section 14 of motion analysis device 10 outputs information on the analysis result for the analysis target motion based on the explanatory information extracted in step S303 above (step S304), and then ends the motion analysis process.

[0094] This makes it possible to display skeletal data on the display of the sensor terminal 20 based on information related to the analysis results.

[0095] As described above, according to the motion analysis device 10 of this embodiment, it is possible to extract information from the motion data that explains how which body part affects the result at which timing of the movement to be analyzed, based on result information estimated by inputting motion data related to the movement to be analyzed into the learned model 19b, and to output an analysis result for the movement to be analyzed based on the extracted information.

[0096] Furthermore, according to the motion analysis device 10 of this embodiment, when the accuracy of the learned model 19b is low, it is possible to extract information that explains how which body part affects the result at which timing of the motion to be analyzed with respect to the result actually produced by the motion to be analyzed.

[0097] Therefore, the motion analysis device 10 according to the present embodiment can present factors that should be considered when making a motion.

[0098] The embodiment of the present invention may be described as in the following appendices. However, the embodiment of the present invention is not limited to the form described in the appendices. In addition, the embodiment of the present invention may be a form in which the descriptions between the appendices are replaced or combined.

[0099] [Appendix 1] A trained model (19b) that is trained to input time-series information about an action taken to achieve a specific purpose and output result information about the result of the action; A receiving unit (12) that receives target time-series information regarding an analysis target motion; an extraction unit (13) that extracts explanatory information indicating factors that contribute to the estimation of the result information in the analysis target motion based on result information estimated by inputting the received target time-series information into the trained model (19b); an output unit (14) that outputs information regarding an analysis result for the analysis target motion based on the extracted explanatory information; A motion analysis device (10) comprising:

[0100] [Appendix 2] The receiving unit (12) further receives target result information regarding a result caused by the analysis target action, The extraction unit (13) extracts explanatory information indicating factors that contribute to the estimation of the target result information in the analysis target action based on result information estimated by inputting the received target time-series information into the trained model. 2. The motion analysis device (10) of claim 1.

[0101] [Appendix 3] The explanatory information includes information indicating any timing between the start and end of the analysis target motion that contributed to the estimation and information indicating a body part that contributed to the estimation. 3. The motion analysis device (10) according to claim 1 or 2.

[0102] [Appendix 4] The time-series information is skeletal data, and the body parts are joints. 4. The motion analysis device (10) of claim 3.

[0103] [Appendix 5] 1. A processor-implemented method, comprising: training a model to input time-series information on an action taken to achieve a specific purpose and output result information on the results of the action, thereby creating a trained model (19b); receiving target time series information relating to a target motion; Extracting explanatory information indicating factors that contribute to the estimation of the result information in the analysis target motion based on result information estimated by inputting the received target time-series information into the trained model (19b); outputting information related to an analysis result for the analysis target motion based on the extracted explanatory information; A motion analysis method comprising:

[0104] [Appendix 6] Computer, A trained model (19b) that is trained to input time-series information about an action taken to achieve a specific purpose and output result information about the result of the action; A receiving unit (12) for receiving target time-series information relating to an analysis target motion; an extraction unit (13) that extracts explanatory information indicating factors that contribute to the estimation of the result information in the analysis target motion based on result information estimated by inputting the received target time-series information into the trained model (19b); an output unit (14) that outputs information regarding an analysis result for the analysis target motion based on the extracted explanatory information; A motion analysis program that functions as a

[0105] [Appendix 7] A motion analysis system (100) including a sensor terminal (20) and a motion analysis device (10), The sensor terminal (20) a detection unit that detects an action taken to achieve a specific purpose and outputs time-series information regarding the action; a transmission unit that transmits the output time-series information as target time-series information regarding an analysis target motion; Equipped with The motion analysis device (10) A trained model (19b) trained to input the time series information and output result information regarding the result caused by the operation; A receiving unit (12) for receiving the target time series information; an extraction unit (13) that extracts explanatory information indicating factors that contribute to the estimation of the result information in the analysis target motion based on result information estimated by inputting the received target time-series information into the trained model (19b); an output unit (14) that outputs information regarding an analysis result for the analysis target motion based on the extracted explanatory information; Equipped with A motion analysis system (100). [Explanation of symbols]

[0106] 10...motion analysis device, 10a...CPU, 10b...RAM, 10c...ROM, 10d...communication device, 10e...input device, 10f...output device, 11...learning unit, 12...receiving unit, 13...extraction unit, 14...output unit, 19...storage unit, 19a...learning data, 19b...learned model, 20...sensor terminal, 100...motion analysis system, N...communication network, A...user, Ma...user image, Mb...skeletal data

Claims

1. A trained model that is trained to input time-series information about an action taken to achieve a specific purpose and output result information about the result of the action; A receiving unit that receives target time-series information regarding an analysis target motion; an extraction unit that extracts explanatory information indicating factors that contribute to an estimation of the result information in the analysis target action based on result information estimated by inputting the received target time-series information into the trained model; an output unit that outputs information related to an analysis result for the analysis target motion based on the extracted explanatory information; Equipped with The receiving unit further receives target result information regarding a result caused by the analysis target action, The extraction unit extracts explanatory information indicating factors that contribute to the estimation of the target result information in the analysis target action based on result information estimated by inputting the received target time-series information into the trained model. Motion analysis device.

2. The explanatory information includes information indicating any timing between the start and end of the analysis target motion that contributed to the estimation and information indicating a body part that contributed to the estimation. The motion analysis device according to claim 1.

3. The time-series information is skeletal data, and the body parts are joints. The motion analysis device according to claim 2.

4. 1. A processor-implemented method, comprising: A model is trained so that time-series information regarding an action taken to achieve a specific purpose is input and result information regarding the result caused by the action is output, thereby creating a trained model; receiving target time series information relating to a target motion; extracting explanatory information indicating factors that contribute to the estimation of the result information in the analysis target motion based on result information estimated by inputting the received target time-series information into the trained model; outputting information related to an analysis result for the analysis target motion based on the extracted explanatory information; Including, The receiving step further includes receiving target result information regarding a result caused by the analyzed action; The extracting step includes extracting explanatory information indicating factors that contribute to the estimation of the target result information in the analysis target action based on result information estimated by inputting the received target time-series information into the trained model. Motion analysis method.

5. Computer, A trained model that is trained to input time-series information about an action taken to achieve a specific purpose and output result information about the result of the action; A receiving unit for receiving target time-series information regarding an analysis target motion; an extraction unit that extracts explanatory information indicating factors that contribute to an estimation of the result information in the analysis target action based on result information estimated by inputting the received target time-series information into the trained model; an output unit that outputs information related to an analysis result for the analysis target motion based on the extracted explanatory information; Function as a The receiving unit further receives target result information regarding a result caused by the analysis target action, The extraction unit extracts explanatory information indicating factors that contribute to the estimation of the target result information in the analysis target action based on result information estimated by inputting the received target time-series information into the trained model. Motion analysis program.

6. A motion analysis system including a sensor terminal and a motion analysis device, The sensor terminal includes: a detection unit that detects an action taken to achieve a specific purpose and outputs time-series information regarding the action; a transmission unit that transmits the output time-series information as target time-series information regarding an analysis target motion; Equipped with The motion analysis device includes: A trained model trained to input the time series information and output result information regarding the result caused by the operation; A receiving unit that receives the target time series information; an extraction unit that extracts explanatory information indicating factors that contribute to an estimation of the result information in the analysis target action based on result information estimated by inputting the received target time-series information into the trained model; an output unit that outputs information related to an analysis result for the analysis target motion based on the extracted explanatory information; Equipped with The receiving unit further receives target result information regarding a result caused by the analysis target action, The extraction unit extracts explanatory information indicating factors that contribute to the estimation of the target result information in the analysis target action based on result information estimated by inputting the received target time-series information into the trained model. Motion analysis system.

Images