Motion analysis method, motion analysis program, and motion analysis system

Abstract

To provide a motion analysis method, a motion analysis program and a motion analysis system which can use moving image data of a general camera, and are suitable for analysis of a motion followed by jump and rotation of a body.SOLUTION: A motion analysis method comprises: a moving image data acquiring step S11(1) for acquiring moving image data which is obtained by imaging from a side part, a subject HK. The method further comprises: a two-dimensional coordinate information generation step S12(1) for using a body feature point estimation model 16 obtained by machine learning for each still image data forming the moving image data, for extracting a plurality of feature points of the body of the subject HK, for generating two-dimensional coordinate information which is time series data of a two-dimensional coordinate of each feature point. The method further comprises a motion analysis step S13(1) for, by analyzing the two-dimensional coordinate information, detecting a time change of a relative positional relationship between specific feature points K**, on the basis of the detection result, generating numeric information indicating a feature of an analysis target motion performed by the subject HK. The feature points include at least two points out of left and right waist joints K4L, K4R, left and right knee joints K5L, K5R.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a motion analysis method, a motion analysis program, and a motion analysis system that analyze video data obtained by imaging a subject performing a predetermined analysis target motion and create numerical information indicating the characteristics of the motion.

Background Art

[0002] Conventionally, as disclosed in Patent Document 1 for example, a feature estimation unit that estimates a plurality of body feature points of a subject by a predetermined body posture recognition method that recognizes the body posture of the subject from a moving image of the subject, a reference storage unit that stores a reference length that is the length in the real world of a predetermined reference part, and based on the ratio between the distance on the image corresponding to the reference part determined from the plurality of body feature points and the reference length, as a value used for evaluating the motion of the subject, a motion analysis unit that obtains a value indicating the motion state of the subject from the distance on the image between the body feature points, and an output unit that outputs a value indicating the motion state. There was a motion state evaluation system provided with.

[0003] In the specification and drawings of Patent Document 1, it is described that the running form of a subject is imaged with an information terminal such as a smartphone, the body feature points of the subject are estimated from the video data, and the position of the body feature points and their time change are analyzed, thereby analyzing the grounding timing and running form parameters (speed, stride, pitch, vertical movement width, forward tilt angle of the torso, movable range of the shoulder joint, movable range of the hip joint, etc.).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The motion state evaluation system described in Patent Document 1 is said to be applicable not only to the analysis of running form, but also to the analysis of swing forms in baseball and golf, swimming forms, skiing forms, and forms in skills such as dance and performance. However, even when attempting to apply it to the analysis of sports or performances in which jumping, handstands, somersaults, twists, and spins are important elements, such as trampolining, gymnastics, and figure skating, Patent Document 1 does not disclose any practical and specific analytical methods.

[0006] The present invention has been made in view of the above-mentioned background art, and provides a motion analysis method, motion analysis program, and motion analysis system that can use video data from a general camera and are suitable for analyzing movements involving jumping and body rotation. [Means for solving the problem]

[0007] The present invention relates to a computer-based motion analysis method comprising: a video data acquisition step of acquiring video data of a subject performing a motion to be analyzed, captured from the side; a two-dimensional coordinate information creation step of extracting multiple feature points of the subject's body using a body feature point estimation model created by machine learning for each still image data constituting the video data, and creating two-dimensional coordinate information which is time-series data of the two-dimensional coordinates of the extracted feature points; and a motion analysis step of detecting the time change in the relative positional relationship between specific feature points by analyzing the two-dimensional coordinate information, and creating numerical information indicating the characteristics of the motion to be analyzed performed by the subject based on the detection results, wherein the feature points extracted using the body feature point estimation model include at least two points from the left and right shoulder joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joints.

[0008] For example, the motion to be analyzed is a rotational motion in which the subject's body rotates forward or backward. In the motion analysis step, the following processes are performed: Recognize a straight line from the two-dimensional coordinate information that connects the hip joint to the knee joint, ankle joint, or shoulder joint, and detect the change in the position of the straight line over time as a rotational motion with the hip joint as the origin; or Recognize a straight line from the two-dimensional coordinate information that connects the shoulder joint to the hip joint, knee joint, or ankle joint, and detect the change in the position of the straight line over time as a rotational motion with the shoulder joint as the origin. Based on this detection result, rotation count information, rotation speed information, or both are created.

[0009] For example, the motion to be analyzed is an axial motion in which the subject's body rotates around a virtual straight line passing through the upper and lower body. In the motion analysis step, the system recognizes the distance between the left and right hip joints from the two-dimensional coordinate information and detects the time change of said distance, or recognizes the distance between the left and right shoulder joints from the two-dimensional coordinate information and detects the time change of said distance. Based on these detection results, the system creates information on the number of axial rotations, axial rotation speed, or both.

[0010] for example, When determining whether the posture of the subject performing the aforementioned action to be analyzed is "extended" or "not extended," in the motion analysis step, a first line connecting the hip joint and the shoulder joint, a second line connecting the hip joint and the knee joint, and a third line connecting the knee joint and the ankle joint are recognized from the two-dimensional coordinate information. If the intersection angle between the first line and the second line is greater than the first reference value, and the intersection angle between the second line and the third line is greater than the second reference value, it is determined to be "extended." If it does not fall under "extended," it is determined to be "not extended." Based on this determination result, posture determination information indicating the change in the subject's posture over time is generated. Create .

[0011] for exampleWhen determining whether the posture of the subject performing the action to be analyzed is "tucked" or "not tucked," in the action analysis step, a first line connecting the hip joint and the shoulder joint, a second line connecting the hip joint and the knee joint, and a third line connecting the knee joint and the ankle joint are recognized from the two-dimensional coordinate information. If the intersection angle between the first line and the second line is smaller than the first reference value, and the intersection angle between the second line and the third line is smaller than the second reference value, it is determined to be "tucked." If it does not fall under "tucked," it is determined to be "not tucked." Based on this determination result, posture determination information indicating the change in the subject's posture over time is generated. Create .

[0012] for example When determining whether the posture of the subject performing the action to be analyzed is "flexed" or "unflexed", in the action analysis step, a first line connecting the hip joint and the shoulder joint, a second line connecting the hip joint and the knee joint, and a third line connecting the knee joint and the ankle joint are recognized from the two-dimensional coordinate information, and the posture is determined to be "flexed" when the intersection angle between the first line and the second line is smaller than the first reference value and the intersection angle between the second line and the third line is larger than the second reference value, and when it does not fall under "flexed", it is determined to be "unflexed", and based on this determination result, posture determination information indicating the change in the subject's posture over time is generated. Create .

[0013] For example, the motion to be analyzed is a kick-out motion in which the subject's posture changes from "bent" or "tucked" to "extended" during a forward or backward somersault, and in the motion analysis step, From the aforementioned two-dimensional coordinate information, the two-dimensional coordinates of a virtual body center point for identifying the height position of the subject's body are calculated, and based on the calculation results, a continuous function showing the time change in the height position of the body center point is derived, or a continuous function derivation process is performed to derive a continuous function showing the time change in the height position of a specific feature point from the aforementioned two-dimensional coordinate information. From the aforementioned two-dimensional coordinate information, a straight line connecting the hip joint and the knee joint, ankle joint, or shoulder joint is recognized, and the change in the position of the straight line over time is detected as a rotational movement with the hip joint as the origin, or a straight line connecting the shoulder joint and the hip joint, knee joint, or ankle joint is recognized, and the change in the position of the straight line over time is detected as a rotational movement with the shoulder joint as the origin, and based on this detection result, a rotational motion detection process is performed to detect the rotational movement of the subject in the forward or backward direction. Based on the aforementioned two-dimensional coordinate information, a first line connecting the hip joint and the shoulder joint, a second line connecting the hip joint and the knee joint, and a third line connecting the knee joint and the ankle joint are recognized. A posture determination process is performed in which the posture is determined to be "extended" when the intersection angle between the first line and the second line is greater than the first reference value, and the intersection angle between the second line and the third line is greater than the second reference value, and the posture is determined to be "not extended" when it does not fall under the category of "extended". The period from when the slope of the continuous function changes from a value less than or equal to zero to a positive value, until it changes from a negative value to a value greater than or equal to zero, is defined as the jump period. If the rotational movement of the subject is detected during the jump period, and the subject's posture changes from "non-extended" to "extended" during that rotational movement, it is determined that a "kick-out movement has occurred," and kick-out movement detection information is created.

[0014] For example, the motion to be analyzed is a down motion in which the subject's posture changes from "extended" to "unextended" during a jump and then lands in that position, and in the motion analysis step, From the aforementioned two-dimensional coordinate information, the two-dimensional coordinates of a virtual body center point for identifying the height position of the subject's body are calculated, and based on the calculation results, a continuous function showing the time change in the height position of the body center point is derived, or a continuous function derivation process is performed to derive a continuous function showing the time change in the height position of a specific feature point from the aforementioned two-dimensional coordinate information. Based on the aforementioned two-dimensional coordinate information, a first line connecting the hip joint and the shoulder joint, a second line connecting the hip joint and the knee joint, and a third line connecting the knee joint and the ankle joint are recognized. A posture determination process is performed in which the posture is determined to be "extended" when the intersection angle between the first line and the second line is greater than the first reference value, and the intersection angle between the second line and the third line is greater than the second reference value, and the posture is determined to be "not extended" when it does not fall under the category of "extended". The period from when the slope of the continuous function changes from a positive value to a negative value, until it changes from a negative value to a value greater than or equal to zero, is defined as the body descent period. If the subject's posture changes from "extended" to "not extended" during the body descent period, it is determined that a "downward movement has occurred," and downward movement detection information is created. In this case, if it is determined in the motion analysis step that a "downward movement has occurred," the intersection angle between the first straight line and the virtual horizontal line or virtual vertical line at the time the change in posture that forms the basis of this determination occurs can be calculated, and intersection angle information can be created based on this calculation result.

[0015] Furthermore, the present invention is a motion analysis program consisting of step execution programs for causing a computer to execute the above motion analysis method.

[0016] Furthermore, the present invention relates to a motion analysis system installed in a computer, comprising: a video data acquisition unit that acquires video data of a subject performing a motion to be analyzed, captured from the side; a two-dimensional coordinate information creation unit that extracts multiple feature points of the subject's body using a body feature point estimation model created by machine learning for each still image data constituting the video data, and creates two-dimensional coordinate information which is time-series data of the two-dimensional coordinates of the extracted feature points; and a motion analysis unit that detects the time change of the relative positional relationship between specific feature points by analyzing the two-dimensional coordinate information, and creates numerical information indicating the characteristics of the motion to be analyzed performed by the subject based on the detection results, wherein the feature points extracted using the body feature point estimation model include at least two points from the left and right shoulder joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joints.

[0017] For example, the operation to be analyzed is a rotational operation in which the body of the subject rotates forward or backward, and the operation analysis unit recognizes a straight line connecting the lumbar joint and the knee joint or the ankle joint or the shoulder joint from the two-dimensional coordinate information, and detects the temporal change in the position of the straight line as a movement rotating around the lumbar joint, or recognizes a straight line connecting the shoulder joint and the lumbar joint or the knee joint or the ankle joint from the two-dimensional coordinate information, and performs a process of detecting the temporal change in the position of the straight line as a movement rotating around the shoulder joint, and creates rotation count information or rotation speed information or both based on this detection result.

[0018] For example, the operation to be analyzed is an axis rotation operation in which the body of the subject rotates about an imaginary straight line passing through the upper body and the lower body, and the operation analysis unit recognizes the distance between the left and right lumbar joints from the two-dimensional coordinate information, and detects the temporal change in the distance, or recognizes the distance between the left and right shoulder joints from the two-dimensional coordinate information, and performs a process of detecting the temporal change in the distance, and creates axis rotation count information, axis rotation speed information or both based on this detection result.

[0019] for example When determining whether the posture of the subject performing the operation to be analyzed is either "stretched" or "not stretched", the operation analysis unit recognizes a first straight line connecting the lumbar joint and the shoulder joint, a second straight line connecting the lumbar joint and the knee joint, and a third straight line connecting the knee joint and the ankle joint from the two-dimensional coordinate information, and determines that it is "stretched" when the intersection angle between the first straight line and the second straight line is greater than a first reference value and the intersection angle between the second straight line and the third straight line is greater than a second reference value, and performs a process of determining "not stretched" when it does not correspond to "stretched", and based on this determination result, posture determination information indicating the temporal change in the posture of the subject Create .

[0020] for exampleWhen determining whether the posture of the subject performing the analysis target motion is either "hugging" or "not hugging", the motion analysis unit recognizes, from the two-dimensional coordinate information, a first straight line connecting the lumbar joint and the shoulder joint, a second straight line connecting the lumbar joint and the knee joint, and a third straight line connecting the knee joint and the ankle joint, and determines "hugging" when the intersection angle between the first straight line and the second straight line is smaller than a first reference value and the intersection angle between the second straight line and the third straight line is smaller than a second reference value, and determines "not hugging" when it does not correspond to "hugging". Based on this determination result, posture determination information indicating the temporal change of the subject's posture is Create .

[0021] for example When determining whether the posture of the subject performing the analysis target motion is either "bending" or "not bending", the motion analysis unit recognizes, from the two-dimensional coordinate information, a first straight line connecting the lumbar joint and the shoulder joint, a second straight line connecting the lumbar joint and the knee joint, and a third straight line connecting the knee joint and the ankle joint, and determines "bending" when the intersection angle between the first straight line and the second straight line is smaller than a first reference value and the intersection angle between the second straight line and the third straight line is larger than a second reference value, and determines "not bending" when it does not correspond to "bending". Based on this determination result, posture determination information indicating the temporal change of the subject's posture is Create .

[0022] For example, the analysis target motion is a kick-out motion in which the posture of the subject changes from "bending" or "hugging" to "stretching" during a forward or backward somersault, and the motion analysis unit calculates the two-dimensional coordinates of a virtual body center point for specifying the height position of the subject's body from the two-dimensional coordinate information, and derives a continuous function indicating the temporal change of the height position of the body center point based on the calculation result, or derives a continuous function indicating the temporal change of the height position of a specific one of the feature points from the two-dimensional coordinate information, and a continuous function derivation process From the aforementioned two-dimensional coordinate information, a straight line connecting the hip joint and the knee joint, ankle joint, or shoulder joint is recognized, and the change in the position of the straight line over time is detected as a rotational movement with the hip joint as the origin, or a straight line connecting the shoulder joint and the hip joint, knee joint, or ankle joint is recognized, and the change in the position of the straight line over time is detected as a rotational movement with the shoulder joint as the origin, and based on this detection result, a rotational motion detection process is performed to detect the rotational movement of the subject in the forward or backward direction. Based on the aforementioned two-dimensional coordinate information, a first line connecting the hip joint and the shoulder joint, a second line connecting the hip joint and the knee joint, and a third line connecting the knee joint and the ankle joint are recognized. A posture determination process is performed in which the posture is determined to be "extended" when the intersection angle between the first line and the second line is greater than the first reference value, and the intersection angle between the second line and the third line is greater than the second reference value, and the posture is determined to be "not extended" when it does not fall under the category of "extended". The period from when the slope of the continuous function changes from a value less than or equal to zero to a positive value, until it changes from a negative value to a value greater than or equal to zero, is defined as the jump period. If the rotational movement of the subject is detected during the jump period, and the subject's posture changes from "non-extended" to "extended" during that rotational movement, it is determined that a "kick-out movement has occurred," and kick-out movement detection information is created.

[0023] For example, the motion to be analyzed is a down motion in which the subject's posture changes from "extended" to "unextended" during a jump and then lands in that position, and the motion analysis unit, From the aforementioned two-dimensional coordinate information, the two-dimensional coordinates of a virtual body center point for identifying the height position of the subject's body are calculated, and based on the calculation results, a continuous function showing the time change in the height position of the body center point is derived, or a continuous function derivation process is performed to derive a continuous function showing the time change in the height position of a specific feature point from the aforementioned two-dimensional coordinate information. Based on the aforementioned two-dimensional coordinate information, a first line connecting the hip joint and the shoulder joint, a second line connecting the hip joint and the knee joint, and a third line connecting the knee joint and the ankle joint are recognized. A posture determination process is performed in which the posture is determined to be "extended" when the intersection angle between the first line and the second line is greater than the first reference value, and the intersection angle between the second line and the third line is greater than the second reference value, and the posture is determined to be "not extended" when it does not fall under the category of "extended". The period from when the slope of the continuous function changes from a positive value to a negative value, until it changes from a negative value to a value greater than or equal to zero, is defined as the body descent period. If the subject's posture changes from "extended" to "not extended" during the body descent period, it is determined that a "down movement has occurred," and down movement detection information is created. In this case, when the motion analysis unit determines that a "down movement has occurred," it can be configured to calculate the intersection angle between the first straight line and a virtual horizontal line or virtual vertical line at the time the change in posture that forms the basis of this determination occurs, and to create intersection angle information based on this calculation result. [Effects of the Invention]

[0024] The motion analysis method, motion analysis program, and motion analysis system of the present invention automatically extract two-dimensional coordinate information of multiple feature points on a subject's body based on video data (video data from a general camera) of a subject performing a motion to be analyzed. By analyzing the two-dimensional coordinate information, the system detects the time-dependent changes in the relative positions of the multiple feature points and, based on the detection results, creates numerical information that represents the characteristics of the motion to be analyzed performed by the subject. Therefore, it is possible to accurately analyze the motion to be analyzed performed by a subject and easily obtain numerical information that objectively represents the characteristics of that motion without using special motion observation equipment such as motion capture or 3D laser sensors. It is particularly well-suited for analyzing motions involving jumping and body rotation. [Brief explanation of the drawing]

[0025] [Figure 1]Figure (a) shows the setup of one embodiment of the motion analysis system of the present invention, and Figure (b) is a block diagram showing the system configuration of the motion analysis system of this embodiment. [Figure 2] Figure 1 shows a flowchart (a) illustrating the flow of rotational motion analysis performed by the motion analysis system [First Embodiment of the Motion Analysis Method of the Present Invention], and a diagram (b) illustrating the characteristics of the rotational motion analysis. [Figure 3] Figure 2(a) shows the body feature point estimation model used in the two-dimensional coordinate information creation step, and Figure 2(b) shows the correspondence between a human body captured as arbitrary still image data and multiple feature points of the body. [Figure 4] Figures (a) to (d) illustrate a method for detecting the rotation angle θ from the movement of the straight line connecting the hip joint and the knee joint, ankle joint, or shoulder joint when the subject performs a rotational movement. [Figure 5] Figures (a) to (d) show a method for detecting the rotation angle θ from the movement of the straight line connecting the shoulder joint and the knee joint, ankle joint, or hip joint when the subject performs a rotational movement. [Figure 6] This is a time chart showing an example of the time evolution of the sine function of the rotation angle θ detected in the motion analysis step in Figure 2(a). [Figure 7] Figure 1 shows a flowchart (a) illustrating the flow of rotational motion and posture analysis performed by the motion analysis system [Second Embodiment of the Motion Analysis Method of the Present Invention], and a diagram (b) illustrating the characteristics of rotational motion and posture analysis. [Figure 8] The diagrams illustrate methods for determining the posture of a subject, showing examples of postures judged as "extended" (a), "bent" (b), and "tucked" (c). [Figure 9] Figure 7(a) shows a time chart illustrating an example of the time evolution of the sine function of the rotation angle θ detected in the motion analysis step, and another time chart illustrating an example of the time evolution of the posture determination. [Figure 10]Figure 1 shows a flowchart (a) illustrating the flow of axial motion analysis performed by the motion analysis system [Third Embodiment of the Motion Analysis Method of the Present Invention], and a diagram (b) illustrating the characteristics of axial motion analysis. [Figure 11] Figure (a) shows the distance D between the left and right shoulder joints and the distance D between the lumbar joints when the front of the subject's body is facing upwards during axial movement, and Figure (b) shows the distance D between the left and right shoulder joints and the distance D between the lumbar joints when the subject is facing sideways. [Figure 12] This is a time chart showing an example of the time change of interval D detected in the motion analysis step of Figure 10(a). [Figure 13] Figure 1 shows a flowchart (a) illustrating the flow of the kick-out motion analysis performed by the motion analysis system [Fourth Embodiment of the Motion Analysis Method of the Present Invention], and a diagram (b) illustrating the characteristics of the kick-out motion analysis. [Figure 14] This figure shows two examples of methods for calculating the two-dimensional coordinates of the body's center point, which is the target of the continuous function derivation process performed in the motion analysis step of Figure 13. [Figure 15] Figure 13 shows a time chart illustrating an example of the continuous function (time change of the yp coordinate of the body's center point) derived in the continuous function derivation process of the motion analysis step, a time chart illustrating an example of the time change of the detection result of the rotational motion detection process, and a time chart illustrating an example of the time change of the judgment result of the posture determination process. [Figure 16] Figure 1 shows a flowchart (a) illustrating the flow of the down motion analysis performed by the motion analysis system [Fifth Embodiment of the Motion Analysis Method of the Present Invention], and a diagram (b) illustrating the characteristics of the down motion analysis. [Figure 17] Figure 16 shows a time chart illustrating an example of the continuous function (time change of the yp coordinate of the body's center point) derived in the continuous function derivation process of the motion analysis step, and a time chart illustrating an example of the time change of posture determination. [Figure 18] This figure shows the angle of intersection between the first straight line and a virtual vertical line or virtual horizontal line when the subject performs a downward motion. [Modes for carrying out the invention]

[0026] The following describes one embodiment of the motion analysis system of the present invention and the motion analysis method performed by this motion analysis system (the first to fifth embodiments of the motion analysis method of the present invention). The embodiments of the motion analysis program of the present invention are programs for executing each step to cause the motion analysis system to perform the above motion analysis method.

[0027] <Action analysis system 10 of one embodiment> One embodiment of the motion analysis system 10 analyzes motions performed (or occurring) during a trampoline competition routine, and analyzes video data of a subject HK performing on the trampoline bed 12a. As shown in Figure 1(a), the video data is captured from the side by a standard camera 14 placed near the trampoline 12, capturing the subject HK's movements.

[0028] The motion analysis system 10 is a system installed within a computer and, as shown in the block diagram in Figure 1(b), consists of a video data storage unit 10a, a video data acquisition unit 10b, a two-dimensional coordinate information creation unit 10c, a motion analysis unit 10d, and a display unit 10e.

[0029] The video data storage unit 10a is a block for storing video data captured by the camera 14 and for storing processing results and analysis results for each block, while the display unit 10e is a display for displaying video data and analysis results. These two blocks are not the main components of the present invention; the other main components of the present invention are the video data acquisition unit 10b, the two-dimensional coordinate information creation unit 10c, and the motion analysis unit 10d.

[0030] The video data acquisition unit 10b is a block that executes the video data acquisition steps S11(1) to S11(5) described later, and generally speaking, it is responsible for acquiring video data of a subject performing the action to be analyzed, captured from the side.

[0031] The two-dimensional coordinate information creation unit 10c is a block that executes the two-dimensional coordinate information creation steps S12(1) to S12(5) described later. In general terms, for each still image data that makes up the video data, it extracts multiple feature points of the subject HK's body using a body feature point estimation model created by machine learning, and creates two-dimensional coordinate information, which is time-series data of the two-dimensional coordinates of the extracted feature points.

[0032] The motion analysis unit 10d is a block that executes the motion analysis steps S13(1) to S13(5) described later. In general terms, it works by analyzing two-dimensional coordinate information to detect the time change in the relative positional relationship between specific feature points, and based on the detection results, it creates numerical information that shows the characteristics of the analysis target motion performed by the subject HK.

[0033] The video data acquisition unit 10b, the two-dimensional coordinate information creation unit 10c, and the motion analysis unit 10d perform various processes depending on the type of motion to be analyzed. These will be described in detail in the description of the first to fifth embodiments of the motion analysis method of the present invention.

[0034] <Method for analyzing motion according to the first embodiment (analysis of rotational motion)> First, the analysis of rotational motion, which is the first embodiment of the motion analysis method performed by the motion analysis system 10, will be explained based on Figures 2 to 6. Rotational motion refers to the movement in which the subject HK's body rotates forward or backward, and the motion analysis system 10 sequentially performs the video data acquisition step S11(1), the two-dimensional coordinate information creation step S12(1), and the motion analysis step S13(1) shown in Figure 2(a).

[0035] In the motion data acquisition step S11(1), video data is acquired by capturing images of the subject HK performing a rotational motion from the side. The video data may be extracted from the video data storage unit 10a, or video data transmitted from the camera 14 may be acquired in real time.

[0036] In the two-dimensional coordinate information creation step S12(1), as shown in Figures 3(a) and (b), first, for each still image data constituting the video data acquired in the motion data acquisition step S11(1), multiple feature points of the subject HK's body are extracted using a body feature point estimation model 16 created by machine learning. The body feature point estimation model 16 is a trained model created by machine learning based on multiple still image data SD of a person in various postures, in which multiple feature points of the human body are each labeled, and this is used as training data. The multiple feature points K** include at least two points from the left and right elbow joints K2L, K2R, left and right shoulder joints K3L, K3R, left and right hip joints K4L, K4R, left and right knee joints K5L, K5R, and left and right ankle joints K6L, K6R.

[0037] In the first embodiment, "feature points necessary for analyzing rotational motion" are extracted. As shown in Figure 2(b), there are six possible combinations of necessary feature points, broadly categorized into cases 1 to 6. Case 1 is the combination of "right hip joint K4R and right knee joint K5R," or the combination of "left hip joint K4L and left knee joint K5L." Regarding the right or left side, for example, if the acquired video data was captured from the right side of the subject HK, the former combination should be selected; if it was captured from the left side, the latter combination should be selected.

[0038] For example, when selecting the combination of "right hip joint K4R and right knee joint K5R" in case 1, the body feature point estimation model 16 is used to extract the body feature points "right hip joint K4R and right knee joint K5R" of subject HK for each still image data that makes up the video data. Then, two-dimensional coordinate information, which is time-series data of the two-dimensional coordinates of each extracted joint, is created for each.

[0039] Case 2 among the other combinations is the combination of "right hip joint K4R and right ankle joint K6R" or "left hip joint K4L and left ankle joint K6L". Case 3 is the combination of "right hip joint K4R and right shoulder joint K3R" or "left hip joint K4L and left shoulder joint K3L". Case 4 is the combination of "right shoulder joint K3R and right knee joint K5R" or "left shoulder joint K3L and left knee joint K5L". Case 5 is the combination of "right shoulder joint K3R and right ankle joint K6R" or "left shoulder joint K3L and left ankle joint K6L". And Case 6 is the combination of "right shoulder joint K3R and right hip joint K4R" or "left shoulder joint K3L and left hip joint K4L". Similarly, when selecting cases 2 to 6, for each still image data that makes up the video data, the corresponding feature points are extracted using the body feature point estimation model 16, and two-dimensional coordinate information, which is time-series data of the two-dimensional coordinates of the extracted feature points, is created for each.

[0040] In the motion analysis step S13(1), a straight line connecting two selected feature points is recognized from the two-dimensional coordinate information created in the two-dimensional coordinate information creation step S12(1), and the time change in the position of this straight line is detected as a rotational movement with one of the feature points as the origin.

[0041] Here, assuming the rotational movement shown in Figure 4(a) is performed, in case 1, the system recognizes a straight line connecting the right hip joint K4R and the right knee joint K5R, and detects the time change of the rotation angle θ as this line rotates with the right hip joint K4R as the origin. In this case, as shown in Figure 4(b), when the rotation angle θ0 at timing t0 is set to 0 degrees (reference), the rotation angle θ1 at timing t1 is detected as 90 degrees, and the rotation angle θ2 at timing t2 is detected as 180 degrees.

[0042] In case 2, a straight line connecting the right hip joint K4R and the right ankle joint K6R is recognized, and the time change of the rotation angle θ of this line, with the right hip joint K4R as the origin, is detected. In this case, as shown in Figure 4(c), when the rotation angle θ0 at timing t0 is set to 0 degrees (reference), the rotation angle θ1 at timing t1 is detected to be approximately 90 degrees, and the rotation angle θ2 at timing t2 is detected to be approximately 180 degrees.

[0043] In Case 3, the system recognizes a straight line connecting the right hip joint K4R and the right shoulder joint K3R, and detects the time change of the rotation angle θ as this line rotates with the right hip joint K4R as the origin. In this case, as shown in Figure 4(d), when the rotation angle θ0 at timing t0 is set to 0 degrees (reference), the rotation angle θ1 at timing t1 is detected to be approximately 90 degrees, and the rotation angle θ2 at timing t2 is detected to be approximately 180 degrees.

[0044] Thus, cases 1 to 3 are methods that detect the time change of the rotation angle θ of each line with the right lumbar joint K4R as the origin, and almost the same detection results are obtained.

[0045] Furthermore, assuming the rotational movement shown in Figure 5(a), i.e., the same rotational movement as in Figure 4(a), in case 4, the system recognizes the straight line connecting the right shoulder joint K3R and the right knee joint K5R, and detects the time change of the rotation angle θ as this straight line rotates with the right shoulder joint K3R as the origin. In this case, as shown in Figure 5(b), when the rotation angle θ0 at timing t0 is set to 0 degrees (reference), the rotation angle θ1 at timing t1 is detected as 90 degrees, and the rotation angle θ2 at timing t2 is detected as 180 degrees.

[0046] In case 5, the system recognizes a straight line connecting the right shoulder joint K3R and the right ankle joint K6R, and detects the time change of the rotation angle θ as this line rotates with the right shoulder joint K3R as the origin. In this case, as shown in Figure 5(c), when the rotation angle θ0 at timing t0 is set to 0 degrees (reference), the rotation angle θ1 at timing t1 is detected to be approximately 90 degrees, and the rotation angle θ2 at timing t2 is detected to be approximately 180 degrees.

[0047] In Case 6, the system recognizes a straight line connecting the right shoulder joint K3R and the right hip joint K4R, and detects the time change of the rotation angle θ as this line rotates with the right shoulder joint K3R as the origin. In this case, as shown in Figure 5(d), when the rotation angle θ0 at timing t0 is set to 0 degrees (reference), the rotation angle θ1 at timing t1 is detected to be approximately 90 degrees, and the rotation angle θ2 at timing t2 is detected to be approximately 180 degrees.

[0048] Thus, cases 4 to 6 are methods that detect the time change of the rotation angle θ of each line with the right shoulder joint K3R as the origin, and almost the same detection results are obtained.

[0049] The time chart in Figure 6 shows an example of the time evolution of the sine function sinθ of the rotation angle θ detected by the method in case 3. From this time chart, it can be seen that there are 3 rotations in jump 1, and 2 rotations each in the following jumps 2 and 3. Also, since the horizontal axis is time, the rotation speed can be easily calculated.

[0050] The output of the motion analysis step S13(1) is rotation count information, rotation speed information, or both, as shown in Figure 2(b). The rotation count information can be any information that allows recognition of the number of rotations, and the unit can be [rotations] or [deg (degrees)], etc. Similarly, the rotation speed information can be any information that allows recognition of the speed of rotation, and the unit can be [rotations / hour] or [deg / hour], etc.

[0051] As described above, according to the first embodiment of the motion analysis method performed by the motion analysis system 10, the rotational movement performed by the subject HK can be accurately analyzed using video data captured by a general-purpose camera 14, and numerical information that objectively represents the characteristics of that rotational movement can be easily obtained.

[0052] When analyzing rotational movements, the choice of which method from case 1 to case 6 to select should be considered in accordance with the subject HK's habits and skill level when performing rotational movements. Also, if the subject HK's posture changes during the rotational movement (for example, transitioning from flexion to extension, or spreading both feet to the sides), there is a possibility of variability in the detection results in case 1 to case 6. However, if a method such as case 3 is chosen, which detects the rotation angle θ from the movement of the straight line connecting the hip joint and the shoulder joint, the variability in the detection of the rotation angle θ due to changes in posture tends to be smaller.

[0053] Furthermore, while the above explanation assumes that the rotation angle θ is detected by focusing only on the joints of the right half of the body (or the joints of the left half), in order to absorb detection variability, it may be possible to perform detection focusing on the joints of both the right and left halves of the body and comprehensively evaluate the results of both detections. Alternatively, the rotational motion may be analyzed based on the movement of a straight line connecting the midpoints of joints at the same position on both sides (for example, a straight line connecting the midpoints of a pair of shoulder joints K3L, K3R and the midpoints of a pair of hip joints K4L, K4R).

[0054] <Action analysis method according to the second embodiment (analysis of rotational motion and posture)> Next, the second embodiment of the motion analysis method performed by the motion analysis system 10, which is the analysis of rotational motion and posture, will be explained based on Figures 7 to 9. The analysis of rotational motion is, as described above, the analysis of the subject HK's body rotating in the forward or backward direction, and the analysis of posture is the determination of whether it is "extended," "bent," or "tucked." The motion analysis system 10 sequentially performs the video data acquisition step S11(2), the two-dimensional coordinate information creation step S12(2), and the motion analysis step S13(2) shown in Figure 7(a).

[0055] In the motion data acquisition step S11(2), video data is acquired by capturing images of the subject performing a rotational movement from the side. This is the same as the motion data acquisition step S11(1) described above.

[0056] In the two-dimensional coordinate information creation step S12(2), first, for each still image data constituting the video data acquired in the motion data acquisition step S11(2), multiple feature points of the subject HK's body are extracted using the body feature point estimation model 16. This is basically the same as the method shown in Figures 3(a) and (b), but in the second embodiment, "feature points necessary for the analysis of rotational motion and posture" are extracted.

[0057] The required combination of feature points is, as shown in Figure 7(b), either the combination of "right shoulder joint K3R, right hip joint K4R, right knee joint K5R, and right ankle joint K6R," or the combination of "left shoulder joint K3L, left hip joint K4L, left knee joint K5L, and left ankle joint K6L." Regarding whether to use the right or left side, for example, if the acquired video data was captured from the right side of the subject HK, the former combination should be selected; if it was captured from the left side, the latter combination should be selected.

[0058] When selecting the former combination of "right shoulder joint K3R, right hip joint K4R, right knee joint K5R, and right ankle joint K6R," the physical feature points of subject HK, "right shoulder joint K3R, right hip joint K4R, right knee joint K5R, and right ankle joint K6R," are extracted for each still image data that makes up the video data using the body feature point estimation model 16. Then, two-dimensional coordinate information, which is time-series data of the two-dimensional coordinates of each extracted joint, is created for each.

[0059] In the motion analysis step S13(2), similar to the motion analysis step S13(1) above, a straight line connecting two selected feature points is recognized from the two-dimensional coordinate information created in the two-dimensional coordinate information creation step S12(2), and the time change in the position of this straight line is detected as a rotational movement with one of the feature points as the origin, that is, a time change in the rotation angle θ is detected. Any of the methods from case1 to case6 may be used here.

[0060] In parallel with the process of detecting the time change of the rotation angle θ, posture analysis is performed. First, from the two-dimensional coordinate information, the first line CY1 connecting the right hip joint K4R and the right shoulder joint K3R, the second line CY2 connecting the right hip joint K4R and the right knee joint K5R, and the third line CY3 connecting the right knee joint K5R and the right ankle joint K6R are recognized, and the intersection angle α between the first line CY1 and the second line CY2, and the intersection angle β between the second line CY2 and the third line CY3 are calculated. Then, the posture is determined by comparing the intersection angles α,β with the first and second reference values ​​αr,βr (for example, αr=βr=135deg).

[0061] Specifically, as shown in Figure 8(a), when the cross angle α > αr and the cross angle β > βr, it is determined to be "extended," as shown in Figure 8(b), when the cross angle α < αr and the cross angle β > βr, it is determined to be "flexed," and as shown in Figure 8(c), when the cross angle α < αr and the cross angle β < βr, it is determined to be "tucked in." Based on these determination results, posture determination information showing the time change in the posture of the subject HK is created.

[0062] The upper time chart in Figure 9 shows an example of the time change of the sine function sinθ of the rotation angle θ (the same as in Figure 6), while the lower time chart shows the time change of the posture judgment. From this time chart, it can be seen that in jump 1, three rotational movements were performed in a "tucked" posture, in the next jump 2, two rotational movements were performed in a "bent" posture, and in the following jump 3, two rotational movements were performed in an "extended" posture.

[0063] The output of the motion analysis step S13(2) is rotation count information, rotation speed information, and posture determination information, as shown in Figure 7(b). The posture determination information can be anything that shows the change in posture over time; for example, it can be a time chart with the horizontal axis representing time and showing the change in posture, as in the lower part of Figure 9, or it can be a posture determination result for each jump or rotational movement.

[0064] As described above, in the second embodiment of the motion analysis method performed by the motion analysis system 10, video data captured by a general-purpose camera 14 of the subject HK is used to accurately analyze the rotational movements and postures performed by the subject HK, and numerical information that objectively represents the characteristics of those rotational movements and postures can be easily obtained.

[0065] Regarding the postural analysis in the motion analysis step S13(2), the above explanation states that the crossing angles α and β are detected by focusing only on the joints of the right half of the body (or the joints of the left half). However, in order to absorb the variability in detection, it is also possible to comprehensively evaluate the posture using both the crossing angles α and β calculated by focusing on the joints of the right half of the body and the crossing angles α and β calculated by focusing on the joints of the left half of the body.

[0066] Furthermore, the posture analysis methods described so far can be applied to movements other than rotational movements, making them highly versatile. For example, they can be used to analyze posture during axial movements, which will be discussed later. In addition, as demonstrated in the analysis of kick-out movements later, it is possible to determine whether the posture is "extended" or "not extended," "flexed" or "not flexed," and whether the posture is "tucked" or "not tucked."

[0067] <Method for analyzing motion according to the third embodiment (analysis of axial motion)> Next, the analysis of axial motion, which is a third embodiment of the motion analysis method performed by the motion analysis system 10, will be explained based on Figures 10 to 12. Axial motion refers to the movement in which the subject HK's body rotates around a virtual straight line 18 that passes through the upper and lower body. The motion analysis system 10 sequentially performs the video data acquisition step S11(3), the two-dimensional coordinate information creation step S12(3), and the motion analysis step S13(3) shown in Figure 10(a).

[0068] In the motion data acquisition step S11(3), video data is acquired by capturing images of the subject HK performing axial movements from the side.

[0069] In the two-dimensional coordinate information creation step S12(3), first, for each still image data that makes up the video data acquired in the motion data acquisition step S11(3), multiple feature points of the subject HK's body are extracted using the body feature point estimation model 16. This is basically the same as the method shown in Figures 3(a) and (b), but in the third embodiment, "feature points necessary for analyzing axial motion" are extracted.

[0070] As shown in Figure 10(b), there are two possible combinations of feature points: case 1 and case 2. Case 1 is the combination of "right shoulder joint K3R and left shoulder joint K3L," and case 2 is the combination of "right lumbar joint K4R and left lumbar joint K4L."

[0071] For example, when selecting the combination of "right shoulder joint K3R and left shoulder joint K3L" in case 1, the physical feature points "right shoulder joint K3R and left shoulder joint K3L" of subject HK are extracted for each still image data that makes up the video data using the physical feature point estimation model 16. Then, two-dimensional coordinate information, which is time-series data of the two-dimensional coordinates of each extracted joint, is created for each.

[0072] Furthermore, when selecting the combination of "right hip joint K4R and left hip joint K4L" in case 2, the physical feature points "right hip joint K4R and left hip joint K4L" of subject HK are extracted for each still image data that makes up the video data using the physical feature point estimation model 16. Then, two-dimensional coordinate information, which is time-series data of the two-dimensional coordinates of each extracted joint, is created for each.

[0073] In the motion analysis step S13(3), the interval D between two selected feature points is recognized from the two-dimensional coordinate information created in the two-dimensional coordinate information creation step S12(3), and a process is performed to detect the time change of the interval D.

[0074] Here, assuming the axial movement shown in Figures 11(a) and (b) is performed, in case 1, at timing ta, when the front of the subject HK's body is facing upward, the interval D (distance between the right shoulder joint K3R and the left shoulder joint K3L) becomes relatively smaller, and at timing tb, when facing sideways, the interval D becomes relatively larger. Similarly in case 2, at timing ta, when facing upward, the interval D (distance between the right lumbar joint K4R and the left lumbar joint K4L) becomes relatively smaller, and at timing tb, when facing sideways, the interval D becomes relatively larger.

[0075] The time chart in Figure 12 shows an example of the time change of interval D detected by the method in case 2. From this time chart, it can be seen that the number of axis rotations is 0.5 in jump 1, 1.5 in the next jump 2, and 1.0 in the next jump 3. Also, since the horizontal axis is time, the axis rotation velocity can be easily calculated.

[0076] The output of the motion analysis step S13(3) is, as shown in Figure 10(b), rotational count information, rotational velocity information, or both. Rotational count information only needs to be information that can recognize the number of rotations, and the unit can be [rotations] or [deg (degrees)]. Similarly, rotational velocity information only needs to be information that can recognize the speed of rotation, and the unit can be [rotations / hour] or [deg / hour].

[0077] As described above, in the third embodiment of the motion analysis method performed by the motion analysis system 10, video data captured by a general-purpose camera 14 of the subject HK is used to accurately analyze the axial movements performed by the subject HK, and numerical information that objectively represents the characteristics of those axial movements can be easily obtained.

[0078] Furthermore, if the subject HK's posture changes during rotation (for example, from extended to non-extended, or if the upper and lower body twists), there is a possibility of variability in the detection results between case 1 and case 2. Therefore, although the above explanation states that the rotation angle θ is detected only at the shoulder joints K3R, K3L or only at the hip joints K4R, K4L, to absorb detection variability, it may be possible to perform detection using both the shoulder joints K3R, K3L and the hip joints K4R, K4L, and comprehensively evaluate the results of both detections.

[0079] <Fourth Embodiment of Motion Analysis Method (Analysis of Kick-Out Motion)> Next, the analysis of a kick-out motion, which is the fourth embodiment of the motion analysis method performed by the motion analysis system 10, will be explained based on Figures 13 to 15. A kick-out motion is a movement in which the posture of the subject HK changes from "bent" or "tucked" to "extended" during a forward or backward somersault. The motion analysis system 10 sequentially performs the video data acquisition step S11(4), the two-dimensional coordinate information creation step S12(4), and the motion analysis step S13(4) shown in Figure 13(a).

[0080] In the motion data acquisition step S11(4), video data is acquired by capturing images from the side of the subject HK performing a series of movements such as a somersault.

[0081] In the two-dimensional coordinate information creation step S12(4), first, for each still image data constituting the video data acquired in the motion data acquisition step S11(4), multiple feature points of the subject HK's body are extracted using the body feature point estimation model 16. This is basically the same as the method shown in Figures 3(a) and (b), but in the fourth embodiment, "feature points necessary for the analysis of the kick-out motion" are extracted.

[0082] Here, as shown in Figure 13(b), 12 feature points are extracted: "left and right wrist joints K1L, K1R, left and right elbow joints K2L, K2R, left and right shoulder joints K3L, K3R, left and right hip joints K4L, K4R, left and right knee joints K5L, K5R, and left and right ankle joints K6L, K6R". Then, two-dimensional coordinate information, which is time-series data of the two-dimensional coordinates of each extracted joint, is created.

[0083] In the motion analysis step S13(4), the continuous function derivation process SR11, the rotational motion detection process SR12, and the posture determination process SR13 are performed in parallel.

[0084] The continuous function derivation process SR11 is a process that calculates the two-dimensional coordinates of a virtual body center point Kp to identify the height position of the subject HK's body from the two-dimensional coordinate information created in the two-dimensional coordinate information creation step S12(4), and derives a continuous function RK that shows the time change in the height position of the body center point Kp based on the calculation result.

[0085] As shown in Figure 14, if the horizontal axis of the two-dimensional coordinate system is the X-axis and the vertical axis is the Y-axis, and the two-dimensional coordinates of the 12 feature points are (x1, y1), (x2, y2), ... (x12, y12), then the two-dimensional coordinates (xp, yp) of the body center point Kp can be calculated, for example, using the formula shown in Example 1 in the figure, to set the centroid of the 12 feature points as the body center point Kp. Alternatively, using the formula shown in Example 2 in the figure, the center of the circumscribing rectangle of the 12 feature points can be set as the body center point Kp. How to set the body center point Kp should be determined appropriately depending on the situation.

[0086] Furthermore, when deriving the continuous function RK, it is preferable to take a moving average of the change in the yp coordinate with respect to time and interpolate missing values ​​using splines. For example, while Lagrangian interpolation may reduce the accuracy of the approximation as the number of missing values ​​increases, spline interpolation ensures the accuracy of the approximation even when the number of missing values ​​increases.

[0087] The rotation motion detection process SR12 is similar to the first embodiment of the motion analysis method described above (analysis of rotation motion). In other words, it recognizes a straight line connecting two specific feature points from the two-dimensional coordinate information created in the two-dimensional coordinate information creation step S12(4), and detects the time change in the position of this straight line as a rotational movement with one of the feature points as the origin, that is, it detects the time change in the rotation angle θ. Here, any of the methods from case1 to case6 shown in Figure 2(b) may be used.

[0088] However, in the case of rotational motion detection process SR12, the purpose is to detect that rotational motion has occurred, and it is unnecessary to create rotation count information or rotation speed information. Therefore, in rotational motion detection process SR12, for example, when the number of rotations is less than 1 / 4 of a rotation (θ < 90 deg), it is determined that "no rotational motion has been detected," and when the number of rotations is 1 / 4 of a rotation or more (θ ≥ 90 deg), it is determined that "rotational motion has been detected." The threshold for determination can be changed as appropriate depending on the situation.

[0089] The posture determination process SR13 is similar to the posture analysis in the second embodiment (rotational motion and posture analysis) of the motion analysis method described above. In other words, if we focus only on the joints of the right half of the body, the two-dimensional coordinate information created in the two-dimensional coordinate information creation step S12(4) recognizes a first straight line CY1 connecting the right hip joint K4R and the right shoulder joint K3R, a second straight line CY2 connecting the right hip joint K4R and the right knee joint K5R, and a third straight line CY3 connecting the right knee joint K5R and the right ankle joint K6R. The intersection angle α between the first straight line CY1 and the second straight line CY2, and the intersection angle β between the second straight line CY2 and the third straight line CY3 are calculated. Then, the posture is determined by comparing the intersection angles α and β with the first reference value αr and the second reference value βr.

[0090] However, in the case of posture determination process SR13, the purpose is to determine whether the posture is "extended" or "not extended," and there is no need to distinguish between "flexed" and "tucked." Therefore, as shown in Figure 8(a), posture determination process SR13 determines that the posture is "extended" when the intersection angle α > αr and the intersection angle β > βr, and determines that the posture is "not extended" when it does not fall under the category of "extended."

[0091] The output of the motion analysis step S13(4) is kick-out motion detection information, as shown in Figure 13(b). The conditions and procedure for creating kick-out motion detection information are described below based on Figure 15. The upper time chart in Figure 15 is an example of the continuous function RK (time change of the yp coordinate of the body center point Kp) derived in the continuous function derivation process SR11, the middle time chart is an example of the time change of the detection result of the rotation motion detection process SR12, and the lower time chart shows an example of the time change of the judgment result of the posture judgment process SR13.

[0092] In the motion analysis step S13(4), the period from when the slope of the continuous function RK changes from a negative value (or zero) to a positive value until it changes from a negative value to a positive value (or zero) again is defined as the jump period. If a rotational movement of the subject HK is detected during the jump period, and the posture of the subject HK changes from "non-extended" to "extended" during that rotational movement, it is determined that a "kick-out movement has occurred," and kick-out movement detection information is created.

[0093] For example, during the jump period T2, a rotational movement (somersault) is detected, and the posture changes from "non-extended" to "extended" during the somersault. Therefore, it is determined that a "kick-out movement occurred" when this change in posture occurred. On the other hand, no somersault occurred during the jump period T1, and during the jump period T3, the somersault was performed while maintaining an "extended" posture, so it is not determined that a "kick-out movement occurred."

[0094] The kick-out action detection information may be a flag indicating that a kick-out action has been detected, or it may include additional information indicating the time and timing of the kick-out action detection.

[0095] As described above, in the fourth embodiment of the motion analysis method performed by the motion analysis system 10, video data captured by a general-purpose camera 14 of the subject HK is used to accurately detect the kick-out action performed by the subject HK, and objective numerical information indicating that the kick-out action has been detected can be easily obtained.

[0096] In step S12(4) of creating two-dimensional coordinate information, 12 feature points are extracted as shown in Figure 13(b). If you want to identify the body center point Kp with higher accuracy, you can add more feature points. On the other hand, the left and right wrist joints K1L, K1R and the left and right elbow joints K2L, K2R among the 12 feature points are used in the continuous function derivation process SR11 to improve the accuracy of the body center point Kp and are not used in other analyses. Therefore, if you want to prioritize speeding up processing over increasing the accuracy of the body center point Kp, you can reduce the number of feature points to the extent that it does not affect other analyses.

[0097] For example, if the midpoint of a line connecting two specific feature points (e.g., the left hip joint K4L and the right hip joint K4R) is used as the body center point Kp, the processing time for the two-dimensional coordinate information creation step S12(4) and the continuous function derivation process SR11 can be significantly reduced. Alternatively, it is also possible to use a single feature point located approximately in the center of the body (e.g., the left hip joint K4 or the right hip joint K4R) directly as the body center point Kp.

[0098] <Fifth Embodiment of Motion Analysis Method (Analysis of Down Motion)> Next, the fifth embodiment of the motion analysis method performed by the motion analysis system 10, which is the analysis of the down motion, will be explained based on Figures 16 to 18. The down motion is the movement in which the posture of the subject HK changes from "extended" to "unextended" during the jump and then lands on the ground. The motion analysis system 10 sequentially performs the video data acquisition step S11(5), the two-dimensional coordinate information creation step S12(5), and the motion analysis step S13(5) shown in Figure 16(a).

[0099] In the motion data acquisition step S11(5), video data is acquired by capturing images from the side of the subject HK as he jumps and performs a series of movements.

[0100] In the two-dimensional coordinate information creation step S12(5), first, for each still image data constituting the video data acquired in the motion data acquisition step S11(5), multiple feature points of the subject HK's body are extracted using the body feature point estimation model 16. This is basically the same as the method shown in Figures 3(a) and (b), but in the fifth embodiment, "feature points necessary for the analysis of down movements" are extracted.

[0101] Here, as shown in Figure 16(b), 12 feature points are extracted: "left and right wrist joints K1L, K1R, left and right elbow joints K2L, K2R, left and right shoulder joints K3L, K3R, left and right hip joints K4L, K4R, left and right knee joints K5L, K5R, and left and right ankle joints K6L, K6R". These 12 feature points are the same as the feature points extracted in the analysis of the kick-out motion. Then, two-dimensional coordinate information, which is time-series data of the two-dimensional coordinates of each extracted joint, is created.

[0102] In the motion analysis step S13(5), the continuous function derivation process SR11 and the posture determination process SR13 are performed in parallel, similar to the process used for analyzing the kick-out motion. The rotation motion detection process SR12 is not required.

[0103] The output of the motion analysis step S13(5) is down motion detection information and intersection angle information, as shown in Figure 16(b). The conditions and procedures for creating the down motion detection information and intersection angle information will be explained below based on Figure 17. The upper time chart in Figure 17 is an example of the continuous function RK (time change of the yp coordinate of the body center point Kp) derived in the continuous function derivation process SR11, and the lower time chart shows an example of the time change of the judgment result of the posture judgment process SR13.

[0104] In the motion analysis step S13(5), the period from when the slope of the continuous function RK changes from a positive value to a negative value, until it changes from a negative value to a positive value (or zero), is defined as the body descent period. If the posture of the subject HK changes from "extended" to "unextended" during the body descent period, it is determined that a "downward movement has occurred," and downward movement detection information and crossing angle information are created.

[0105] For example, during the body descent period Ta, the posture changes from "extended" to "non-extended" before implantation, so when this change in posture occurs, it is determined that "the kick-out movement has occurred." The same applies to the body descent periods Tb and Tc.

[0106] The down operation detection information may be a flag indicating that a down operation has been detected, or it may include additional information indicating the time and timing of the down operation detection.

[0107] The intersection angle information, as shown in Figure 18, indicates the angle of intersection between the first straight line CY1 and the virtual vertical line 20 or virtual horizontal line 22 when a change in posture occurs that serves as the basis for determining that a "down movement has occurred." For example, the intersection angle φa in the figure is the angle of intersection between the first straight line CY1 and the virtual vertical line 20, and is generally called the down angle. The down angle is one of the indicators used in scoring trampoline competitions, and the score changes depending on the magnitude of the down angle. Therefore, the down angle is an indicator that should be checked not only when scoring a performance, but also when practicing a performance.

[0108] Furthermore, the intersection angle φb, like the intersection angle φa, is the intersection angle between the first straight line CY1 and the virtual vertical line 20, and the down angle can be easily obtained by subtracting φb from 180deg. Similarly, the intersection angle φc is the intersection angle between the first straight line CY1 and the virtual horizontal line 22, and the down angle can be easily obtained by adding 90deg to φc. Therefore, the intersection angle information may represent the calculated result of the intersection angle φa, φb, or φc as is, or it may represent the value converted to the down angle.

[0109] As described above, in the fifth embodiment of the motion analysis method performed by the motion analysis system 10, video data captured by a general-purpose camera 14 of the subject HK is used to accurately detect the downward movement performed by the subject HK, and numerical information objectively representing the detection of the downward movement and its characteristics can be easily obtained.

[0110] <Other embodiments, modifications, etc.> It should be noted that the motion analysis method, motion analysis program, and motion analysis system of the present invention are not limited to the embodiments described above. Each of the embodiments described above analyzes four movements performed in trampoline competitions (rotation, axial movement, kick-out movement, and down movement) and is configured to analyze each movement individually. However, it is also possible to combine these to analyze the four movements simultaneously, which allows for quick and accurate analysis of the individual movements that make up a complex routine, such as a "double somersault with one twist in a tucked position." In addition, it is also possible to analyze movements other than the four movements described above (rotation, axial movement, kick-out movement, and down movement).

[0111] The movements analyzed by this invention are not limited to those performed in trampoline competitions. For example, the rotational movements described above are also performed in gymnastics, dance, and other performing arts, and the above methods for analyzing rotational movements and postures can be applied to these. Furthermore, the axial rotational movements described above are also performed in gymnastics and figure skating, and the above methods for analyzing axial rotational movements and postures can be applied to these as well.

[0112] Furthermore, kick-out and down movements are similar to those performed in gymnastics, for example, and the analysis methods for kick-out and down movements described above can be applied. However, there is a difference in that the floor surface (mat 12a) in trampoline competitions deforms elastically, whereas in gymnastics and similar competitions the floor surface hardly deforms at all. Therefore, when applying this to gymnastics and similar competitions, it is preferable to slightly modify the method of recognizing the jumping period and the body descent period.

[0113] To explain in more detail, in gymnastics and similar sports, the floor surface hardly deforms, so from the time the subject HK jumps and lands until the next jump and leaves the floor, the height position (yp coordinate) of the body's center point Kp remains almost constant. Therefore, during this period, the slope of the continuous function RK (the slope of yp's change over time) is kept at zero. For this reason, when analyzing the kick-out motion in gymnastics and similar sports, the jumping period is preferably defined as "the period from when the slope of the continuous function changes from a value less than or equal to zero to a positive value, and then from a negative value to a value greater than or equal to zero." Similarly, when analyzing the down motion in gymnastics and similar sports, the body descent period is preferably defined as "the period from when the slope of the continuous function changes from a positive value to a negative value, and then from a negative value to a value greater than or equal to zero."

[0114] The motion analysis system 10 described above is configured to connect a camera 14 externally as needed, but the camera may also be provided as part of the system. Furthermore, although the motion analysis system 10 has a display unit 10e such as a display, the display unit 12e may be omitted, and a dedicated external display device may be connected for display. Additionally, the motion analysis unit 10d may transmit the information it generates to external devices (such as a printer or management server) via a communication line. [Explanation of Symbols]

[0115] 10. Video analysis device 10b Video data acquisition unit 10c Two-dimensional coordinate information creation unit 10d Motion analysis section 16. Models for Estimating Physical Characteristics 18. Virtual straight line (axis of rotation for motion around an axis) 20 Virtual vertical line 22 Virtual Horizon CY1 First straight line CY2 Second straight line CY3 third straight line HK Subject Joints (characteristic points) of each part: K1L~K6L, K1R~K6R Kp body center point S11(1)~S11(5) Video data acquisition steps S12(1)~S12(5) Steps for creating two-dimensional coordinate data S13(1)~S13(5) Video Analysis Steps α is the angle of intersection between the first and second lines. αr First Reference Value β is the angle of intersection between the second and third lines. βr Second Reference Value θ Rotation angle φa, φb are the angles of intersection between the first straight line and the virtual vertical line. φc is the angle of intersection between the first straight line and the virtual horizontal line.

Claims

1. A computer-based method for analyzing computer behavior, A video data acquisition step involves acquiring video data by capturing images of the subject performing the action to be analyzed from the side, and For each still image data constituting the video data, a two-dimensional coordinate information creation step is performed, in which a body feature point estimation model created by machine learning is used to extract multiple feature points of the subject's body, and two-dimensional coordinate information is created, which is time-series data of the two-dimensional coordinates of the extracted feature points. The motion analysis step includes detecting the time change in the relative positional relationship between specific feature points by analyzing the two-dimensional coordinate information, and creating numerical information that indicates the characteristics of the action to be analyzed performed by the subject based on the detection results. The feature points extracted using the aforementioned physical feature point estimation model include at least two points from among the left and right shoulder joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joints. The motion to be analyzed is a rotational motion in which the subject's body rotates forward or backward. In the aforementioned motion analysis step, From the aforementioned two-dimensional coordinate information, a straight line connecting the hip joint and the knee joint, ankle joint, or shoulder joint is recognized, and the time change in the position of the straight line is detected as a rotational movement with the hip joint as the origin. Alternatively, the system recognizes a straight line connecting the shoulder joint and the hip joint, knee joint, or ankle joint from the two-dimensional coordinate information, and processes the change in the position of the straight line over time as a rotational movement with the shoulder joint as the origin. A motion analysis method characterized by generating rotation count information, rotation speed information, or both, based on these detection results.

2. A computer-based method for analyzing computer behavior, A video data acquisition step involves acquiring video data by capturing images of the subject performing the action to be analyzed from the side, and For each still image data constituting the video data, a two-dimensional coordinate information creation step is performed, in which a body feature point estimation model created by machine learning is used to extract multiple feature points of the subject's body, and two-dimensional coordinate information is created, which is time-series data of the two-dimensional coordinates of the extracted feature points. The motion analysis step includes detecting the time change in the relative positional relationship between specific feature points by analyzing the two-dimensional coordinate information, and creating numerical information that indicates the characteristics of the action to be analyzed performed by the subject based on the detection results. The feature points extracted using the aforementioned physical feature point estimation model include at least two points from among the left and right shoulder joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joints. The motion to be analyzed is an axial motion in which the subject's body rotates around a virtual straight line passing through the upper and lower body. In the aforementioned motion analysis step, A process that recognizes the distance between the left and right hip joints from the aforementioned two-dimensional coordinate information and detects the time change of said distance. Alternatively, the system recognizes the distance between the left and right shoulder joints from the two-dimensional coordinate information and performs a process to detect the time change of said distance. A motion analysis method characterized by generating information on the number of rotations around an axis, information on the speed around an axis, or both, based on these detection results.

3. A computer-based method for analyzing computer behavior, A video data acquisition step involves acquiring video data by capturing images of the subject performing the action to be analyzed from the side, and For each still image data constituting the video data, a two-dimensional coordinate information creation step is performed, in which a body feature point estimation model created by machine learning is used to extract multiple feature points of the subject's body, and two-dimensional coordinate information is created, which is time-series data of the two-dimensional coordinates of the extracted feature points. The motion analysis step includes detecting the time change in the relative positional relationship between specific feature points by analyzing the two-dimensional coordinate information, and creating numerical information that indicates the characteristics of the action to be analyzed performed by the subject based on the detection results. The feature points extracted using the aforementioned physical feature point estimation model include at least two points from among the left and right shoulder joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joints. When determining whether the posture of the subject performing the aforementioned action to be analyzed is "extended" or "not extended", In the aforementioned motion analysis step, From the aforementioned two-dimensional coordinate information, a first straight line connecting the hip joint and the shoulder joint, a second straight line connecting the hip joint and the knee joint, and a third straight line connecting the knee joint and the ankle joint are recognized. The process is performed to determine "extended posture" when the intersection angle between the first line and the second line is greater than the first reference value, and the intersection angle between the second line and the third line is greater than the second reference value, and to determine "not extended posture" when it does not fall under the category of "extended posture". A motion analysis method characterized by creating posture judgment information that shows the change in the subject's posture over time, based on this judgment result.

4. A computer-based method for analyzing computer behavior, A video data acquisition step involves acquiring video data by capturing images of the subject performing the action to be analyzed from the side, and For each still image data constituting the video data, a two-dimensional coordinate information creation step is performed, in which a body feature point estimation model created by machine learning is used to extract multiple feature points of the subject's body, and two-dimensional coordinate information is created, which is time-series data of the two-dimensional coordinates of the extracted feature points. The motion analysis step includes detecting the time change in the relative positional relationship between specific feature points by analyzing the two-dimensional coordinate information, and creating numerical information that indicates the characteristics of the action to be analyzed performed by the subject based on the detection results. The feature points extracted using the aforementioned physical feature point estimation model include at least two points from among the left and right shoulder joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joints. When determining whether the posture of the subject performing the aforementioned action to be analyzed is "hugging" or "not hugging", In the aforementioned motion analysis step, From the aforementioned two-dimensional coordinate information, a first straight line connecting the hip joint and the shoulder joint, a second straight line connecting the hip joint and the knee joint, and a third straight line connecting the knee joint and the ankle joint are recognized. The process is performed to determine if "encompassing" occurs when the intersection angle between the first line and the second line is smaller than the first reference value, and the intersection angle between the second line and the third line is smaller than the second reference value, and if it does not fall under "encompassing", it is determined to be "not encompassing". A motion analysis method characterized by creating posture judgment information that shows the change in the subject's posture over time, based on this judgment result.

5. A computer-based method for analyzing computer behavior, A video data acquisition step involves acquiring video data by capturing images of the subject performing the action to be analyzed from the side, and For each still image data constituting the video data, a two-dimensional coordinate information creation step is performed, in which a body feature point estimation model created by machine learning is used to extract multiple feature points of the subject's body, and two-dimensional coordinate information is created, which is time-series data of the two-dimensional coordinates of the extracted feature points. The motion analysis step includes detecting the time change in the relative positional relationship between specific feature points by analyzing the two-dimensional coordinate information, and creating numerical information that indicates the characteristics of the action to be analyzed performed by the subject based on the detection results. The feature points extracted using the aforementioned physical feature point estimation model include at least two points from among the left and right shoulder joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joints. When determining whether the posture of the subject performing the aforementioned action to be analyzed is "flexed" or "unflexed", In the aforementioned motion analysis step, From the aforementioned two-dimensional coordinate information, a first straight line connecting the hip joint and the shoulder joint, a second straight line connecting the hip joint and the knee joint, and a third straight line connecting the knee joint and the ankle joint are recognized. The process is performed to determine that the object is "flexed" when the intersection angle between the first line and the second line is smaller than the first reference value and the intersection angle between the second line and the third line is larger than the second reference value, and to determine that it is "not flexed" when it does not fall under the category of "flexed". A motion analysis method characterized by creating posture judgment information that shows the change in the subject's posture over time, based on this judgment result.

6. A computer-based method for analyzing computer behavior, A video data acquisition step involves acquiring video data by capturing images of the subject performing the action to be analyzed from the side, and For each still image data constituting the video data, a two-dimensional coordinate information creation step is performed, in which a body feature point estimation model created by machine learning is used to extract multiple feature points of the subject's body, and two-dimensional coordinate information is created, which is time-series data of the two-dimensional coordinates of the extracted feature points. The motion analysis step includes detecting the time change in the relative positional relationship between specific feature points by analyzing the two-dimensional coordinate information, and creating numerical information that indicates the characteristics of the action to be analyzed performed by the subject based on the detection results. The feature points extracted using the aforementioned physical feature point estimation model include at least two points from among the left and right shoulder joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joints. The aforementioned movement under analysis is a kick-out movement in which the subject's posture changes from "bent" or "tucked" to "extended" during a forward or backward somersault. In the aforementioned motion analysis step, From the aforementioned two-dimensional coordinate information, the two-dimensional coordinates of a virtual body center point for identifying the height position of the subject's body are calculated, and based on the calculation results, a continuous function showing the time change in the height position of the body center point is derived, or a continuous function derivation process is performed to derive a continuous function showing the time change in the height position of a specific feature point from the aforementioned two-dimensional coordinate information. From the aforementioned two-dimensional coordinate information, a straight line connecting the hip joint and the knee joint, ankle joint, or shoulder joint is recognized, and the change in the position of the straight line over time is detected as a rotational movement with the hip joint as the origin, or a straight line connecting the shoulder joint and the hip joint, knee joint, or ankle joint is recognized, and the change in the position of the straight line over time is detected as a rotational movement with the shoulder joint as the origin, and based on this detection result, a rotational motion detection process is performed to detect the rotational movement of the subject in the forward or backward direction. Based on the aforementioned two-dimensional coordinate information, a first line connecting the hip joint and the shoulder joint, a second line connecting the hip joint and the knee joint, and a third line connecting the knee joint and the ankle joint are recognized. A posture determination process is performed in which the posture is determined to be "extended" when the intersection angle between the first line and the second line is greater than the first reference value, and the intersection angle between the second line and the third line is greater than the second reference value, and the posture is determined to be "not extended" when it does not fall under the category of "extended". A motion analysis method characterized in that the period from when the slope of the continuous function changes from a value less than or equal to zero to a positive value, until when it changes from a negative value to a value greater than or equal to zero, is defined as the jump period, the rotational movement of the subject is detected during the jump period, and if the subject's posture changes from "non-extended" to "extended" during the rotational movement, it is determined that a "kick-out movement has occurred," and kick-out movement detection information is created.

7. The aforementioned movement under analysis is a downward movement in which the subject's posture changes from "extended" to "non-extended" during the jump and then lands in that position. In the aforementioned motion analysis step, From the aforementioned two-dimensional coordinate information, the two-dimensional coordinates of a virtual body center point for identifying the height position of the subject's body are calculated, and based on the calculation results, a continuous function showing the time change in the height position of the body center point is derived, or a continuous function derivation process is performed to derive a continuous function showing the time change in the height position of a specific feature point from the aforementioned two-dimensional coordinate information. Based on the aforementioned two-dimensional coordinate information, a first line connecting the hip joint and the shoulder joint, a second line connecting the hip joint and the knee joint, and a third line connecting the knee joint and the ankle joint are recognized. A posture determination process is performed in which the posture is determined to be "extended" when the intersection angle between the first line and the second line is greater than the first reference value, and the intersection angle between the second line and the third line is greater than the second reference value, and the posture is determined to be "not extended" when it does not fall under the category of "extended". The motion analysis method according to claim 6, wherein the period from when the slope of the continuous function changes from a positive value to a negative value until it changes from a negative value to a value greater than or equal to zero is defined as the body descent period, and when the posture of the subject changes from "extended" to "not extended" during the body descent period, it is determined that "a down movement has been performed" and down movement detection information is created.

8. A computer-based method for analyzing computer behavior, A video data acquisition step involves acquiring video data by capturing images of the subject performing the action to be analyzed from the side, and For each still image data constituting the video data, a two-dimensional coordinate information creation step is performed, in which a body feature point estimation model created by machine learning is used to extract multiple feature points of the subject's body, and two-dimensional coordinate information is created, which is time-series data of the two-dimensional coordinates of the extracted feature points. The motion analysis step includes detecting the time change in the relative positional relationship between specific feature points by analyzing the two-dimensional coordinate information, and creating numerical information that indicates the characteristics of the action to be analyzed performed by the subject based on the detection results. The feature points extracted using the aforementioned physical feature point estimation model include at least two points from among the left and right shoulder joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joints. The aforementioned movement under analysis is a downward movement in which the subject's posture changes from "extended" to "non-extended" during the jump and then lands in that position. In the aforementioned motion analysis step, From the aforementioned two-dimensional coordinate information, the two-dimensional coordinates of a virtual body center point for identifying the height position of the subject's body are calculated, and based on the calculation results, a continuous function showing the time change in the height position of the body center point is derived, or a continuous function derivation process is performed to derive a continuous function showing the time change in the height position of a specific feature point from the aforementioned two-dimensional coordinate information. Based on the aforementioned two-dimensional coordinate information, a first line connecting the hip joint and the shoulder joint, a second line connecting the hip joint and the knee joint, and a third line connecting the knee joint and the ankle joint are recognized. A posture determination process is performed in which the posture is determined to be "extended" when the intersection angle between the first line and the second line is greater than the first reference value, and the intersection angle between the second line and the third line is greater than the second reference value, and the posture is determined to be "not extended" when it does not fall under the category of "extended". A motion analysis method characterized in that the period from when the slope of the continuous function changes from a positive value to a negative value, until when it changes from a negative value to a value greater than or equal to zero, is defined as the body descent period, and if the posture of the subject changes from "extended" to "not extended" during the body descent period, it is determined that "a down movement has occurred," and down movement detection information is created.

9. In the aforementioned motion analysis step, The motion analysis method according to claim 8, which, when it is determined that a "downward movement has occurred", calculates the intersection angle between the first straight line and a virtual horizontal line or virtual vertical line at the time when the change in posture that forms the basis of this determination occurs, and creates intersection angle information based on this calculation result.

10. A motion analysis program comprising step execution programs for causing a computer to execute the motion analysis method described in any one of claims 1 to 9.

11. A motion analysis system provided within a computer, A video data acquisition unit acquires video data by capturing images of the subject performing the action to be analyzed from the side, A two-dimensional coordinate information creation unit extracts multiple feature points of the subject's body using a body feature point estimation model created by machine learning for each still image data constituting the video data, and creates two-dimensional coordinate information which is time-series data of the two-dimensional coordinates of the extracted feature points. The system includes a motion analysis unit that analyzes the two-dimensional coordinate information to detect changes in the relative positional relationship between specific feature points over time, and based on the detection results, creates numerical information that indicates the characteristics of the action performed by the subject to be analyzed. The feature points extracted using the aforementioned physical feature point estimation model include at least two points from among the left and right shoulder joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joints. The motion to be analyzed is a rotational motion in which the subject's body rotates forward or backward. The aforementioned motion analysis unit, From the aforementioned two-dimensional coordinate information, a straight line connecting the hip joint and the knee joint, ankle joint, or shoulder joint is recognized, and the time change in the position of the straight line is detected as a rotational movement with the hip joint as the origin. Alternatively, the system recognizes a straight line connecting the shoulder joint and the hip joint, knee joint, or ankle joint from the two-dimensional coordinate information, and processes the change in the position of the straight line over time as a rotational movement with the shoulder joint as the origin. A motion analysis system characterized by generating rotation count information, rotation speed information, or both, based on these detection results.

12. A motion analysis system provided within a computer, A video data acquisition unit acquires video data by capturing images of the subject performing the action to be analyzed from the side, A two-dimensional coordinate information creation unit extracts multiple feature points of the subject's body using a body feature point estimation model created by machine learning for each still image data constituting the video data, and creates two-dimensional coordinate information which is time-series data of the two-dimensional coordinates of the extracted feature points. The system includes a motion analysis unit that analyzes the two-dimensional coordinate information to detect changes in the relative positional relationship between specific feature points over time, and based on the detection results, creates numerical information that indicates the characteristics of the action performed by the subject to be analyzed. The feature points extracted using the aforementioned physical feature point estimation model include at least two points from among the left and right shoulder joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joints. The motion to be analyzed is an axial motion in which the subject's body rotates around a virtual straight line passing through the upper and lower body. The aforementioned motion analysis unit, A process that recognizes the distance between the left and right hip joints from the aforementioned two-dimensional coordinate information and detects the time change of said distance. Alternatively, the system recognizes the distance between the left and right shoulder joints from the two-dimensional coordinate information and performs a process to detect the time change of said distance. A motion analysis system characterized by generating information on the number of rotations around an axis, information on the speed around an axis, or both, based on these detection results.

13. A motion analysis system provided within a computer, A video data acquisition unit acquires video data by capturing images of the subject performing the action to be analyzed from the side, A two-dimensional coordinate information creation unit extracts multiple feature points of the subject's body using a body feature point estimation model created by machine learning for each still image data constituting the video data, and creates two-dimensional coordinate information which is time-series data of the two-dimensional coordinates of the extracted feature points. The system includes a motion analysis unit that analyzes the two-dimensional coordinate information to detect changes in the relative positional relationship between specific feature points over time, and based on the detection results, creates numerical information that indicates the characteristics of the action performed by the subject to be analyzed. The feature points extracted using the aforementioned physical feature point estimation model include at least two points from among the left and right shoulder joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joints. When determining whether the posture of the subject performing the aforementioned action to be analyzed is "extended" or "not extended", The aforementioned motion analysis unit, From the aforementioned two-dimensional coordinate information, a first straight line connecting the hip joint and the shoulder joint, a second straight line connecting the hip joint and the knee joint, and a third straight line connecting the knee joint and the ankle joint are recognized. The process is performed to determine "extended posture" when the intersection angle between the first line and the second line is greater than the first reference value, and the intersection angle between the second line and the third line is greater than the second reference value, and to determine "not extended posture" when it does not fall under the category of "extended posture". A motion analysis system characterized by creating posture judgment information that shows the change in the subject's posture over time, based on this judgment result.

14. A motion analysis system provided within a computer, A video data acquisition unit acquires video data by capturing images of the subject performing the action to be analyzed from the side, A two-dimensional coordinate information creation unit extracts multiple feature points of the subject's body using a body feature point estimation model created by machine learning for each still image data constituting the video data, and creates two-dimensional coordinate information which is time-series data of the two-dimensional coordinates of the extracted feature points. The system includes a motion analysis unit that analyzes the two-dimensional coordinate information to detect changes in the relative positional relationship between specific feature points over time, and based on the detection results, creates numerical information that indicates the characteristics of the action performed by the subject to be analyzed. The feature points extracted using the aforementioned physical feature point estimation model include at least two points from among the left and right shoulder joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joints. When determining whether the posture of the subject performing the aforementioned action to be analyzed is "hugging" or "not hugging", The aforementioned motion analysis unit, From the aforementioned two-dimensional coordinate information, a first straight line connecting the hip joint and the shoulder joint, a second straight line connecting the hip joint and the knee joint, and a third straight line connecting the knee joint and the ankle joint are recognized. The process is performed to determine if "encompassing" occurs when the intersection angle between the first line and the second line is smaller than the first reference value, and the intersection angle between the second line and the third line is smaller than the second reference value, and if it does not fall under "encompassing", it is determined to be "not encompassing". A motion analysis system characterized by creating posture judgment information that shows the change in the subject's posture over time, based on this judgment result.

15. A motion analysis system provided within a computer, A video data acquisition unit acquires video data by capturing images of the subject performing the action to be analyzed from the side, A two-dimensional coordinate information creation unit extracts multiple feature points of the subject's body using a body feature point estimation model created by machine learning for each still image data constituting the video data, and creates two-dimensional coordinate information which is time-series data of the two-dimensional coordinates of the extracted feature points. The system includes a motion analysis unit that analyzes the two-dimensional coordinate information to detect changes in the relative positional relationship between specific feature points over time, and based on the detection results, creates numerical information that indicates the characteristics of the action performed by the subject to be analyzed. The feature points extracted using the aforementioned physical feature point estimation model include at least two points from among the left and right shoulder joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joints. When determining whether the posture of the subject performing the aforementioned action to be analyzed is "flexed" or "unflexed", The aforementioned motion analysis unit, From the aforementioned two-dimensional coordinate information, a first straight line connecting the hip joint and the shoulder joint, a second straight line connecting the hip joint and the knee joint, and a third straight line connecting the knee joint and the ankle joint are recognized. The process is performed to determine that the object is "flexed" when the intersection angle between the first line and the second line is smaller than the first reference value and the intersection angle between the second line and the third line is larger than the second reference value, and to determine that it is "not flexed" when it does not fall under the category of "flexed". A motion analysis system characterized by creating posture judgment information that shows the change in the subject's posture over time, based on this judgment result.

16. A motion analysis system provided within a computer, A video data acquisition unit acquires video data by capturing images of the subject performing the action to be analyzed from the side, A two-dimensional coordinate information creation unit extracts multiple feature points of the subject's body using a body feature point estimation model created by machine learning for each still image data constituting the video data, and creates two-dimensional coordinate information which is time-series data of the two-dimensional coordinates of the extracted feature points. The system includes a motion analysis unit that analyzes the two-dimensional coordinate information to detect changes in the relative positional relationship between specific feature points over time, and based on the detection results, creates numerical information that indicates the characteristics of the action performed by the subject to be analyzed. The feature points extracted using the aforementioned physical feature point estimation model include at least two points from among the left and right shoulder joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joints. The aforementioned movement under analysis is a kick-out movement in which the subject's posture changes from "bent" or "tucked" to "extended" during a forward or backward somersault. The aforementioned motion analysis unit, From the aforementioned two-dimensional coordinate information, the two-dimensional coordinates of a virtual body center point for identifying the height position of the subject's body are calculated, and based on the calculation results, a continuous function showing the time change in the height position of the body center point is derived, or a continuous function derivation process is performed to derive a continuous function showing the time change in the height position of a specific feature point from the aforementioned two-dimensional coordinate information. From the aforementioned two-dimensional coordinate information, a straight line connecting the hip joint and the knee joint, ankle joint, or shoulder joint is recognized, and the change in the position of the straight line over time is detected as a rotational movement with the hip joint as the origin, or a straight line connecting the shoulder joint and the hip joint, knee joint, or ankle joint is recognized, and the change in the position of the straight line over time is detected as a rotational movement with the shoulder joint as the origin, and based on this detection result, a rotational motion detection process is performed to detect the rotational movement of the subject in the forward or backward direction. Based on the aforementioned two-dimensional coordinate information, a first line connecting the hip joint and the shoulder joint, a second line connecting the hip joint and the knee joint, and a third line connecting the knee joint and the ankle joint are recognized. A posture determination process is performed in which the posture is determined to be "extended" when the intersection angle between the first line and the second line is greater than the first reference value, and the intersection angle between the second line and the third line is greater than the second reference value, and the posture is determined to be "not extended" when it does not fall under the category of "extended". A motion analysis system characterized in that the period from when the slope of the continuous function changes from a value less than or equal to zero to a positive value, until when it changes from a negative value to a value greater than or equal to zero, is defined as the jump period, the rotational movement of the subject is detected during the jump period, and if the subject's posture changes from "non-extended" to "extended" during the rotational movement, it is determined that a "kick-out movement has occurred," and kick-out movement detection information is created.

17. A motion analysis system installed within a computer, A video data acquisition unit acquires video data by capturing images of the subject performing the action to be analyzed from the side, A two-dimensional coordinate information creation unit extracts multiple feature points of the subject's body using a body feature point estimation model created by machine learning for each still image data constituting the video data, and creates two-dimensional coordinate information which is time-series data of the two-dimensional coordinates of the extracted feature points. The system includes a motion analysis unit that analyzes the two-dimensional coordinate information to detect changes in the relative positional relationship between specific feature points over time, and based on the detection results, creates numerical information that indicates the characteristics of the action performed by the subject to be analyzed. The feature points extracted using the aforementioned physical feature point estimation model include at least two points from among the left and right shoulder joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joints. The aforementioned movement under analysis is a downward movement in which the subject's posture changes from "extended" to "non-extended" during the jump and then lands in that position. The aforementioned motion analysis unit, From the aforementioned two-dimensional coordinate information, the two-dimensional coordinates of a virtual body center point for identifying the height position of the subject's body are calculated, and based on the calculation results, a continuous function showing the time change in the height position of the body center point is derived, or a continuous function derivation process is performed to derive a continuous function showing the time change in the height position of a specific feature point from the aforementioned two-dimensional coordinate information. Based on the aforementioned two-dimensional coordinate information, a first line connecting the hip joint and the shoulder joint, a second line connecting the hip joint and the knee joint, and a third line connecting the knee joint and the ankle joint are recognized. A posture determination process is performed in which the posture is determined to be "extended" when the intersection angle between the first line and the second line is greater than the first reference value, and the intersection angle between the second line and the third line is greater than the second reference value, and the posture is determined to be "not extended" when it does not fall under the category of "extended". The motion analysis system according to claim 16, wherein the period from when the slope of the continuous function changes from a positive value to a negative value until it changes from a negative value to a value greater than or equal to zero is defined as the body descent period, and when the posture of the subject changes from "extended" to "not extended" during the body descent period, it is determined that "a down movement has been performed" and down movement detection information is created.

18. A motion analysis system provided within a computer, A video data acquisition unit acquires video data by capturing images of the subject performing the action to be analyzed from the side, A two-dimensional coordinate information creation unit extracts multiple feature points of the subject's body using a body feature point estimation model created by machine learning for each still image data constituting the video data, and creates two-dimensional coordinate information which is time-series data of the two-dimensional coordinates of the extracted feature points. The system includes a motion analysis unit that analyzes the two-dimensional coordinate information to detect changes in the relative positional relationship between specific feature points over time, and based on the detection results, creates numerical information that indicates the characteristics of the action performed by the subject to be analyzed. The feature points extracted using the aforementioned physical feature point estimation model include at least two points from among the left and right shoulder joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joints. The aforementioned movement under analysis is a downward movement in which the subject's posture changes from "extended" to "non-extended" during the jump and then lands in that position. The aforementioned motion analysis unit, From the aforementioned two-dimensional coordinate information, the two-dimensional coordinates of a virtual body center point for identifying the height position of the subject's body are calculated, and based on the calculation results, a continuous function showing the time change in the height position of the body center point is derived, or a continuous function derivation process is performed to derive a continuous function showing the time change in the height position of a specific feature point from the aforementioned two-dimensional coordinate information. Based on the aforementioned two-dimensional coordinate information, a first line connecting the hip joint and the shoulder joint, a second line connecting the hip joint and the knee joint, and a third line connecting the knee joint and the ankle joint are recognized. A posture determination process is performed in which the posture is determined to be "extended" when the intersection angle between the first line and the second line is greater than the first reference value, and the intersection angle between the second line and the third line is greater than the second reference value, and the posture is determined to be "not extended" when it does not fall under the category of "extended". A motion analysis system characterized in that the period from when the slope of the continuous function changes from a positive value to a negative value, until when it changes from a negative value to a value greater than or equal to zero, is defined as the body descent period, and when the posture of the subject changes from "extended" to "not extended" during the body descent period, it is determined that "a down movement has occurred" and down movement detection information is created.

19. The aforementioned motion analysis unit, The motion analysis system according to claim 17 or 18, which, when it is determined that a "downward movement has occurred," calculates the intersection angle between the first straight line and a virtual horizontal line or virtual vertical line at the time the change in posture that forms the basis of this determination occurs, and creates intersection angle information based on this calculation result.

Images