User-defined action matching scoring method based on three-dimensional posture recognition

Abstract

The invention discloses a user-defined action matching scoring method based on three-dimensional posture recognition. The method comprises the following steps: acquiring a standard action image; extracting a first rotation Euler angle parameter of the standard motion limb from the standard motion image; obtaining a to-be-evaluated action image; extracting a second rotation Euler angle parameter of the to-be-evaluated action limb from the to-be-evaluated action image; calculating the cosine similarity of the first rotation Euler angle parameter and the second rotation Euler angle parameter; and obtaining an evaluation result according to a calculation result. According to the user-defined action matching scoring method based on three-dimensional posture recognition, action matching is not limited by the visual angle of an action image through three-dimensional posture estimation, manual repeated labor during action standard making is reduced, and the labor cost is saved.

Description

Technical Field

[0001] The invention belongs to the technical field of posture evaluation, and in particular relates to a user-defined action matching scoring method based on three-dimensional posture recognition. Background Art

[0002] Motion matching is often used in various fields. For example, in the field of sports and fitness, when following sports videos to do exercise movements, motion matching is needed to inform the user whether the movements are standard and to avoid injuries caused by improper movements. In the field of science and education, schools need to evaluate students' morning exercises and need to match movements according to standard movements to score the movements. In the field of driving, the vehicle needs to respond to the driver's condition through motion matching. For example, if there are movements other than standard driving movements, prompts and warnings need to be given.

[0003] The existing three-dimensional motion matching scheme that is not limited by viewing angle requires the use of inertial sensors for motion capture, which captures human body movements based on the sensors and returns them for data processing. However, the sensors need to be worn by the person to be evaluated, which is not applicable in many scenarios. Without the help of inertial sensors for motion capture, in the current two-dimensional motion matching scheme based only on images, the action to be evaluated must be photographed at the same viewing angle as the standard action, which has very large limitations. In addition, if matching scores are to be performed for different actions, it is necessary to repeatedly define the action matching rules for multiple actions. For example, in sports scenes, both Tai Chi and Ba Duan Jin require developers to redefine the rules for body movements, manually select and limit the parameters of the two sets of actions for the system, which is very inconvenient for migration applications. Summary of the invention

[0004] The present invention provides a user-defined action matching scoring method based on three-dimensional posture recognition to solve the above-mentioned technical problems, and specifically adopts the following technical solutions:

[0005] A user-defined action matching scoring method based on three-dimensional posture recognition, comprising:

[0006] Get standard action images;

[0007] Extracting first rotation Euler angle parameters of standard action limbs from the standard action image;

[0008] Acquire an action image to be evaluated;

[0009] Extracting the second rotation Euler angle parameter of the action limb to be evaluated from the action image to be evaluated;

[0010] Calculating the cosine similarity of the first rotation Euler angle parameter and the second rotation Euler angle parameter;

[0011] The evaluation result is obtained according to the calculation result.

[0012] Furthermore, the specific method of extracting the first rotation Euler angle parameter of the standard action limb from the standard action image is:

[0013] Input the standard action image into the yolov5 model to perform human target detection on the image;

[0014] Inputting the detected target frame coordinates and the standard action image into the OSX three-dimensional posture detection model to obtain a human posture SMPL model;

[0015] Select the axis angle parameters representing the limb rotation in the SMPL model of human posture;

[0016] The rotation parameter is converted from the axis-angle representation to the Euler angle representation to obtain the first rotation Euler angle parameter.

[0017] Furthermore, the specific method of extracting the second rotation Euler angle parameter of the standard action limb from the action image to be evaluated is:

[0018] Input the action image to be evaluated into the yolov5 model to perform human target detection on the image;

[0019] Input the detected target frame coordinates and the action image to be evaluated into the OSX three-dimensional posture detection model to obtain a human posture SMPL model;

[0020] Select the axis angle parameters representing the limb rotation in the SMPL model of human posture;

[0021] The rotation parameter is converted from the axis-angle representation to the Euler angle representation to obtain the second rotation Euler angle parameter.

[0022] Furthermore, the specific method for selecting the axis angle parameters representing the rotation of limbs in the human posture SMPL model is:

[0023] Select the Rot6D rotation parameter representing the limb rotation from the posture feature parameter θ in the human posture SMPL model;

[0024] Convert the selected Rot6D rotation parameters into rotation angles to get the axis angle parameters.

[0025] Furthermore, the specific method of converting the rotation parameter from the axis-angle representation to the Euler angle representation to obtain the first rotation Euler angle parameter or the second rotation Euler angle parameter is:

[0026] The axis angle parameters are converted into a rotation matrix using the following formula:

[0027]

[0028] Among them, the axis angle parameter is (k, ω), where k represents the rotation axis, x, y, z represent the three dimensions of the rotation axis, ω represents the rotation angle, and R(k, ω) is the rotation matrix;

[0029] Then convert the rotation matrix into Euler angles. Euler angles have three dimensions: Yaw, pitch, and roll, which represent the yaw angle, pitch angle, and roll angle, respectively. Yaw is α, pitch is β, and roll is γ. The formula for obtaining Euler angles from the rotation matrix is:

[0030] α=atan2(R 13 , R 33 )

[0031] β=arcsin(-R 23 )

[0032] y=atan2(R 21 , R 22 ).

[0033] Further, a plurality of standard action images are obtained;

[0034] Extracting a plurality of first rotation Euler angle parameters of a standard action limb from a plurality of the standard action images;

[0035] Acquire multiple action images to be evaluated;

[0036] Extracting a plurality of second rotation Euler angle parameters of the limb to be evaluated from a plurality of the action images to be evaluated;

[0037] Calculating a plurality of the first rotation Euler angle parameters and a plurality of cosine similarities corresponding to the second rotation Euler angle parameters;

[0038] Calculate the average of multiple cosine similarities;

[0039] The evaluation result is obtained according to the calculation result.

[0040] Furthermore, the specific method of obtaining multiple standard action images is:

[0041] Acquire a standard action video with background music, and decompose the standard action video into a plurality of standard action images;

[0042] The specific method of obtaining multiple action images to be evaluated is:

[0043] An action video to be evaluated with the same background music is obtained, and the action video to be evaluated is decomposed into a plurality of action images to be evaluated.

[0044] Furthermore, the specific method of obtaining multiple standard action images is:

[0045] Acquire a standard action video with background music, and extract a plurality of standard action images from the standard action video at preset time intervals;

[0046] The specific method of obtaining multiple action images to be evaluated is:

[0047] A to-be-evaluated action video with the same background music is obtained, and a plurality of to-be-evaluated action images are extracted from the to-be-evaluated action video at the same preset time interval.

[0048] Further, for each of the action images to be evaluated, after extracting a plurality of the action images to be evaluated from the action video to be evaluated at the same preset time interval, the auxiliary evaluation action images of the preset number of frames before and after the action images to be evaluated are obtained;

[0049] Extracting a plurality of third rotation Euler angle parameters of the action limb to be evaluated from a plurality of the auxiliary evaluation action images;

[0050] For each of the action images to be evaluated, after calculating the cosine similarity between the corresponding first rotation Euler angle parameter and the corresponding second rotation Euler angle parameter, the cosine similarity between the corresponding first rotation Euler angle parameter and the corresponding plurality of third rotation Euler angle parameters is calculated, and whether the action corresponding to the action image to be evaluated is ahead or behind is determined according to the calculated cosine similarity result between the first rotation Euler angle parameter and the corresponding plurality of third rotation Euler angle parameters.

[0051] Furthermore, the preset time interval is determined according to the beat of the background music.

[0052] The invention is beneficial in that the user-defined action matching scoring method based on three-dimensional posture recognition is provided, which innovatively uses the three-dimensional human posture estimation model for the user-defined action matching scoring scheme. The image-based posture estimation model makes the action estimation not rely on the inertial sensor used for motion capture, which broadens the application scenarios; the three-dimensional posture estimation makes the action matching not limited by the perspective of the action image, and the picture can be successfully judged at any angle; only one picture of the standard action is input, and the parameters of the standard action can be automatically extracted and stored. This can reduce the workload of developers by customizing the standard action scheme, reduce the manual duplication of work when formulating action standards, and save labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0054] Figure 1 A schematic diagram of a user-defined action matching scoring method based on three-dimensional posture recognition provided by an embodiment of the present invention;

[0055] Figure 2 A schematic diagram of a three-dimensional posture recognition process provided by an embodiment of the present invention;

[0056] Figure 3 A schematic diagram of another implementation of a user-defined action matching scoring method based on three-dimensional posture recognition provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0058] like Figure 1 The present application shows a user-defined action matching scoring method based on three-dimensional posture recognition, comprising: S1: obtaining a standard action image. S2: extracting the first rotation Euler angle parameters of the standard action limb from the standard action image. S3: obtaining the action image to be evaluated. S4: extracting the second rotation Euler angle parameters of the action limb to be evaluated from the action image to be evaluated. S5: calculating the cosine similarity of the first rotation Euler angle parameter and the second rotation Euler angle parameter. S6: obtaining an evaluation result based on the calculation result. Through the above steps, the user-defined action matching scoring method based on three-dimensional posture recognition of the present application is not limited by the viewing angle limitation, and can directly determine the action matching based on the image. The above steps are described in detail below.

[0059] For step S1: obtain a standard action image.

[0060] First, a customized standard motion image is determined and used as a comparison object.

[0061] For step S2: extract the first rotation Euler angle parameters of the standard action limbs from the standard action image.

[0062] The specific method of extracting the first rotation Euler angle parameters of the standard action limbs from the standard action image is:

[0063] Input the standard action image into the yolov5 model to detect human targets in the image, and return the coordinates of a target box containing the human body. Input the detected target box coordinates and the standard action image into the OSX 3D posture detection model, and output the human posture SMPL model. Select the axis-angle parameters representing the rotation of the limbs in the human posture SMPL model. Convert the rotation parameters from the axis-angle representation to the Euler angle representation to obtain the first rotation Euler angle parameters.

[0064] Specifically, in 2D posture comparison, whether the action is the same is often determined by whether the direction of the human skeleton is the same. However, the same bone direction can actually be ambiguous; therefore, this patent uses the three-dimensional rotation Euler angle of each joint relative to its parent joint point to characterize the action, and the joints are shown in Table 1. Since the relative rotation in three-dimensional space can be uniquely determined, ambiguity is eliminated.

[0065] Table 1 Human joint parameter correspondence table

[0066] serial number Joint Name Variable Name serial number Joint Name Joint Name 1 Left hip L_Hip 12 neck Neck 2 Right hip R_Hip 13 Left clavicle L_Collar 3 Spine Point 1 Spine_1 14 Right clavicle R_Collar 4 Left knee L_Knee 15 head Head 5 right knee R_Knee 16 Left shoulder L_Shoulder 6 Spine Point No. 2 Spine_2 17 Right shoulder R_Shoulder 7 Left ankle L_Ankle 18 left elbow L_Elbow 8 Right ankle R_Ankle 19 right elbow R_Elbow 9 Spine Point No. 3 Spine_3 20 left wrist L_Wrist 10 Left foot L_Foot 21 right wrist R_Wrist 11 Right foot R_Foot

[0067] like Figure 2 As shown in the figure, in the 3D posture recognition stage, for the effectiveness of human posture estimation, the yolov5 target detection model is first used to frame the area where the human body is located. Then this part is input into the OSX human posture estimation model. The model first inputs the RGB image containing part of the human body into the embedding layer to obtain the high-dimensional feature vector body_tokens representing the body posture, and then generates the human posture feature parameters through the Transformer encoder and the human posture regression function. The parameters of the general image embedding layer, Transformer encoder and human posture regression function are all trained from a large amount of data. Then the human posture SMPL model can be obtained.

[0068] The posture characteristic parameter θ in the human posture SMPL model is the focus of action matching. In the implementation of the present application, the specific method of selecting the axis angle parameter representing the limb rotation in the human posture SMPL model is:

[0069] The Rot6D rotation parameter representing the limb rotation is selected from the posture feature parameter θ in the SMPL model of human posture. The selected Rot6D rotation parameter is converted into an angle representation to obtain an axis-angle parameter. In motion matching, the Rot6D rotation parameter is not suitable for human model visualization or comparison. Therefore, for the output posture feature parameter θ, the Rot6d is first converted into an axis-angle representation. The axis-angle representation is convenient for drawing 3D human images as a SMPL human model parameter. The axis-angle parameter uses 21×3 dimensions to describe the human body's motion posture; 21 refers to 21 joints, and the specific joints and their numbering relationships are shown in Table 1. 3 refers to the axis-angle value representing the three-dimensional rotation.

[0070] In the 3D posture recognition stage, the rotation angle parameters representing the limb rotation of the human body in the image are finally obtained, but the axis angle representation is still relatively abstract and not very interpretable. In this application, the rotation angle parameters are converted into Euler angles for similarity estimation.

[0071] As a preferred implementation, the specific method of converting the rotation parameter from the axis angle representation to the Euler angle representation to obtain the first rotation Euler angle parameter is:

[0072] The axis angle parameters are converted into a rotation matrix using the following formula:

[0073]

[0074] The axis-angle parameter is (k, ω). k represents the rotation axis, x, y, z represent the three dimensions of the rotation axis, ω represents the rotation angle, and R(k, ω) is the rotation matrix.

[0075] Then convert the rotation matrix into Euler angles. Euler angles have three dimensions: Yaw, pitch, and roll, which represent the yaw angle, pitch angle, and roll angle, respectively. Yaw is written as α, pitch is written as β, and roll is written as γ. The formula for obtaining Euler angles from the rotation matrix is:

[0076] α=atan2(R 13 , R 33 )

[0077] β=arcsin(-R 23 )

[0078] γ=atan2(R 21 , R 22 ).

[0079] For step S3: obtain the action image to be evaluated.

[0080] For step S4: extract the second rotation Euler angle parameters of the action limb to be evaluated from the action image to be evaluated.

[0081] The specific method of extracting the second rotation Euler angle parameters of the standard action limb from the action image to be evaluated is:

[0082] Input the action image to be evaluated into the yolov5 model to perform human target detection on the image.

[0083] The detected target frame coordinates and the action image to be evaluated are input into the OSX 3D posture detection model to obtain the human posture SMPL model.

[0084] Select the axis angle parameters representing the limb rotation in the SMPL model of human posture.

[0085] The rotation parameter is converted from the axis-angle representation to the Euler angle representation to obtain the second rotation Euler angle parameter.

[0086] The specific process refers to step S2 and will not be described again here.

[0087] For step S5: calculate the cosine similarity between the first rotation Euler angle parameter and the second rotation Euler angle parameter.

[0088] For step S6: obtain an evaluation result according to the calculation result.

[0089] Specifically, a preset threshold range may be set, and the evaluation result may be obtained by comparing the calculated cosine similarity with the threshold range.

[0090] The method described above can evaluate a single action. It is understandable that the method of the present application can also perform a comprehensive evaluation on multiple actions. Specifically, Figure 3 As shown, another user-defined action matching scoring method based on three-dimensional posture recognition of the present application comprises:

[0091] S10: Acquire multiple standard action images.

[0092] S20: Extracting a plurality of first rotation Euler angle parameters of a standard action limb from a plurality of standard action images.

[0093] S30: Acquire multiple action images to be evaluated.

[0094] S40: extracting a plurality of second rotation Euler angle parameters of the action limb to be evaluated from a plurality of action images to be evaluated.

[0095] S50: Calculate a plurality of first rotation Euler angle parameters and a plurality of cosine similarities corresponding to the second rotation Euler angle parameters.

[0096] S60: Calculate multiple cosine similarity averages.

[0097] S70: Obtaining an evaluation result according to the calculation result.

[0098] Specifically, the specific method of obtaining multiple standard action images is:

[0099] A standard motion video with background music is obtained, and the standard motion video is decomposed into multiple standard motion images.

[0100] The specific method of obtaining multiple action images to be evaluated is:

[0101] Obtain an action video to be evaluated with the same background music, and decompose the action video to be evaluated into multiple action images to be evaluated.

[0102] It is understandable that in actual operation, it is often necessary to evaluate the standard degree of a continuous action. For example, to evaluate a dance. In this case, a standard action video with background music is directly split into multiple standard action images according to the number of frames. Similarly, a to-be-evaluated action video with the same background music is split into multiple to-be-evaluated action images according to the number of frames. Then, according to the parameter extraction and comparison method described above, each frame is compared frame by frame. The average value of multiple cosine similarities is calculated. The evaluation result is obtained based on the average value.

[0103] As another preferred implementation, the specific method for obtaining multiple standard motion images is:

[0104] A standard action video with background music is obtained, and a plurality of standard action images are extracted from the standard action video at preset time intervals.

[0105] The specific method of obtaining multiple action images to be evaluated is:

[0106] A to-be-evaluated action video with the same background music is obtained, and a plurality of to-be-evaluated action images are extracted from the to-be-evaluated action video at the same preset time interval.

[0107] It is understandable that comparing each frame of a video requires a large amount of calculation. Therefore, using the same preset time interval to select the same number of images from the video and then compare them one by one can reduce the amount of calculation. In the embodiment of the present application, the preset time interval is determined according to the beat of the background music.

[0108] It is understandable that the previous comparison method obtains images according to the rhythm of music and then compares them one by one. Such a comparison method may not fully reflect the performance status of the performer. For example, some performers perform their actions very standardly, but do not fully match the music beat. Through the aforementioned evaluation method, the evaluation results may be poor. However, for such performers, more problems are the matching of actions and music. In order to solve such problems, for each action image to be evaluated, after extracting multiple action images to be evaluated from the action video to be evaluated at the same preset time interval, the auxiliary evaluation action image of the preset number of frames before and after is obtained. Then, multiple third rotation Euler angle parameters of the action limb to be evaluated are extracted from the multiple auxiliary evaluation action images. For each action image to be evaluated, after calculating the cosine similarity of the corresponding first rotation Euler angle parameter and the corresponding second rotation Euler angle parameter, the cosine similarity of the corresponding first rotation Euler angle parameter and the corresponding multiple third rotation Euler angle parameters is calculated, and the action corresponding to the action image to be evaluated is judged whether it is ahead or behind according to the calculation result.

[0109] It is understandable that after extracting multiple action images to be evaluated from the action video to be evaluated at the same preset time interval, for each action image to be evaluated, the auxiliary evaluation action images of the preset number of frames before and after it are obtained, such as the auxiliary evaluation action images of 10 frames before and after. All 21 frames of images are compared with the corresponding standard images to find the frame with the highest similarity. According to the frame with the highest similarity, it is the action image to be evaluated extracted from the action video to be evaluated at the preset time interval, which indicates that there is no problem with the rhythm. If the frame with the highest similarity is in the 10 frames before the action image to be evaluated, it indicates that the action is snapped. According to its position in these 10 frames, it can be judged how much the action is snapped. If the frame with the highest similarity is in the 10 frames after the action image to be evaluated, it indicates that the action is delayed. Similarly, according to its position in these 10 frames, it can be judged how much the action is delayed.

[0110] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any form, and any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A user-defined action matching scoring method based on three-dimensional posture recognition, characterized in that: Include: Get standard action images; Extracting first rotation Euler angle parameters of standard action limbs from the standard action image; Acquire an action image to be evaluated; Extracting the second rotation Euler angle parameter of the action limb to be evaluated from the action image to be evaluated; Calculating the cosine similarity of the first rotation Euler angle parameter and the second rotation Euler angle parameter; The evaluation result is obtained according to the calculation result.

2. The user-defined action matching scoring method based on three-dimensional posture recognition according to claim 1, characterized in that: The specific method of extracting the first rotation Euler angle parameter of the standard action limb from the standard action image is: Input the standard action image into the yolov5 model to perform human target detection on the image; Inputting the detected target frame coordinates and the standard action image into the OSX three-dimensional posture detection model to obtain a human posture SMPL model; Select the axis angle parameters representing the limb rotation in the SMPL model of human posture; The rotation parameter is converted from the axis-angle representation to the Euler angle representation to obtain the first rotation Euler angle parameter.

3. The user-defined action matching scoring method based on three-dimensional posture recognition according to claim 2, characterized in that: The specific method of extracting the second rotation Euler angle parameters of the standard action limb from the action image to be evaluated is: Input the action image to be evaluated into the yolov5 model to perform human target detection on the image; Input the detected target frame coordinates and the action image to be evaluated into the OSX three-dimensional posture detection model to obtain a human posture SMPL model; Select the axis angle parameters representing the limb rotation in the SMPL model of human posture; The rotation parameter is converted from the axis-angle representation to the Euler angle representation to obtain the second rotation Euler angle parameter.

4. The user-defined action matching scoring method based on three-dimensional posture recognition according to claim 3, characterized in that: The specific method for selecting the axis angle parameters representing the rotation of limbs in the human posture SMPL model is: Select the Rot6D rotation parameter representing the limb rotation from the posture feature parameter θ in the human posture SMPL model; Convert the selected Rot6D rotation parameters into rotation angles to get the axis angle parameters.

5. The user-defined action matching scoring method based on three-dimensional posture recognition according to claim 4, characterized in that: The specific method of converting the rotation parameter from the axis-angle representation to the Euler angle representation to obtain the first rotation Euler angle parameter or the second rotation Euler angle parameter is: The axis angle parameters are converted into a rotation matrix using the following formula: Among them, the axis angle parameter is (k, ω), where k represents the rotation axis, x, y, z represent the three dimensions of the rotation axis, ω represents the rotation angle, and R(k, ω) is the rotation matrix; Then convert the rotation matrix into Euler angles. Euler angles have three dimensions: Yaw, pitch, and roll, which represent the yaw angle, pitch angle, and roll angle, respectively. Yaw is α, pitch is β, and roll is γ. The formula for obtaining Euler angles from the rotation matrix is: α=atan2(R 13 ,R 33 ) β=arcsin(-R 23 ) γ=atan2(R 21 ,R 22 )。 6. The user-defined action matching scoring method based on three-dimensional posture recognition according to claim 1, characterized in that: Acquire multiple standard action images; Extracting a plurality of first rotation Euler angle parameters of a standard action limb from a plurality of the standard action images; Acquire multiple action images to be evaluated; Extracting a plurality of second rotation Euler angle parameters of the limb to be evaluated from a plurality of the action images to be evaluated; Calculating a plurality of the first rotation Euler angle parameters and a plurality of cosine similarities corresponding to the second rotation Euler angle parameters; Calculate the average of multiple cosine similarities; The evaluation result is obtained according to the calculation result.

7. The user-defined action matching scoring method based on three-dimensional posture recognition according to claim 6, characterized in that: The specific method of obtaining multiple standard action images is: Acquire a standard action video with background music, and decompose the standard action video into a plurality of standard action images; The specific method of obtaining multiple action images to be evaluated is: An action video to be evaluated with the same background music is obtained, and the action video to be evaluated is decomposed into a plurality of action images to be evaluated.

8. The user-defined action matching scoring method based on three-dimensional posture recognition according to claim 7, characterized in that: The specific method of obtaining multiple standard action images is: Acquire a standard action video with background music, and extract a plurality of the standard action images from the standard action video at a preset time interval; The specific method of obtaining multiple action images to be evaluated is: A to-be-evaluated action video with the same background music is obtained, and a plurality of to-be-evaluated action images are extracted from the to-be-evaluated action video at the same preset time interval.

9. The user-defined action matching scoring method based on three-dimensional posture recognition according to claim 8, characterized in that: For each of the action images to be evaluated, after extracting a plurality of the action images to be evaluated from the action video to be evaluated at the same preset time interval, the auxiliary evaluation action images of the preset number of frames before and after the action images to be evaluated are obtained; Extracting a plurality of third rotation Euler angle parameters of the action limb to be evaluated from a plurality of the auxiliary evaluation action images; For each of the action images to be evaluated, after calculating the cosine similarity between the corresponding first rotation Euler angle parameter and the corresponding second rotation Euler angle parameter, the cosine similarity between the corresponding first rotation Euler angle parameter and the corresponding plurality of third rotation Euler angle parameters is calculated, and whether the action corresponding to the action image to be evaluated is ahead or behind is determined according to the calculated cosine similarity result between the first rotation Euler angle parameter and the corresponding plurality of third rotation Euler angle parameters.

10. The user-defined action matching scoring method based on three-dimensional posture recognition according to claim 8, characterized in that: The preset time interval is determined according to the beat of the background music.

Images