A Method for Evaluating the Normative Performance of Standing Long Jump Based on Posture Estimation

Abstract

A method for evaluating the standardization of standing long jump movements based on posture estimation, belonging to the field of human motion posture correction technology, is characterized by the following steps: Step 1, using a deep learning-based human posture estimation algorithm to capture movements in a video stream and estimate human posture; Step 2, identifying key movements in the standing long jump, constructing a key point sequence of key movements, comparing it frame-by-frame with the test video, and finally selecting the key movements for testing; Step 3, extracting the key movements and performing a standardization analysis with standard movements to obtain the similarity between the movements and standard movements. In this method for evaluating the standardization of standing long jump movements based on posture estimation, a deep learning-based posture estimation algorithm is used to obtain the key points of the long jump movement. By comparing the similarity between each frame of movement and the target movement, the key movements of the long jump are extracted, and the standardization of the long jump is analyzed to evaluate the long jump movement and provide improvement suggestions.

Description

TECHNICAL FIELD

[0001] The application relates to a standing long jump action normative evaluation method based on pose estimation, and belongs to the technical field of human body movement pose correction. BACKGROUND

[0002] In sports, whether the action is normative directly determines the training effect. If the action is not normative, the problems such as conditioned reflex of incorrect action, increased risk of sports injury, muscle coordination transfer, decreased body sensation ability and reduced training efficiency will be caused. The standing long jump is a track and field project with relatively strict action normative requirement. Whether the action is normative directly influences the training effect of athletes and the final result. In the traditional standing long jump training, the teacher generally makes subjective evaluation on the action of students. The students cannot intuitively find whether the action is standard, so an objective method is needed to evaluate the normative of the long jump action.

[0003] To analyze the normative of the long jump action, the movement pose needs to be captured first. The early human body action recognition needs the assistance of external equipment to perceive the change of the human body pose and then recognize the human body action. In the document Dowling AV, Favre J, Andriacchi TP. Inertial Sensor-Based Feedback Can Reduce Key Risk Metrics for Anterior Cruciate Ligament Injury During Jump Landings [J]. American Journal of Sports Medicine, 2012, 40(5): 1075-1083, the inertial sensor device is adopted to detect the knee bending angle, trunk inclination and thigh coronal speed, and is applied to the detection and recognition of anterior cruciate ligament injury. In the document Pansiot, Lo, Yang G Z. Swimming Stroke Kinematic Analysis with BSN [C] / / International Conference on Body Sensor Networks. IEEE, 2010, the technology scheme based on the micro sensor of the accelerometer is disclosed. The pitch angle and roll angle features extracted from the recorded acceleration are used to identify the human body pose and basic movement index. A system for detecting swimming performance is developed and can be applied to guide training. Therefore, in the current technology scheme for movement pose capture, although the sensor can accurately infer the human body action, the sensor needs to be worn every time, which is not convenient.

[0004] The standardization of an action is judged by calculating its similarity to a standard action, and many methods have been proposed for calculating the similarity between actions. In the literature "Jiang Ying. Research on sports-assisted training based on Kinect [J]. Automation Technology and Application, 2019, 38(9): 4", Euclidean distance is used to calculate the similarity. The difference between the joints is compared based on the obtained coordinates to evaluate the action. However, this method requires the two videos to correspond in height at the same time point, and when the trainees are different in height or weight, the calculation results will be greatly offset due to the change in coordinate position. Therefore, Euclidean distance has been gradually replaced by other methods. In the literature "Yu Jinghua, Wang Qing, Chen Hong. A motion-sensing dance interactive system based on action evaluation algorithm [J]. Computer and Modernization, 2018(6): 9", interpolation wavelet is used to extract the key frames of the reference action, the DTW algorithm is used to match the reference action and the comparison action, and finally the average distance between the matched key frames is normalized to obtain the similarity between the two action sequences. Using DTW to calculate similarity can effectively assess the similarity between time series, but it can only provide general similarity of actions and cannot provide detailed suggestions on actions.

[0005] To address the above problems, designing a standardized analysis method for the standing long jump, evaluating the movements of the standing long jump, and providing improvement suggestions are technical solutions that urgently need to be solved in this field. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a standing long jump action standardization evaluation method based on attitude estimation. This method uses a deep learning-based attitude estimation algorithm to obtain the key points of the long jump action, extracts the key long jump action by comparing the similarity between the action of each frame and the target action, and analyzes the standardization of the long jump action to evaluate the long jump action and give improvement suggestions.

[0007] The technical solution adopted by this invention to solve its technical problem is: a method for evaluating the standardization of standing long jump movements based on posture estimation, characterized by including the following steps:

[0008] Step 1: Use a deep learning-based human pose estimation algorithm to capture the actions in the video stream and estimate the human pose.

[0009] Step 2: Identify the key movements in the standing long jump, construct the key point sequence of the key movements, compare it frame by frame with the test video, and finally select the key movements for testing.

[0010] Step 3: Extract key actions and perform a normative analysis between the extracted key actions and standard actions to obtain the similarity between the test subject's actions and the standard actions.

[0011] Preferably, step 1 includes the following steps:

[0012] Step 1-1: Determine the human pose estimation dataset;

[0013] Steps 1-2 involve downsampling the input image;

[0014] Steps 1-3: Determine the human pose estimation algorithm, input all key points in the image, and use heatmaps to identify human key points;

[0015] Steps 1-4 involve applying different loss functions to the heatmaps generated in steps 1-3.

[0016] Preferably, in step 1-1, a subset of COCO (2017) and Al_Challenger (2018) is used and cropped so that each image contains only one person.

[0017] Preferably, in steps 1-3, a bottom-up approach is adopted: first, all key points in the input image are identified, then key points belonging to the same person are classified. When using heatmaps to identify human body key points, MobileNet-v3 is used as the backbone network, and features are extracted by combining feature pyramids to generate four heatmaps: the center point of the human body, the set of all human body key points, the offset of the key points, and the quantization error of the key points.

[0018] Preferred key points include the following parts of the human body: left and right eyes, left and right ears, nose, left and right shoulder joints, left and right elbow joints, left and right hands, left and right hip joints, left and right knee joints, and left and right feet.

[0019] Preferably, step 3 includes the following steps:

[0020] Step 3-1: Calculate the normality of local joints;

[0021] Step 3-2: Calculate the overall motion standardization;

[0022] Step 3-3: Calculate the standardization of the foot trajectory;

[0023] Steps 3-4, Steps 3-4, scoring of action standardization.

[0024] Preferably, in step 3-1, when determining the similarity of actions, dynamic programming is used to calculate the similarity of actions, and the angles of each joint are calculated using cosine similarity.

[0025] Preferably, in step 3-2, based on the characteristics of the standing long jump event and combined with the priority matching rule, features with higher priority are given larger feature indices. Finally, the feature similarity is calculated according to the feature indices as the evaluation result, obtaining the overall similarity between the skeletal keypoint vectors of the test posture and the standard posture:

[0026] C i =(1-λ) w d+λ(|cos(a i )|+1) q

[0027] Among them, C i The characteristic similarity of each joint is represented by cos(α). i ) represents the cosine similarity of each joint, d represents the DTW value of the two actions, λ∈(0,1), its optimal value is 0.73, w and q are feature indices.

[0028] Preferably, in steps 3-4, the motion standardization calculation uses a stepped scoring method, and the feature cosine similarity of each joint and the integral difference of the foot trajectory are weighted:

[0029]

[0030]

[0031] Among them, S i C represents the stepped score for each joint. i The feature similarity of each joint is represented by w and q, which are feature indices. The highest score for each joint is 20. The scores of each joint are added together and then the integral difference of the overall movement is subtracted to obtain the score of the key movement. k represents the scoring threshold, a represents the take-off point, and b represents the landing point.

[0032] Compared with the prior art, the beneficial effects of this invention are:

[0033] In this method for evaluating the standardization of standing long jump movements based on attitude estimation, a deep learning-based attitude estimation algorithm is used to obtain the key points of the long jump movement. By comparing the similarity between each frame of the movement and the target movement, the key movements of the long jump are extracted, and the standardization of the long jump is analyzed in order to evaluate the long jump movement and provide improvement suggestions. Attached Figure Description

[0034] Fig. 1 This is a flowchart of a method for evaluating the standardization of standing long jump movements based on posture estimation.

[0035] Fig. 2 Prepare a motion DTW value curve for a posture estimation-based method for evaluating the standardization of standing long jump movements.

[0036] Fig. 3 This is a graph showing the DTW value of aerial movements in a standing long jump based on attitude estimation for evaluating the standardization of the movement.

[0037] Fig. 4 This is a graph showing the DTW value of the landing action in a standing long jump action standardization evaluation method based on attitude estimation.

[0038] Fig. 5 This is a curve showing the DTW value of the buffered action in the standing long jump motion standardization evaluation method based on posture estimation.

[0039] Fig. 6 This is a distribution map of foot key points before removing non-long jump key points in the standardization assessment method of standing long jump based on posture estimation.

[0040] Fig. 7 This is a distribution diagram of foot key points after removing non-long jump key points in the standing long jump action standardization evaluation method based on posture estimation. Detailed Implementation

[0041] Figs. 1-7 This is the preferred embodiment of the present invention, which is described below in conjunction with the accompanying drawings. Figs. 1-7 The present invention will be further described below.

[0042] like Fig. 1 As shown, a method for evaluating the standardization of standing long jump movements based on posture estimation (hereinafter referred to as the evaluation method) includes the following steps:

[0043] Step 1, Human pose estimation;

[0044] In existing technologies, methods for human pose estimation include top-down approaches such as CPM and AlphaPose, and bottom-up approaches such as OpenPose and PersonLab. However, traditional human pose estimation schemes suffer from keypoint localization errors and slow model speeds. Therefore, this evaluation method addresses these issues by employing a deep learning-based human pose estimation algorithm to capture actions in video streams. Vision-based intelligent human pose estimation is one of the most challenging directions in computer vision in recent years. It identifies human behavior by detecting human actions in video sequences, extracting action features, and learning action features. Specifically, it includes the following steps:

[0045] Step 1-1: Determine the human pose estimation dataset;

[0046] In this evaluation method, subsets of COCO (2017) and Al_Challenger (2018) were used and cropped so that each image contained only one person. 1362 images of human skeletal key points from the standing long jump event were annotated and added to the dataset. Additionally, some original images were mirrored before annotation. By increasing the quantity and quality of annotated images, the problem of key point localization errors was effectively reduced.

[0047] Steps 1-2 involve downsampling the input image. During model training, the resolution of the input image is typically reduced to decrease training difficulty and resource consumption, and training is performed at the downsampled resolution. To enable model training using heatmaps, the skeletal keypoint coordinates in the original image need to be converted to the downsampled resolution during downsampling, and then transformed into a heatmap using Gaussian blur. After heatmap prediction, the resolution of the downsampled image is restored to the original pixels, and the predicted keypoint coordinates are found in the original coordinate space.

[0048] Steps 1-3: Determine the human pose estimation algorithm.

[0049] This estimation method employs a bottom-up approach: first, all keypoints in the input image are identified, and then keypoints belonging to the same person are classified. The model uses heatmaps to identify human keypoints, employing MobileNet-v3 as the backbone network and combining it with Feature Pyramid Network (FPN) to extract features, generating four heatmaps: the center point of the human body (Center), the set of all human keypoints (Keypoints), the keypoint offset (Reg), and the keypoint quantization error (Offset).

[0050] The obtained heatmap is processed by extracting 2K values ​​corresponding to the coordinate positions of the 2K channels in header_Reg, and then adding the center point coordinates to obtain the rough keypoint locations. Next, header_Keypoints is divided by a weight matrix, and the coordinates of the maximum values ​​for each of the K channels are calculated to obtain the refined coordinates of 17 keypoints. These 17 keypoints, from top to bottom, are: left and right eyes, left and right ears, nose, left and right shoulder joints, left and right elbow joints, left and right hands, left and right hip joints, left and right knee joints, and left and right feet, numbered from 1 to 17.

[0051] Steps 1-4 involve applying a different loss function to each heatmap.

[0052] In this estimation method, weighted MSE and L1 Loss are used as the loss function. In the heatmap generated in steps 1-3, the center point of the human body and the set of all human body keypoints are weighted by MSE to balance positive and negative samples. The keypoint offset and the keypoint quantization error are weighted by L1 Loss. Finally, the weights of each loss are set to 1:1:1:1.

[0053] Step 2, key action extraction;

[0054] A single long jump practice session can consist of hundreds of frames of motion. Analyzing each frame not only increases system complexity but also reduces its robustness. Therefore, before performing a standardized analysis of the motions, it is necessary to extract the key motions of the long jump. In this estimation method, the take-off, flight, landing, and cushioning motions are identified as the four key motions for completing a standing long jump. A keypoint sequence for each of these four motions is constructed and compared frame-by-frame with the test video to ultimately select the key motions for testing.

[0055] The comparison method for test videos and key actions employed Dynamic Time Warping (DTW), a nonlinear warping technique that combines distance measurement and time warping to measure the similarity between two time series of unequal lengths.

[0056] If the target point is defined as (x, y), then the DTW calculation formula for the target point is:

[0057]

[0058] Where K∈[max(x, y), x+y-1], W i This represents the sequence distance value for each path. The optimal path W must satisfy the following conditions:

[0059] 1. Boundary property: The starting and ending points of W must be the starting and ending points of the diagonal of the plane, i.e., W1 = (1, 1), W K = (x, y).

[0060] 2. Continuity: For two adjacent points W i (x i y i ) and W i-1 (x i-1 y i-1 ), where x i -x i-1 ≤1, y i -y i-1≤1, which makes adjacent points continuous.

[0061] 3. Monotonicity: For two adjacent points W i (x i y i ) and W i-1 (x i-1 y i-1 ), where x i -x i-1 ≥0, y i -y i-1 If W is greater than or equal to 0, then all elements on W are non-retrogressive.

[0062] Calculate the DTW value between the action sequence and the standard action sequence in all frames. The smaller the DTW value, the more similar the two sequences are. For example... Figs. 2-5 As shown, the minimum value among the four curves is taken as the key action.

[0063] Step 3, Analysis of Action Standardization;

[0064] After extracting key movements, a standardization analysis is needed to compare them with standard movements. This estimation method uses joint angle differences combined with feature indices to determine the standardization of movements. Joint angle differences are used to segment the test time series, and then DTW (Time-Divergence Wave) is used to calculate the distance difference between the test sequence and the standard sequence, thus obtaining the similarity between the tester's movements and the standard movements. Based on the joint angle calculation, feature indices are added to improve the accuracy of similarity calculation. The specific steps include:

[0065] Step 3-1: Calculate the normality of local joints;

[0066] Due to differences in human body shape, dynamic programming is needed to calculate the similarity of movements when determining their similarity. The angles of each joint are calculated using cosine similarity. Cosine similarity measures the difference in direction between two vectors: two n-dimensional vectors A = (a1, a2, ..., a...). n B = (b1, b2, ..., b) n The range of the cosine of the included angle is [-1, 1]. The cosine value is inversely proportional to the angle between the vectors. When the two vectors are in opposite directions, the value is -1, and when the two vectors are in the same direction, the value is 1.

[0067]

[0068] Among them: A i B i Let i represent a dimensional vector, where i takes values ​​from 1 to n, and θ represents the angles of each joint.

[0069] Step 3-2: Calculate the overall motion standardization;

[0070] Based on the characteristics of the standing long jump event and combined with priority matching rules, features with higher priority are assigned larger feature indices. Finally, the feature similarity is calculated based on the feature indices as the evaluation result, thus obtaining the overall similarity between the skeletal keypoint vectors of the test posture and the standard posture:

[0071] C i =(1-λ) w d+λ(|cos(a i )|+1) q

[0072] Among them, C i This represents the similarity of features among the joints; the smaller the value, the more precise the movement. cos(α) i ) represents the cosine similarity of each joint, d represents the DTW value of the two actions, λ∈(0,1), its optimal value is 0.73, w and q are feature indices.

[0073] Because the angle of movement at each joint is different, the resulting cosine similarity difference is relatively large. Therefore, w and q correspond to different values ​​for each joint to ensure that the weight of the score is the same, as shown in the table below:

[0074] Table 1. Characteristic indices for each joint

[0075]

[0076] Step 3-3: Calculate the standardization of the foot trajectory;

[0077] To ensure the standardization of the overall long jump procedure, the trajectory of key foot points is filtered out, and the absolute value of the integral between the key foot points and the horizontal line of the lowest point is calculated and compared with the standard movement deviation. Before calculation, key foot points need to be filtered out, eliminating key points outside the long jump process, such as... Figs. 6-7 As shown.

[0078] Since each node is discrete, the overall parabola is not a continuous and smooth curve, and the standard answer cannot be directly calculated using existing formulas. However, an approximate value can be obtained using the definition of the Riemann integral, as shown in the following formula:

[0079]

[0080] In this equation, the left side represents the integral value of the continuous function, and the right side represents the approximate integral value of the discrete function. Where x... i+1 -x i f(t) represents the difference in x-coordinates between adjacent points. i ) is x i+1 With x iFor the sake of consistency, we take the maximum value of a function within the interval. Multiplying the above values ​​and summing the results of i ranging from 0 to n-1, we can obtain an approximate value of the integral. 'a' represents the starting point, and 'b' represents the landing point.

[0081] Steps 3-4: Performance evaluation based on the correctness of the movements.

[0082] The final calculation of movement standardization uses a stepped scoring system, weighted by the difference between the feature cosine similarity of each joint and the integral of the foot trajectory. The formula is as follows:

[0083]

[0084]

[0085] In order to standardize the scoring, S i It uses a stepped scoring system for each joint, with a maximum score of 20 for each joint. The scores of each joint are summed and then the overall motion score is subtracted to obtain the score for the key motion. Since the range of motion of each joint is different, different scoring thresholds k are set. k is represented by feature indices w and q, such that the feature similarity C... i They can be represented using the same scoring criteria, where a represents the take-off point and b represents the landing point.

[0086] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for evaluating the standardization of standing long jump action based on pose estimation, characterized in that: Comprise the following steps: Step 1, using a human pose estimation algorithm based on deep learning, the action in the video stream is captured, the human pose is estimated; Step 2, determine the key action in standing long jump, construct the key point sequence of key action, compare with test video frame by frame, finally select the key action of test; Step 3, extract the key action, and compare the extracted key action with the standard action, get the similarity of the action of the tester and the standard action; Step 3, comprising the following steps: Step 3-1, calculate the local joint specification; Step 3-2, calculate the overall action specification; Step 3-3, calculate the specification of foot trajectory; Step 3-4, step 3-4, action specification score; In step 3-2, according to the characteristics of standing long jump project, combined with priority matching rule, give priority to the characteristics of greater feature index, finally calculate the feature similarity according to the feature index as the evaluation result, get the overall similarity between the measured posture and the standard posture skeleton key point vector: wherein, C i represents the feature similarity of each joint, cos( α i ) represents the cosine similarity of each joint, d represents the DTW value of two actions, Lambda ∈ (0, 1), the optimal value is 0.73, w , q is a feature index; In step 3-4, the action specification calculation adopts ladder type score, the feature cosine similarity of each joint and the integral difference value of foot trajectory are weighted: wherein, S i represents the ladder score of each joint, C i represents the feature similarity of each joint, w , q is the feature index, the highest score of each joint is 20, the scores of each joint are added and then subtracted by the integral difference value of the overall action to obtain the score of the key action, k represents the score threshold, a represents the take-off point, b represents the landing point.

2. The gesture estimation based standing long jump motion standard evaluation method according to claim 1, characterized in that: Step 1 includes the following steps: Step 1-1, determine the human pose estimation dataset; Step 1-2, downsample the input picture; Step 1-3, determine the human pose estimation algorithm, input all the key points in the image, and use the way of heat map to realize the recognition of human key points; Step 1-4, for the heat map formed in step 1-3, different loss functions are used respectively.

3. The gesture estimation based standing long jump motion standard evaluation method according to claim 2, characterized in that: In step 1-1, the subsets of COCO (2017) and Al_Challenger (2018) are used, and are cut to make each picture have only one person.

4. The gesture estimation based standing long jump motion standard evaluation method according to claim 2, characterized in that: In step 1-3, a bottom-up method is adopted: first, all the key points in the input image are recognized, then the key points belonging to the same person are classified, when using the way of heat map to realize the recognition of human key points, using MobileNet-v3 as the main network, combining with feature pyramid to extract features, generating four heat maps: the center point of the human body, the set of all human body key points, the offset of the key points, and the quantization error of the key points.

5. The gesture estimation based standing long jump motion standard evaluation method according to claim 1 or 4, characterized in that: The key points include the left and right eyes, left and right ears, nose, left and right shoulder joints, left and right elbow joints, left and right hands, left and right hip joints, left and right knee joints and left and right feet.

6. The gesture estimation based standing long jump movement normative evaluation method according to claim 1, characterized in that: In step 3-1, when judging the action similarity, dynamic programming method is used to calculate the similarity of the action, and cosine similarity calculation method is used to realize the angle of each joint.

Images