Human body posture estimation method applied to sprint starting teaching

Abstract

The invention discloses a human body posture estimation method applied to sprint starting teaching. The method comprises the following steps: firstly, designing a high-resolution network FD-HRNet with frequency domain information utilization capability, wherein the high-resolution network FD-HRNet is used for determining human body key point position information in an input image; according to the high-resolution network FD-HRNet, a frequency domain branch is added on the basis of a traditional HRNet network, and a fast Fourier residual block containing a complex weight matrix is used for replacing a traditional residual block in the network, so that extraction of frequency domain features is achieved. Experiments prove that the recognition accuracy of the model is obviously improved. Secondly, according to the technical characteristics of the sprint starting action, corresponding judgment criteria including angle values of a front leg knee joint angle, a rear leg knee joint angle, a trunk front rake angle and a head attitude angle are formulated, in order to avoid misjudgment caused by model recognition errors, an error threshold value is added to the angle values, and the actual use requirements are met.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision, relates to automated and intelligent sports training teaching, and specifically relates to a human body posture estimation method applied to sprint start teaching. Background Art

[0002] Human posture estimation is an important research topic in the field of computer vision. It aims to identify and locate the key points of the human body from images or videos. It is widely used in sports training and medical health. Intelligent motion correction technology based on human posture estimation is becoming an important tool to improve athletes' performance and prevent injuries during exercise.

[0003] In the teaching of sprint start, by comparing the data of standard movements with the actual detected movement data, the differences between the movements are identified, and based on these differences, specific evaluations or improvement suggestions are provided to athletes to help them maintain correct movements. This not only allows them to maintain a good body balance during the competition and reduce the risk of injury due to incorrect movements, but also helps them quickly reach their personal maximum speed and improve their competitive level.

[0004] The detection methods of human posture estimation can be divided into top-down methods and bottom-up methods. The top-down method performs better in terms of accuracy in processing individual postures and adaptability to complex postures. At the same time, since the sprint start action is relatively complex, the detection accuracy of the model is required to be high, so the top-down method is more suitable for the detection of sprint start actions. However, top-down method models such as SimpleBaseline, HRNet, and SimCC mainly focus on modeling in the time and space domains, and have some obvious deficiencies in the feature extraction of frequency domain information. However, frequency domain features have an important influence on human posture estimation in fast-changing scenes such as sprinting. Summary of the invention

[0005] In view of the shortcomings of the existing technology, the present invention proposes a human posture estimation method applied to the teaching of sprint start. A frequency domain branch and a residual block with a frequency domain information extraction branch are designed in combination with Fourier transform to solve the problem of insufficient utilization of frequency domain features. At the same time, according to the technical characteristics of the sprint start action, a special evaluation criterion is formulated to give corresponding suggestions for incorrect postures.

[0006] A human body posture estimation method applied to sprint start teaching specifically comprises the following steps:

[0007] Step 1: Standardize and normalize the size of the input image of the athlete's sprint start action, and then perform two convolution operations to obtain preliminary features.

[0008] Step 2: Use Fourier transform to create a frequency domain branch, perform Fourier transform on the preliminary features, convert the features from the spatial domain to the frequency domain, extract the frequency domain features in the image, process and enhance the frequency domain features; finally, perform inverse Fourier transform to reconstruct the features back to the original space.

[0009] Step 3: Perform feature fusion on the preliminary features extracted in step 1 and the frequency domain features reconstructed back to the original space in step 2 to obtain a fused feature map.

[0010] Step 4: Design a fast Fourier residual block including a complex weight matrix to replace the traditional residual block in the high-resolution network HRNet, and obtain a high-resolution network FD-HRNet with enhanced frequency domain features.

[0011] Step 5: Input the fused feature map obtained in step 3 into the high-resolution network FD-HRNet to perform multi-level feature extraction operations, where each level includes multiple sub-networks with different resolutions, and the sub-networks exchange feature map information by upsampling or downsampling. The outputs of all sub-networks in the last level are merged to obtain a high-resolution feature map.

[0012] Step 6: Use the high-resolution feature map with frequency domain features obtained in step 5 to output a series of heat maps. Each heat map corresponds to a key point, indicating the confidence of the key point appearing at different positions in the image. Use the non-maximum suppression method to accurately locate the area with the highest confidence on the heat map, and combine the sub-pixel refinement method to further obtain the exact position and complete the positioning of the key points of the human body in the starting action. Use annotated known human posture image data to train the network.

[0013] Step 7. Based on the past movements of professional male 100-meter athletes and relevant teaching materials, define the range of values ​​of each joint angle in the sprint starting movement and establish a standard table for judging the starting posture.

[0014] Step 8. Input the photo of the sprinter's starting posture into the high-resolution network FD-HRNet trained in step 6 to obtain the position information of the key points. After connecting the lines, calculate the angle values ​​of the front leg knee joint angle, the back leg knee joint angle, the trunk forward tilt angle and the head posture angle. According to the starting posture, judge whether the technical action is correct, and give corresponding improvement suggestions for incorrect actions.

[0015] The present invention has the following beneficial effects:

[0016] 1. In view of the limitations of HRNet in extracting frequency domain information, a frequency domain branch is added to HRNet for multi-scale fusion, and a fast Fourier residual block with a complex weight matrix is ​​proposed to replace the original residual block. The detection accuracy of the new model FD-HRNet has been greatly improved, which can meet the requirements of model accuracy for sprint starting actions.

[0017] 2. The evaluation criteria are set for the starting posture of sprinting. Taking into account the impact of model accuracy on the detection results of key points of the human body, an error threshold is designed in the evaluation criteria to achieve an effective evaluation of the standardization of the starting movements of sprint beginners and provide specific improvement suggestions for improper movements, which will help improve the athletic performance of sprint beginners and reduce the risk of injury caused by incorrect movements. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a schematic diagram of the FD-HRNet model structure constructed in the embodiment;

[0019] Figure 2 Schematic diagram of the structure of the fast Fourier residual block constructed in the embodiment;

[0020] Figure 3 This is a schematic diagram of the starting posture for sprinting;

[0021] Figure 4 This is the first detection example diagram in the embodiment;

[0022] Figure 5 This is the second detection example diagram in the embodiment;

[0023] Figure 6 This is the third detection example picture in the embodiment. DETAILED DESCRIPTION

[0024] The present invention will be further explained below with reference to the accompanying drawings;

[0025] like Figure 1 As shown, a human body posture estimation method applied to sprint start teaching specifically includes the following steps:

[0026] Step 1: The input data image is resized and normalized, and then downsampled twice through convolution operations to obtain preliminary features.

[0027] Step 2: Use Fourier transform to create a frequency domain branch. First, perform Fourier transform on the extracted preliminary features to convert the preliminary features from the spatial domain to the frequency domain. Then, process and enhance them through 1x1 convolution, ReLU activation function, and another layer of 1x1 convolution. Finally, perform inverse Fourier transform to reconstruct the features back to the original space.

[0028] By performing image analysis in the frequency domain, the basic features of the image, such as texture and edge information, can be effectively extracted, which helps to identify rapid changes and tiny movements of the athlete's body.

[0029] Step 3: Fuse the preliminary features extracted in step 1 and the features extracted in the frequency domain in step 2 to obtain a fused feature map.

[0030] Step 4: Design a fast Fourier residual block with complex weight matrix to replace the traditional residual block in the high-resolution network HRNet, and obtain the high-resolution network FD-HRNet with enhanced frequency domain features. Figure 2 As shown, the fast Fourier residual block adds a frequency domain branch with a complex weight matrix Wc on the basis of the traditional residual block.

[0031] In the traditional residual block, the input features are sequentially subjected to 3x3 convolution, ReLU activation function and 3x3 convolution, and then fused with the original input features to obtain spatial domain features. In the frequency domain branch, the features are first converted from the spatial domain to the frequency domain using Fourier transform, and then processed and enhanced by 1x1 convolution, ReLU activation function, and 1x1 convolution, and then a learnable complex weight matrix Wc is applied, and then converted back to spatial domain features through inverse Fourier transform, which is fused with the output of the traditional residual block.

[0032] The complex weight matrix Wc is expressed as:

[0033] Wc=Wc real +j*Wc imag (1)

[0034] Where j is the imaginary unit, Wc real and Wc imag Represent the real and imaginary parts of the weight respectively. real and Wc imag Initialization is performed. In this embodiment, it is initialized to a normal distribution with a mean of 0 and a standard deviation of 0.02. The weights are automatically updated according to the gradient descent method through the forward propagation of the network and the feedback of the loss function. The optimizer adjusts the gradient of the weights according to the loss function to find the optimal weight configuration. If the enhancement of a frequency component shows performance improvement on the verification data, the weight corresponding to the frequency component will increase during the training process; conversely, if it causes performance degradation, the corresponding weight will be reduced. Through continuous learning and self-adjustment of the network, the representation of frequency domain features is optimized to achieve better performance:

[0035] W real ,Wc imag ~N(0,0.02 2 )(2)

[0036] This initialization strategy helps to provide appropriate randomness in the early stages of network training, preventing instability caused by too large initial weight values ​​and preventing the inability to effectively propagate gradients due to too small initial weight values.

[0037] Resize the real and imaginary parts:

[0038] W real _resized=Wc real [:C:C] (3)

[0039] W imag _resized=Wc imag [:C:C] (4)

[0040] Where C is the number of channels of frequency domain features, Wc real _resized and Wc imag _resized is the adjusted real and imaginary parts.

[0041] Wc=Wc real _resized+j*Wc imag _resized (5)

[0042] For the frequency domain branch enhanced The complex weight matrix Wc performs weighted processing on it:

[0043] Y”=Y'*Wc (6)

[0044] Step 5: Input the fused feature map obtained in step 4 into the high-resolution network, and perform a 4-level feature extraction operation on the fused feature map. For the fused feature map, first reduce the resolution by downsampling, then pass through the two fast Fourier residual blocks in the first stage, downsample the output of the high-resolution fast Fourier residual block, upsample the output of the low-resolution fast Fourier residual block, and then fuse it with the output of the fast Fourier residual block, and then downsample the output of the low-resolution fast Fourier residual block, and input the feature maps of the three resolutions into the three fast Fourier residual blocks in the second stage. In the second stage, the outputs of the fast Fourier residual blocks of different resolutions are also upsampled or downsampled and fused. After repeating the fast Fourier residual and fusion operation four times, the output of the lowest resolution fast Fourier residual block is downsampled, and the feature maps of the four resolutions are respectively input into the four fast Fourier residual blocks in the third stage. After repeating the fast Fourier residual and fusion operation twice, they are input into the four fast Fourier residual blocks in the fourth stage. The outputs of the three low-resolution fast Fourier residual blocks are upsampled and fused with the output of the highest resolution fast Fourier residual block to obtain the final high-resolution feature map.

[0045] Step 6: Use the obtained high-resolution feature map with frequency domain features to output a series of heatmaps, each of which corresponds to a key point on the human body. It indicates the confidence level of the key point appearing at different positions in the image. The non-maximum suppression method is used to accurately locate the area with the highest confidence on the heatmap, and the sub-pixel refinement method is used to further obtain the exact position, completing the positioning of 17 key points of the human body in the starting action, such as Figure 3 As shown, it includes left hip, right hip, left knee, right knee and left ankle, right ankle, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left ear, right ear, left eye, right eye and nose. The network is trained using annotated known human pose image data.

[0046] Step 7. Combining the past movements of professional male 100-meter athletes and relevant teaching materials, the reasonable range of angles of each joint in the sprint start movement is defined in detail, and the preliminary starting posture evaluation criteria are established as shown in Table 1. The evaluation angle range is equal to the standard value μ plus or minus the allowable error range σ:

[0047] Table 1

[0048]

[0049] Step 8: In order to avoid the deviation of joint angle caused by the error in the positioning of the key points of the human body, the preliminary starting posture evaluation criteria shown in Table 1 are further improved. Through the Labelme data annotation platform, 100 data images of sprint postures are annotated according to the key point annotation format of the public data set COCO, such as Figure 3 As shown, based on the annotated key point position information, the actual value of each joint angle is calculated, and the average value is taken as the actual mean R:

[0050]

[0051] Among them, θ i represents the actual joint angle value of the i-th image, and N is the total number of images.

[0052] In addition, the 100 labeled data images of sprinting postures were input into the model respectively, and the actual values ​​of each joint angle were calculated through the key point position information predicted by the model, and the average value was taken to compare with the actual value to correct the judgment standard. In order to prevent the predicted value greater than the actual mean and the predicted value less than the actual mean from being averaged to neutralize the error, resulting in unreliable actual results and inability to effectively judge the accuracy of the action, the key point position information P = {p1, p2, ..., p N} is greater than the actual mean P b and less than the actual mean P s :

[0053] P b ={p∈P:p>R} (8)

[0054] P s ={p∈P:p <R} (9)

[0055] Among them, P i Represents the predicted joint angle value of the i-th image. Calculate the larger predicted mean P based on the classification results ba With a smaller predicted mean P sa :

[0056]

[0057] N Pb is the number of predicted values ​​greater than the actual mean R, P bi is the predicted value that is greater than the actual mean R. Ps is the number of predicted values ​​that are less than the actual mean R, P si is the i-th predicted value that is less than the actual mean R.

[0058] Then calculate the predicted mean P ba , P sa The absolute value P of the difference between the actual mean R ba1 , P sa1 , used to measure the deviation between the predicted mean and the actual mean, setting the allowable error range σ new :

[0059] P ba1 =|P ba -R| (12)

[0060] P sa1 =|P sa -R| (13)

[0061] P error =max(P ba1 ,P sa1 ) (14)

[0062] σ new =P error +σ (15)

[0063] Wherein, σ represents the angle value of the allowable error range.

[0064] Based on Table 1, add the allowable error range σ new , and the new evaluation criteria are shown in Table 2:

[0065] Table 2

[0066]

[0067] Step 8. Input the photo of the sprinter's starting posture into the high-resolution network FD-HRNet trained in step 6 to obtain the position information of the key points, calculate the angle values ​​of the front leg knee joint angle, the rear leg knee joint angle, the trunk forward tilt angle and the head posture angle, and judge whether the athlete's technical movements are correct according to Table 2. If the angle value of each joint falls within the angle range defined by the judging criteria, the posture is judged to be correct; if it is not within this range, the posture is considered incorrect, ensuring the accuracy of the judging process and the feasibility of practical application.

[0068] Using average precision AP and recall score AR as evaluation criteria, the performance of this method is compared with that of common methods. The experimental results are shown in Table 3:

[0069] Table 3

[0070]

[0071] Simple-baselines simplifies the network structure and realizes effective feature extraction and fusion at all levels of the network. It achieves excellent key point detection performance through reasonable feature fusion and the use of context information. HRNet maintains high-resolution representation throughout the network by connecting convolutional networks of different resolutions in parallel, realizes information interaction at different scales, retains the spatial accuracy of high-resolution features, and integrates the semantic information of low-resolution features, so that it can more effectively capture the details of key points of the human body. The experimental results show that the average precision AP and recall score AR of this method are significantly higher than those of the other two methods, indicating that the recognition accuracy of this method is higher.

[0072] This method is applied to sprint posture detection. Three sprint posture pictures are randomly selected and the angles detected are as follows: Figure 4 , 5 , 6, compared with the evaluation criteria given in Table 2, the evaluation results and improvement suggestions are shown in Table 4:

[0073] Table 4

[0074]

[0075] This method achieves effective evaluation of sprint beginners through the improved human posture estimation model and established evaluation criteria, and can provide specific improvement suggestions for improper movements.

Claims

1. A human body posture estimation method for teaching sprint start, characterized by: The specific steps include: Step 1: Standardize and normalize the size of the input image of the athlete's sprint start action, and then perform two convolution operations to obtain preliminary features; Step 2: Perform Fourier transform on the preliminary features, then process and enhance them through 1x1 convolution, ReLU activation function and another layer of 1x1 convolution, and finally perform inverse Fourier transform to reconstruct the features back to the original space; Step 3: Fusing the preliminary features extracted in step 1 and the frequency domain features reconstructed back to the original space in step 2 to obtain a fused feature map; Step 4: Design a fast Fourier residual block including a complex weight matrix Wc to replace the traditional residual block in the high-resolution network HRNet, and obtain a high-resolution network FD-HRNet with enhanced frequency domain features; Step 5: Input the fused feature map obtained in step 3 into the high-resolution network FD-HRNet to obtain a high-resolution feature map; Step 6: Use the high-resolution feature map containing frequency domain features obtained in step 5 to output a series of heat maps, each heat map corresponds to a key point, and complete the positioning of the key points of the human body in the starting action; use the annotated known human posture image data to train the network; Step 7: Input the photo of the sprinter's starting posture into the high-resolution network FD-HRNet trained in step 6 to obtain the location information of the key points and complete the posture estimation.

2. A method for estimating human posture applied to sprint start teaching as claimed in claim 1, characterized in that: The fast Fourier residual block adds a frequency domain branch with a complex weight matrix Wc on the basis of the traditional residual block.

3. A method for estimating human posture applied to sprint start teaching as claimed in claim 2, characterized in that: In the fast Fourier residual block, the input features are first subjected to 3x3 convolution, ReLU activation function and 3x3 convolution in sequence, and then fused with the original input features to obtain spatial domain features; then on the frequency domain branch, the features are first converted from the spatial domain to the frequency domain using Fourier transform, and then processed and enhanced through 1x1 convolution, ReLU activation function, and 1x1 convolution in sequence, and then a learnable complex weight matrix Wc is applied, and then it is returned to the spatial domain through inverse Fourier transform and fused with the spatial domain features of traditional residual processing.

4. A method for estimating human posture applied to sprint start teaching as claimed in claim 1, characterized in that: The complex weight matrix Wc is expressed as: Wc=Wc real +j*toilet imag (1) Where j is the imaginary unit, Wc real and Wc imag represent the real and imaginary parts of the weight respectively.

5. A method for estimating human posture used in teaching sprint start as claimed in claim 4, characterized in that: The complex weight matrix Wc real 、Wc imag Initialize to a normal distribution with mean 0 and standard deviation 0.

02.

6. A method for estimating human posture used in teaching sprint start as claimed in claim 1, characterized in that: The trained high-resolution network FD-HRNet obtains the positioning of 17 key points of the human body in the starting action, including the left hip, right hip, left knee, right knee and left ankle, right ankle, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left ear, right ear, left eye, right eye and nose.

7. A method for estimating human posture used in teaching sprint start as claimed in claim 1, characterized in that: Combining the past movements of professional male 100-meter athletes and relevant teaching materials, the range values ​​of each joint angle in the sprint starting action were defined, and a standard table for judging the starting posture was established, including the standard angle values ​​and allowable error ranges of the front leg knee angle, back leg knee angle, trunk forward tilt angle, and head posture angle.

Images