Athletics standard motion construction method based on video analysis

Abstract

The application discloses a track and field standard motion construction method based on video analysis, comprising the following steps: (1) constructing a 2D human body motion model; and (2) self-defining a standard motion posture. The method adopts 2D posture analysis, simplifies technical implementation complexity and reduces cost, overcomes the defect of high difficulty in high-speed motion capture, guarantees implementation simplicity, fully considers different environments such as indoor and outdoor environments, different illumination and weather conditions and the like, realizes close-to-real-time feedback and high accuracy, supports track and field informatization development and teaching guidance, can meet diversified track and field teaching requirements, and provides a solid technical foundation for athlete training optimization and competition analysis.

Description

Technical Field

[0001] This invention relates to the field of combining computer graphics, machine vision technology and human motion behavior simulation, specifically a method for constructing standard movements in track and field based on video analysis. Background Technology

[0002] In track and field instruction, establishing accurate standard postures is crucial for guiding athletes to master techniques. Traditional methods, such as drawing, diagrammatic analysis, and detailed observation, are often time-consuming, have limited accuracy, and lack interactivity.

[0003] Currently, human pose estimation mainly uses pose estimation models based on convolutional neural networks (CNN) [1][2] and sequence pose estimation models based on recurrent neural networks (RNN) [3]. These techniques learn the key points of the human body by analyzing a large amount of image data, thereby effectively capturing and analyzing the dynamic pose of the human body. In particular, Wu et al. [4] proposed a new human pose estimation model that uses high-resolution CNN to extract texture information from images. The model uses a parallel Transformer structure to learn the spatial dependencies between features, which provides more accurate pose estimation for analyzing track and field movements such as sprint start and long jump take-off. In addition, the introduction of 3D pose estimation technology provides a more in-depth and comprehensive perspective. The EgoEgo model developed by Li et al. [5] efficiently predicts 3D human poses from first-person perspective videos by processing them in stages. Athletes can better understand their movement habits and areas for improvement from the first-person perspective. Kundu et al. [6] proposed a self-supervised uncertainty perception method for 3D human pose estimation, which aims to improve prediction accuracy and learning efficiency. This can provide more accurate and reliable pose feedback when dealing with complex track and field movements, such as pole vaulting and other highly technical events.

[0004] Although deep learning technology has made significant progress in the field of human pose estimation, track and field sports are characterized by high dynamism, environmental diversity, and real-time characteristics, and existing methods still have some fundamental shortcomings. First, current technologies [1, 2, 3] mainly focus on the direct mapping between image features and key points, often ignoring the temporal information of human movement and the dynamic interaction between limbs, which limits the effectiveness of these models in handling high-speed movement and complex action sequences. Second, deep learning models usually rely on a large amount of labeled data for training, which is not only time-consuming and labor-intensive, but also limits the generalization ability of the model to the diversity and coverage of the training dataset. In addition, although high-resolution CNN [4] models have achieved certain success in human pose estimation, they may be difficult to accurately capture the texture information and details of athletes in some complex environments. Although the self-supervised 3D human pose estimation method proposed by Kundu et al. [6] can improve prediction accuracy and learning efficiency, the training of the model may be limited in track and field sports scenarios without sufficient labeled data. 3D pose estimation technology [6] requires higher computing resources, which may become a limiting factor in real-time or resource-constrained scenarios. Secondly, the complexity of acquiring and processing 3D data is far greater than that of 2D data. Not only does data acquisition require specialized equipment, such as depth cameras or motion capture systems, but data preprocessing and annotation are also more complex and time-consuming. Although 3D models can provide detailed spatial information, challenges remain in occlusion handling, rapid motion capture, and generalization capabilities across different environments. Summary of the Invention

[0005] The purpose of this invention is to address the limitations of traditional posture construction techniques—including those based on coach observation, simple video playback, and drawing and diagrammatic analysis—such as significant subjective judgment discrepancies, low motion capture accuracy, low time efficiency, and high demands on professional knowledge. This invention proposes a video analysis-based method for constructing standard movements in track and field. This method employs 2D posture analysis, simplifying implementation complexity and reducing costs. It overcomes the high difficulty of high-speed motion capture, ensuring ease of implementation while fully considering different environments such as indoor and outdoor settings, varying lighting conditions, and weather conditions. It achieves near real-time feedback and high accuracy, supporting the informatization of track and field and providing instructional guidance. Furthermore, it meets the diverse needs of track and field teaching, providing a solid technical foundation for athlete training optimization and competition analysis.

[0006] The technical solution to achieve the objective of this invention is:

[0007] A method for constructing standard movements in track and field based on video analysis includes the following steps:

[0008] (1) Constructing a 2D human motion model: The first step is to use the object detection model detr, i.e., M d The first step involves detecting human figures in the image and cropping the human information to obtain an image containing only the human body. The second step utilizes a self-built human pose estimation model, namely M... e The analysis of 2D human motion videos yields 2D human poses, specifically including the following steps:

[0009] S1. Divide the video into consecutive frames: Upload a video of human movement, which can be any track and field video, and the moving human body in the video must be clear; the straight-line distance between the shooting device and the human body should be 3-6 meters; the shooting frame rate should be above 120 frames per second, and the shooting resolution should be above 1080p. The video is marked as V. Use image processing technology to divide the video V into consecutive frames: V={v1,v2,...,v...} i}, v i Indicates the i-th frame;

[0010] S2. Human body location detection: Obtain the location of the human body and simultaneously crop out an image containing only the human body (the cropped human body location is determined by model M). d The output consists of the coordinates of the top left and bottom right corners of the human body.

[0011] S2-1. Image Processing: For the obtained video frame v i First, change v i The colors are arranged in RGB order, resulting in a three-channel color feature image, labeled P, where P∈R. H×W×3 H represents the height of the image, and W represents the width of the image;

[0012] S2-2. Extracting Human Body Location: Using Model M d The human body's position in image P is detected and marked with candidate bounding boxes. The top-left corner of each candidate bounding box is (x1, y1), and the bottom-right corner is (x2, y2). Finally, the image is cropped based on the candidate bounding box position information, and the background (excluding the human body) is removed to obtain feature map I. The formula is as follows:

[0013] (x1, y1, x2, y2) = M d (P) (1)

[0014] Model M d It consists of three steps: feature extraction, self-attention, and candidate box prediction. The formula for the first step, feature extraction, is as follows:

[0015] F = CNN(P) (2)

[0016] CNN stands for Convolutional Neural Network. By inputting P into a CNN, it can effectively extract preliminary feature information of the human body in an image, F∈R.H×W′×C C represents the number of channels in the feature map; next, in the second step of self-attention, F is flattened and then fed into the self-attention through position encoding to extract human feature information to the greatest extent, as shown in formula (3):

[0017] F′=PositionalEncoding(Flatten(F)) (3)

[0018] R = Transformer(F′) (4)

[0019] R represents the output of the self-attention algorithm, with a size of N×D, where N is a predefined upper limit on the number of targets (here, N is 1, indicating that only human bodies are detected); D is the output dimension for each target, including the category and candidate box coordinates. Finally, during candidate box prediction, R is transformed into candidate boxes B and corresponding category labels L. Invalid values ​​are removed through a linear layer and a specific activation function, as shown in the following formula:

[0020] B,L=Linear(R) (5)

[0021] B = Sigmoid(B)·[W,H,W,H] (6)

[0022] L = Softmax(L) (7)

[0023] Each element in B is mapped to the range [0,1] using the Sigmoid function. Finally, the coordinates are obtained by multiplying the height and width of the original image P. L uses the Softmax function to obtain the probability of each candidate box. The candidate box is only valid if it is higher than a certain threshold. Here, the threshold is set to 0.75, which means that the probability of the model predicting that the candidate box is a human body is greater than 0.75 before it is considered to be a human body.

[0024] S2-3. Image cropping: Finally, image I is obtained by cropping. The formula is as follows: CF represents the cropping function. The cropped image I is located in the region (x1,y1) to (x2,y2) of the original image P.

[0025] CF(P,(x1,y1),(x2,y2))=I (8)

[0026] S3. Extract key point features to obtain the coordinates of human joints and construct a 2D human pose feature map, i.e., a pose point map:

[0027] S3-1. Image Processing: To accurately identify key human body points, image I is processed through four different convolutional kernels to obtain four feature maps I1, I2, I3, and I4 of different sizes. I1, I2, I3, and I4 are then sequentially fed into model M. e Model M eBy learning the outputs of four feature maps I1, I2, I3, and I4, a generalization ability to process images of different sizes is obtained. The shapes of the four feature maps are as follows:

[0028]

[0029] S3-2. Feature Extraction: Design a window-based self-attention module. This module segments the image, performs individual self-attention on each segment, and then aggregates the self-attention data. Specifically, for a given feature map I... i The dimensions are h×w, and I i According to size p h ×p w Cut to obtain A window is created; self-attention is applied to each window, and then the self-attention data is aggregated to obtain feature map I. i The formula is as follows:

[0030]

[0031] w j Let I represent the j-th window. However, such window division leads to a lack of information exchange between windows, resulting in poor recognition of human key points. Therefore, a cross-window information exchange module was designed to process the feature map I after aggregation and self-attention. i The cross-window information exchange module, which provides supplementary information, consists of three convolutional kernels, as shown in the following formula:

[0032]

[0033] The window self-attention module and the cross-window communication module are combined into a feature extraction module. For each feature map, there is a corresponding feature extraction module, which are then concatenated, i.e., In(I i+1 =Out(I) i This allows different feature maps to share parameters and feature information.

[0034] S3-3. Feature Fusion: After the feature extraction module, four feature maps are obtained after feature extraction. By using four feature maps By unifying the dimensions and performing feature fusion, the model's ability to extract key points of the human body is further improved. The formula is as follows:

[0035]

[0036] The final feature obtained is a shape of A heatmap representing Q key points;

[0037] S3-4. Heatmap Decoding: Through the steps S3-1 to S3-3 above, a heatmap representing key points is obtained, where Q is set to 17, representing seventeen human body key points. The heatmap is decoded using a heatmap decoder to obtain the corresponding key points.

[0038]

[0039] (2) Customizing Standard Motion Posture: First, the 17 key points are normalized to obtain standardized coordinates. Then, based on the structure of the human body's key points, lines are drawn between each pair of key points. Finally, a 2D human posture is drawn on a blank canvas, and the posture point map is marked as follows. Compared to the original video frames V={V1,V2,V... i}, j≤i, for frames in the original video where there are no people, no graph will be drawn;

[0040] After obtaining the image sequence composed of continuous posture point maps, n point maps are selected as the standard movements of a certain track and field event, and the normalized point coordinates are saved as a JSON file for later use.

[0041] The 2D human posture in step (1) includes 17 points: 0: "nose", 1: "left eye", 2: "right eye", 3: "left ear", 4: "right ear", 5: "left shoulder", 6: "right shoulder", 7: "left elbow", 8: "right elbow", 9: "left wrist", 10: "right wrist", 11: "left hip", 12: "right hip", 13: "left knee", 14: "right knee", 15: "left ankle", 16: "right ankle".

[0042] The key point of this technical solution is:

[0043] 1. Based on the analysis of real-world motion videos, 2D human poses are extracted, ensuring that the extracted poses conform to the key structures of the real human body. This avoids the tedious process of building a solution from scratch, improving efficiency. It also allows for custom modifications based on existing motion models, giving it strong scalability.

[0044] 2. A new human key point extraction model M is proposed. e This model possesses multi-scale extraction capabilities, can adapt to images of different sizes, and exhibits greater robustness and accuracy compared to other approaches. Furthermore, the proposed window self-attention model effectively reduces computational resources, making it easy to deploy on lightweight devices.

[0045] 3. This method, by employing artificial intelligence technology and learning from data, avoids the drawbacks of manually designing features, thus possessing excellent generalization ability. This method can not only be used in track and field but can also be extended to other scenarios such as teaching ball sports.

[0046] The beneficial effects of this technical solution are as follows:

[0047] This technical solution fully considers the structure of real human joints and utilizes advanced artificial intelligence technology to more conveniently and effectively construct human postures. This solution not only saves computational costs but also allows for analysis of videos of different sizes and permits the creation of custom-designed standard motion schemes.

[0048] This technical solution employs a simple yet efficient AI-driven approach in track and field motion analysis, resulting in significant advancements. This innovation transcends the limitations of traditional motion capture techniques, which typically require extensive manual annotation and high computational resources. By integrating deep learning algorithms, the system automatically identifies and tracks athlete movements within video frame sequences, drastically reducing the computational load and time required for analysis. This efficiency enables kinesiology researchers and coaches to quickly gain detailed insights into athlete performance without the need for expensive motion capture equipment. Frame-by-frame analysis tasks that previously took tens of minutes can now be completed in minutes, providing coaches and athletes with immediate feedback.

[0049] Another key advantage of this technical solution lies in its robustness and adjustability. Unlike traditional methods that struggle to handle videos of varying qualities or require predefined video formats, this solution can seamlessly adapt to the size of the input video, ensuring that athletes and coaches can utilize video captured by a wide range of devices, from professional-grade cameras to smartphones. Furthermore, the system's modular design allows for the customization of standard movement templates. Coaches can tailor these templates to the specific needs and goals of individual athletes or teams, focusing on optimizing performance in the areas most critical to success. Attached Figure Description

[0050] Figure 1 This is a 2D human posture feature point map in the embodiment;

[0051] Figure 2 Flowchart for an embodiment;

[0052] Figure 3 The video is a reference image taken for the example.

[0053] Figure 4 This is a video frame and its corresponding 2D human pose feature point map in the example;

[0054] Figure 5 This example demonstrates the construction of a visual custom standard action diagram;

[0055] Figure 6 This is a 2D human posture feature point map before modification in the example;

[0056] Figure 7This is a modified 2D human posture feature point map in the embodiment. Detailed Implementation

[0057] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the invention.

[0058] Example:

[0059] Reference Figure 2 A method for constructing standard movements in track and field based on video analysis includes the following steps:

[0060] (1) Constructing a 2D human motion model: The first step is to use the object detection model detr, i.e., M d The first step involves detecting human figures in the image and cropping the human information to obtain an image containing only the human body. The second step utilizes a self-built human pose estimation model, namely M... e The analysis of 2D human motion videos yields 2D human poses, specifically including the following steps:

[0061] S1. Divide the video into consecutive frames: Upload a video of human movement, which can be any track and field video, such as... Figure 3 As shown, moving human figures in the video must be clearly visible; the straight-line distance between the shooting device and the human figure must be 3-6 meters; the shooting frame rate must be above 120 frames per second, and the shooting resolution must be above 1080p. The video is labeled V, and the video V is divided into continuous frames using image processing technology: V = {v1, v2, ..., v...} i}, v i Indicates the i-th frame;

[0062] S2. Human body location detection: Obtain the location of the human body and simultaneously crop out an image containing only the human body (the cropped human body location is determined by model M). d The output consists of the coordinates of the top left and bottom right corners of the human body.

[0063] S2-1. Image Processing: For the obtained video frame v i First, change v i The colors are arranged in RGB order, resulting in a three-channel color feature image, labeled P, where P∈R. H×W×3 H represents the height of the image, and W represents the width of the image;

[0064] S2-2. Extracting Human Body Location: Using Model M d The human body's position in image P is detected and marked with candidate bounding boxes. The top-left corner of each candidate bounding box is (x1, y1), and the bottom-right corner is (x2, y2). Finally, the image is cropped based on the candidate bounding box position information, and the background (excluding the human body) is removed to obtain feature map I. The formula is as follows:

[0065] (x1, y1, x2, y2) = M d (P) (1)

[0066] Model M d It consists of three steps: feature extraction, self-attention, and candidate box prediction. The formula for the first step, feature extraction, is as follows:

[0067] F = CNN(P) (2)

[0068] CNN stands for Convolutional Neural Network. By inputting P into a CNN, it can effectively extract preliminary feature information of the human body in an image, F∈R. H′×W′×C C represents the number of channels in the feature map. In the second step of self-attention, F is flattened and then fed into the self-attention process through position encoding to extract human feature information to the greatest extent, as shown in formula (3):

[0069] F′=PositionalEncoding(Flatten(F)) (3)

[0070] R = Transformer(F′) (4)

[0071] R represents the output of the self-attention algorithm, with a size of N×D, where N is a predefined upper limit on the number of targets (here, N is 1, indicating that only human bodies are detected); D is the output dimension for each target, including the category and candidate box coordinates. Finally, during candidate box prediction, R is transformed into candidate boxes B and corresponding category labels L. Invalid values ​​are removed through a linear layer and a specific activation function, as shown in the following formula:

[0072] B,L=Linear(R) (5)

[0073] B = Sigmoid(B)·[W,H,W,H] (6)

[0074] L = Softmax(L) (7)

[0075] Each element in B is mapped to the range [0,1] using the Sigmoid function. Finally, the coordinates are obtained by multiplying the height and width of the original image P. L uses the Softmax function to obtain the probability of each candidate box. The candidate box is only valid if it is higher than a certain threshold. Here, the threshold is set to 0.75, which means that the probability of the model predicting that the candidate box is a human body is greater than 0.75 before it is considered to be a human body.

[0076] S2-3. Image cropping: Finally, image I is obtained by cropping. The formula is as follows: CF represents the cropping function. The cropped image I is located in the region (x1,y1) to (x2,y2) of the original image P.

[0077] CF(P,(x1,y1),(x2,y2))=I (8)

[0078] S3. Extract key point features to obtain the coordinates of human joints and construct a 2D human pose feature map, i.e., a pose point map:

[0079] S3-1. Image Processing: To accurately identify key human body points, image I is processed through four different convolutional kernels to obtain four feature maps I1, I2, I3, and I4 of different sizes. I1, I2, I3, and I4 are then sequentially fed into model M. e Model M e By learning the outputs of four feature maps I1, I2, I3, and I4, a generalization ability to process images of different sizes is obtained. The shapes of the four feature maps are as follows:

[0080]

[0081] S3-2. Feature Extraction: Design a window-based self-attention module. This module segments the image, performs individual self-attention on each segment, and then aggregates the self-attention data. Specifically, for a given feature map I... i The dimensions are h×w, and I i According to size p h ×p w Cut to obtain A window is created; self-attention is applied to each window, and then the self-attention data is aggregated to obtain feature map I. i The formula is as follows:

[0082]

[0083] w j Let I represent the j-th window. However, such window division leads to a lack of information exchange between windows, resulting in poor recognition of human key points. Therefore, a cross-window information exchange module was designed to process the feature map I after aggregation and self-attention. i The cross-window information exchange module, which provides supplementary information, consists of three convolutional kernels, as shown in the following formula:

[0084]

[0085] The window self-attention module and the cross-window communication module are combined into a feature extraction module. For each feature map, there is a corresponding feature extraction module, which are then concatenated, i.e., In(I i+1 =Out(I) i This allows different feature maps to share parameters and feature information.

[0086] S3-3. Feature Fusion: After the feature extraction module, four feature maps are obtained after feature extraction. By using four feature maps By unifying the dimensions and performing feature fusion, the model's ability to extract key points of the human body is further improved. The formula is as follows:

[0087]

[0088] The final feature obtained is a shape of A heatmap representing Q key points;

[0089] S3-4. Heatmap Decoding: Through the steps S3-1 to S3-3 above, a heatmap representing key points is obtained, where Q is set to 17, representing seventeen human body key points. The heatmap is decoded using a heatmap decoder to obtain the corresponding key points.

[0090]

[0091] (2) Customizing Standard Motion Pose: First, normalize the 17 key points to obtain standardized coordinates. Then, based on the structure of the human body's key points, connect each key point in pairs, and finally draw a 2D human pose on a blank canvas, as shown in the image. Figure 4 As shown, the attitude point map is marked as Compared to the original video frames V={V1,V2,V... i}, j≤i, for frames in the original video where there are no people, no graph will be drawn.

[0092] After obtaining a sequence of images composed of continuous posture dot maps, n dot maps are selected as the standard movements for a certain track and field event, such as... Figure 5 As shown, the normalized point coordinates are saved as a JSON file for later use.

[0093] To represent a 2D human pose, a common practice is to draw the pose using feature point-line diagrams. Points represent prominent body parts such as shoulders, and line segments connect these points, ultimately drawing the pattern on a 2D plane. A standard 2D human pose should include at least the following 17 points: {0: "nose", 1: "left eye", 2: "right eye", 3: "left ear", 4: "right ear", 5: "left shoulder", 6: "right shoulder", 7: "left elbow", 8: "right elbow", 9: "left wrist", 10: "right wrist", 11: "left hip", 12: "right hip", 13: "left knee", 14: "right knee", 15: "left ankle", 16: "right ankle"}. The visual structure is as follows: Figure 1 As shown.

[0094] The human body possesses geometric and topological structural properties, with specific connections between different parts. These connections are also reflected in the points and lines derived from the abstract human body structure. If these relationships are not considered when drawing points and lines, the abstracted 2D human posture may not conform to realistic movement. Therefore, constructing a posture solution from scratch without considering actual movement is extremely difficult.

[0095] Based on the above, this example employs a video analytics approach to abstract human posture. By inputting a track and field video, and utilizing computer vision and artificial intelligence technologies, 2D human postures are extracted end-to-end from the video without any post-processing, thus constructing a 2D human motion model. Since the video is based on real-world footage of people, the extracted 2D human postures conform to anatomical structure.

[0096] By recording standard athletes' movement videos, this example can extract key technical movements such as hip extension and abdominal contraction in the straight-up long jump. Through data processing such as normalization, standard movements for track and field can be constructed. Although this video analysis-based standard movement construction method can meet most scenarios, due to subjective factors such as differences in athlete technique, there will still be instances where certain key movements are not entirely standard, such as... Figure 6 As shown. Based on the above shortcomings, this example also designs a supplementary solution for keypoint adjustment. This solution allows for pixel-level adjustment of keypoint positions through visual operations such as dragging the mouse, thereby meeting the needs for fine-tuning existing actions or customizing standard actions, such as... Figure 7 As shown.

[0097] References:

[0098] [1]Toshev A, Szegedy C.Deeppose: Human pose estimation via deep neural networks[C] / / Proceedings of the IEEE conference on computer vision and patternrecognition.2014:1653-1660.

[0099] [2]Luo Y, Ren J, Wang Z, et al. Lstm pose machines [C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 5207-5215.

[0100] [3]Liu Z,Chen H,Feng R,et al.Deep dual consecutive network for humanposeestimation[C] / / Proceedings of the IEEE / CVF conference on computer visionand patternrecognition.2021:525-534.

[0101] [4]Wu C,Wei X,Li S,et al.MSTPose:Learning-Enriched Visual InformationwithMulti-Scale Transformers for Human Pose Estimation[J].Electronics,2023,12(15):3244.

[0102] [5]Li J,Liu K,Wu J.Ego-body pose estimation via ego-headposeestimation[C] / / Proceedings of the IEEE / CVF Conference on Computer Visionand PatternRecognition.2023:17142-17151.

[0103] [6]Kundu J N,Seth S,YM P,et al.Uncertainty-aware adaptation for self-supervised 3dhuman pose estimation[C] / / Proceedings of the IEEE / CVF conferenceon computer visionand pattern recognition.2022:20448-20459。

Claims

1. A method for constructing standard movements in track and field based on video analysis, characterized in that, Includes the following steps: (1) Constructing a 2D human motion model: The first step is to use the target detection model detr, i.e. The first step involves detecting human figures in the image and cropping the human information to obtain an image containing only the human body; the second step utilizes a self-built human pose estimation model, i.e. The analysis of 2D human motion videos yields 2D human poses, specifically including the following steps: S1. Divide the video into consecutive frames: Upload a video of human movement, which can be any track and field event video. The moving human body in the video must be clear; the straight-line distance between the shooting device and the human body should be 3-6 meters; the shooting frame rate should be above 120 frames per second, and the shooting resolution should be above 1080p. The video should be tagged as... Through image processing technology, video Divided into consecutive frames: , Indicates the first frame; S2. Human Body Position Detection: Obtain the position of the human body and simultaneously crop out an image containing only the human body. The cropped human body position is determined by the model. The output consists of the coordinates of the top left and bottom right corners of the human body; S2-1. Image Processing: For the obtained video frames First change The colors are arranged in RGB order, resulting in a three-channel color feature image, labeled as follows. , , Represents the height of the image. Represents the width of the image; S2-2. Extracting human body location: using a model The detection of human body in the image The positions in the text are marked with candidate boxes, and the coordinates of the top-left corner of the candidate box are... The coordinates of the bottom right corner of the candidate box are Finally, the image is cropped based on the position information of the candidate boxes, and the background (excluding the human body) is removed to obtain the feature map. The formula is as follows: Model It consists of three steps: feature extraction, self-attention, and candidate box prediction. The formula for the first step, feature extraction, is as follows: To represent a specific convolutional neural network, by... enter In this process, it can effectively extract preliminary feature information of the human body in an image. , This represents the number of channels in the feature map; next, in the second step of self-attention, After being flattened, the information is fed into the self-attention system through positional encoding to extract human feature information to the greatest extent, as shown in formula (3): This represents the result of self-attention output, with a size of ,in It is a predefined upper limit for the target quantity, here A value of 1 indicates that only human bodies are detected; This is the output dimension for each target, including the category and candidate box coordinates; finally, during the candidate box prediction process, Converted into candidate boxes and corresponding category labels Invalid values ​​are removed using a linear layer and an activation function, as shown in the following formula: Each element in the data is passed through The function is mapped to Within the range, finally multiply by the original image. The coordinate values ​​can be obtained by calculating the height and width. pass The function obtains the probability of each candidate box. Only candidate boxes with a probability higher than a threshold are valid. Here, the threshold is set to 0.75, which means that the model predicts that the probability of a candidate box being a human body is greater than 0.75 before it is considered to be a human body. S2-3. Image Cropping: Finally, the image is obtained by cropping. The formula is as follows: This represents the cropping function, which describes the cropped image. Located in the original image of arrive within the area; S3. Extract key point features to obtain the coordinates of human joints and construct a 2D human pose feature map, i.e., a pose point map: S3-1. Image Processing: To accurately identify key human body points, the image... Four feature maps of different sizes are obtained by using four different convolution kernels. Feature map They were fed into the model in sequence. ,Model Through learning The output of the four feature maps is used to obtain the generalization ability to process images of different sizes. The shapes of the four feature maps are as follows: S3-2. Feature Extraction: Design a window-based self-attention module. This module segments the image, performs individual self-attention on each segment, and then aggregates the self-attention data. Specifically, for a given feature map... The size is ,Will By size Cut to obtain Each window is processed with self-attention; then the self-attention is aggregated to obtain a feature map. The formula is as follows: Indicates the first However, dividing the screen into individual windows leads to a lack of information exchange between them, resulting in poor recognition of key human features. Therefore, a cross-window information exchange module was designed to process the feature maps after aggregation and self-attention. To supplement information, the cross-window information exchange module consists of three convolutional kernels, as shown in the following formula: The window self-attention module and the cross-window communication module are combined into a feature extraction module. For each feature map, there is a corresponding feature extraction module, which are then connected in a concatenated manner. This allows different feature maps to share parameters and feature information; S3-3. Feature Fusion: After the feature extraction module, four feature maps are obtained after feature extraction. By using four feature maps By unifying the dimensions and performing feature fusion, the model's ability to extract key points of the human body is further improved. The formula is as follows: The final feature obtained is a shape of ,express Heatmap of key points; S3-4. Heatmap Decoding: Through steps S3-1 to S3-3 above, a heatmap representing the key points is obtained, where... Setting it to 17 represents seventeen key points on the human body. A heatmap decoder is used to decode the heatmap to obtain the corresponding key points. (2) Custom standard motion posture: First, the 17 key points are normalized to obtain standardized coordinates. Then, based on the structure of the human body key points, the key points are connected in pairs, and a 2D human posture is drawn on a blank canvas. The posture point map is marked as follows. Compared to the original video frame , For frames in the original video where there are no people, no graphs will be drawn; After obtaining the image sequence consisting of continuous attitude point maps, a custom selection is made of it. A point graph is used as a standard action in a certain track and field sport, and the normalized point coordinates are saved as a JSON file for later use.

2. The method for constructing standard movements in track and field based on video analysis according to claim 1, characterized in that, The 2D human posture in step (1) includes 17 points: 0: "nose", 1: "left eye", 2: "right eye", 3: "left ear", 4: "right ear", 5: "left shoulder", 6: "right shoulder", 7: "left elbow", 8: "right elbow", 9: "left wrist", 10: "right wrist", 11: "left hip", 12: "right hip", 13: "left knee", 14: "right knee", 15: "left ankle", 16: "right ankle".

Images