Motion evaluation method and device, electronic equipment and storage medium

Abstract

The invention provides an exercise evaluation method and device, electronic equipment and a storage medium, and belongs to the technical field of exercise evaluation.The method comprises the steps that in the preparation stage, a preset human shape detection model is called to detect an input image frame so as to position a human body frame, and a tester located in a take-off area is determined according to the human body frame; in the take-off stage, a preset human body posture estimation model and a preset key frame positioning model are called, and whether the tester starts to take off or not is judged, so that a take-off frame is determined; in the flying stage and the landing stage, based on the take-off frame, the human body posture estimation model and the key frame positioning model are called, whether the tester starts to land or not is judged so as to determine a landing frame, the landing point of the tester is determined according to the landing frame, and the standing long jump score is calculated. Through automatic and intelligent technical means, the problems existing in traditional standing long jump evaluation are effectively solved, and a more accurate, objective and convenient evaluation scheme is provided.

Description

Technical Field

[0001] The present application relates to the technical field of motion evaluation, and in particular to a motion evaluation method, device, electronic device and storage medium. Background Art

[0002] The existing standing long jump assessment usually relies on manual measurement by physical education teachers or the use of infrared measuring equipment. Manual measurement involves teachers determining the long jump distance based on visual inspection or the use of measuring tools, while infrared equipment uses a complex system to capture and calculate the long jump results. These two methods are widely used in physical education to ensure the accuracy and fairness of student physical fitness tests.

[0003] However, there are some problems with the existing standing long jump evaluation. First, manual measurement is easily affected by the subjective judgment of teachers, which may lead to differences in the scores of the same test takers. Secondly, although infrared measurement equipment is relatively accurate, it is complex and bulky to deploy, which is not conducive to the simplicity and flexibility of physical education. In addition, the computer vision-based evaluation schemes that have emerged in recent years, although they have utilized advanced technology, often rely on single-frame posture comparisons, and the key frame positioning accuracy is insufficient, resulting in large errors in performance measurement, making it difficult to meet the requirements of high-precision national standards. These problems affect the accuracy and efficiency of standing long jump evaluation. Summary of the invention

[0004] The present application provides a motion evaluation method, device, electronic device and storage medium, aiming to solve the problems of subjectivity of manual measurement, complexity of infrared equipment and insufficient key frame positioning accuracy of computer vision solutions in standing long jump evaluation.

[0005] In a first aspect, the present application provides a motion evaluation method, which is applied to standing long jump, covering the preparation stage, take-off stage, flight stage and landing stage, and the motion evaluation method comprises:

[0006] In the preparation stage, a preset human figure detection model is called to detect the input image frame to locate the human body frame, and the test person located in the take-off area is determined according to the human body frame;

[0007] In the take-off phase, the preset human posture estimation model and the preset key frame positioning model are called to determine whether the test person has started to jump, so as to determine the take-off frame;

[0008] In the flight phase and the landing phase, based on the take-off frame, the human body posture estimation model and the key frame positioning model are called to determine whether the test person has started to land, so as to determine the landing frame, and the landing point of the test person is determined according to the landing frame to calculate the standing long jump result.

[0009] In a second aspect, the present application further provides a sports evaluation device, which is applied to standing long jump, covering the preparation stage, take-off stage, flight stage and landing stage, and the sports evaluation device includes:

[0010] A tester positioning module is used to call a preset human shape detection model to detect the input image frame in the preparation stage to locate the human body frame, and determine the tester located in the take-off area according to the human body frame;

[0011] The take-off detection module is used to call a preset human posture estimation model and a preset key frame positioning model during the take-off phase to determine whether the test person has started to take off, so as to determine the take-off frame;

[0012] A landing detection module is used to call the human posture estimation model and the key frame positioning model based on the take-off frame during the flight stage and the landing stage to determine whether the test person has started to land, so as to determine the landing frame;

[0013] The score calculation module is used to determine the landing point of the tester according to the landing frame and calculate the standing long jump score.

[0014] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, the steps of the motion assessment method as described in any one of the first aspects are implemented.

[0015] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the motion assessment method as described in any one of the first aspects.

[0016] The present application provides a motion evaluation method, device, electronic device and storage medium, which are intended to solve the problems of subjectivity of manual measurement, complexity of infrared equipment and insufficient accuracy of key frame positioning of computer vision solutions in standing long jump evaluation. The method can objectively identify and locate the key actions of the tester in the standing long jump process, such as take-off and landing, through an automated human detection model and a human posture estimation model. In addition, by using a preset computer vision model, it only relies on image input, such as video frames, without the need for additional high-cost hardware equipment, simplifying the deployment and maintenance of the evaluation system. The key frame positioning model is used to accurately identify and locate key action frames in the standing long jump process, such as take-off frames and landing frames. By combining human posture estimation and key frame positioning technology, subtle changes in the tester's movements can be more accurately captured, thereby improving the accuracy of the evaluation.

[0017] Therefore, the present application effectively solves the problems existing in the traditional standing long jump evaluation through automated and intelligent technical means, and provides a more accurate, objective and convenient evaluation solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 It is one of the flow charts of the motion evaluation method provided by this application;

[0020] Figure 2 This is the second flow chart of the motion evaluation method provided by this application;

[0021] Figure 3 is a schematic diagram of a calibrated sports venue provided in an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of a first key frame positioning model provided in an embodiment of the present application;

[0023] Figure 5 is a schematic diagram of a second key frame positioning model provided in an embodiment of the present application;

[0024] Figure 6 is a schematic diagram of the state machine decision logic provided in an embodiment of the present application;

[0025] Figure 7a is a schematic diagram of a target tracking model provided in an embodiment of the present application;

[0026] Figure 7b is a schematic diagram of a human body frame and key points of a human body provided in an embodiment of the present application;

[0027] Figure 8 It is a schematic diagram of calculating standing long jump results provided by an embodiment of the present application;

[0028] Fig. 9 It is a structural schematic diagram of the sports evaluation device provided by the present application;

[0029] Fig.10 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0032] In order to solve the problems of subjectivity of manual measurement, complexity of infrared equipment, and insufficient accuracy of key frame positioning of computer vision solutions in standing long jump evaluation, the present application provides a motion evaluation method, device, electronic device and storage medium. The method integrates human body detection, posture estimation and key frame positioning technology to achieve automated evaluation of the entire process of standing long jump. It not only eliminates the subjectivity of manual measurement, but also avoids the limitations of relying on complex infrared equipment, while improving the accuracy of key action frame positioning. Through the comprehensive application of these technologies, the present application provides an objective, accurate and easy-to-implement evaluation solution for standing long jump, thereby optimizing the evaluation process and improving the reliability of the evaluation results.

[0033] Combine the following Figure 1-Figure 10 The present application describes a motion assessment method, device, electronic device and storage medium.

[0034] Please refer to Figure 1 , Figure 2 , Figure 1 is a flow chart of the motion evaluation method provided by this application, Figure 2 This is the second flow chart of the motion evaluation method provided by this application. A motion evaluation method is applied to standing long jump, covering the preparation stage, take-off stage, flight stage and landing stage, and the motion evaluation method includes:

[0035] S110, in the preparation stage, calling a preset human figure detection model to detect the input image frame to locate the human body frame, and determining the test person located in the take-off area according to the human body frame.

[0036] Specifically, the current frame can be obtained from a video stream or a series of images as input. The preset human detection model is called to analyze the input image frame. The human detection model can be a target detection algorithm based on deep learning, such as YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector) or Faster R-CNN (a deep learning model for target detection), etc., which can identify multiple human bodies in the image and give the position of each human body in the image, that is, a rectangular frame surrounding the human body, also known as a human body frame or a human body bounding box. According to the position information of the human body frame, it is determined whether these human bodies are located in the predefined take-off area. The take-off area is a specific ground area where the tester prepares and takes off. The purpose of this step is to automatically identify and locate the human bodies in the video or image, and further determine whether these human bodies are located in the specified take-off area, so as to provide basic data for the subsequent take-off stage, the air stage and the landing stage evaluation, which can reduce the need for manual monitoring and marking, and improve the efficiency and accuracy of the evaluation.

[0037] S120, in the take-off phase, calling a preset human posture estimation model and a preset key frame positioning model to determine whether the test person has started to jump, so as to determine a take-off frame.

[0038] Specifically, the preset human posture estimation model is called to analyze the posture of the tester. The model can identify and track the key points of the human body, thereby estimating the posture and action of the tester. At the same time, the preset key frame positioning model is called to identify the key frames in the video. Key frames refer to frames that can represent important action changes, such as the moment when the tester takes off. Combining the outputs of the human posture estimation model and the key frame positioning model, the system determines whether the tester has started to jump, for example, by identifying specific action patterns, such as acceleration of the legs and body, and leaving the ground. Once the system determines that the tester has started to jump, the key frame positioning model will further determine the specific frame where the jump occurs, that is, the take-off frame. The take-off frame is used for subsequent analysis, such as calculating the distance of the jump and evaluating the compliance of the action. The purpose of this step is to automatically capture the key moment of the tester's take-off, provide accurate time points for the subsequent evaluation of the air and landing stages, improve the degree of automation of the evaluation, and reduce the subjectivity and errors of manual judgment.

[0039] S130, during the flight phase and the landing phase, based on the take-off frame, the human body posture estimation model and the key frame positioning model are called to determine whether the test person has begun to land, so as to determine the landing frame, and the landing point of the test person is determined according to the landing frame, and the standing long jump result is calculated.

[0040] Specifically, the system will continue to analyze the tester's movements starting from the previously determined take-off frame. During the take-off and landing phases, the human posture estimation model is used to track the tester's body posture and the location of key points. This helps the system understand the tester's movements and posture changes in the air. The keyframe positioning model is used to identify the important moment that marks the tester's landing, namely the landing frame. By analyzing the output of the human posture estimation model and the keyframe positioning model, the system will determine whether the tester has begun to touch the ground. Once the system confirms that the tester has begun to land, the keyframe positioning model will accurately determine the specific frame where the landing occurs, namely the landing frame. Based on the tester's position information in the landing frame, the system will determine the tester's landing point. For example, identify the location where the tester's feet touch the ground. Finally, the system will calculate the standing long jump score based on the tester's landing point. For example, by measuring the straight-line distance between the take-off point and the landing point, necessary adjustments can be made according to the rules of the competition. The purpose of this step is to track the tester's landing movements through automation technology and calculate the tester's score based on this, which improves the accuracy and efficiency of the evaluation and reduces the need for manual intervention.

[0041] In some embodiments, the method further comprises:

[0042] S140, after determining the tester located in the take-off area, calling a preset target tracking model, and starting the personnel tracking logic after the tester reaches a ready state.

[0043] Specifically, in the aforementioned step S110, a humanoid detection model has been used to identify and locate the tester in the take-off area. When the tester enters the ready state, that is, starts to make preparatory movements before taking off (such as squatting, swinging arms, etc.), the system will start the personnel tracking logic. This means that the system will begin to pay close attention to the tester's movements in order to accurately evaluate the tester when he takes off, flies into the air, and lands. The purpose of step S140 is to ensure that before the tester starts the key action of standing long jump, the system is ready and able to continuously track the position and movement of the tester for subsequent evaluation and analysis, thereby improving the continuity and accuracy of the evaluation and reducing the risk of tracking loss due to the movement of the tester.

[0044] The above steps S110 to S140 are described in detail below.

[0045] In some embodiments, the motion assessment method further comprises:

[0046] S10, training a target detection model to calibrate the sports field, the target detection model includes a first detection model and a second detection model, and the training steps include S11 to S14:

[0047] S11, acquiring standing long jump images, and using a preset annotation tool to annotate each scale point in each image to form dense annotations.

[0048] Specifically, we need to prepare a dataset for training automatic calibration of standing long jump. This dataset requires using annotation tools (such as labelme) in each image to mark each scale point with a rectangular box.

[0049] S12, training a preset first detection model to detect four vertices of the sports field, wherein the vertices are key points for determining the position of the field.

[0050] Specifically, a preset first detection model (such as a YOLOX model) is used to perform field detection on the image, with the goal of detecting four vertices corresponding to the sports field.

[0051] S13, cropping the image according to the detected four vertices to extract the sports field.

[0052] Specifically, the image is cropped based on the four detected vertices so as to extract the sports field from the original image, which helps to reduce background interference and make the subsequent dense annotation detection more accurate.

[0053] S14, training a preset second detection model to perform dense annotation detection on the cropped image and identify all scale points to complete the training of the target detection model.

[0054] Specifically, a preset second detection model (such as another YOLOX model) is used to perform dense annotation detection on the cropped image. The purpose of this step is to identify all scale points on the sports field, thereby obtaining a trained target detection model.

[0055] Therefore, through the above steps S11 to S14, the system can train a target detection model that can accurately detect and calibrate the standing long jump sports field. The model can be used to automatically identify and measure the distance of the standing long jump in practical applications.

[0056] In some embodiments, the method further includes a reasoning stage of the target detection model, wherein the reasoning steps include S20 to S22:

[0057] S20, based on the detection result of the second detection model, the detected scale point frames are sorted according to their positions in the image, and the sorted frames are divided into upper and lower rows of frames by identifying distance mutation points.

[0058] Specifically, the four vertices of the sports field are detected using the trained first detection model. According to the detected four vertices, the sports field is cropped from the original image. The cropped sports field is densely labeled and detected using the trained second detection model to identify all scale points and output a series of rectangular boxes, each of which corresponds to a detected scale point.

[0059] The detected rectangular boxes are sorted according to their positions in the image (such as x-axis and y-axis coordinates), which helps to identify and process the layout of these boxes. And in the sorted boxes, find those points where the positions on the x-axis or y-axis change significantly. These mutations mark the boundaries between rows of boxes. According to the distance mutation, the boxes are divided into upper and lower rows, and each row of boxes corresponds to a row of scale points in the standing long jump area.

[0060] Furthermore, for the upper and lower boxes, the intersection over union (IoU) between the boxes is calculated. If the IoU of two boxes exceeds a preset threshold, it is considered that the two boxes overlap too much and may be a false detection (false alarm), so one of the boxes is deleted.

[0061] S21, for the upper row and the lower row of boxes, count the width of each box and the size of the interval between the boxes, and traverse from left to right, and determine whether there is a missed box based on the width of the box, the size of the interval, and the distance between the two boxes; if there is a missed box, perform linear interpolation based on the information of the known boxes to complete the missing box.

[0062] S22, determining whether the number of completed boxes meets the preset requirement.

[0063] Specifically, after completing all missing boxes, a final check is performed to ensure that the number of completed boxes meets the set requirements of the scale points. For example, please refer to Figure 3 , Figure 3 : is a schematic diagram of calibrating a sports field provided in an embodiment of the present application. The setting requirement is 61 frames in the upper and lower frames, a total of 122 frames, to ensure that all scale points are correctly detected and marked.

[0064] Therefore, through the above steps S20 to S22, the output of the target detection model is optimized to ensure that all scale points are accurately identified and marked in dense point detection, while reducing false detection and missed detection.

[0065] In some embodiments, the method further comprises training a human shape detection model, a target tracking model, a human posture estimation model, and a landing point detection model, wherein:

[0066] S30, using a preset target detection framework, using a data set containing a first number of key points and labeled long jump scene human body frame data for training, to obtain a trained human figure detection model.

[0067] For example, based on the YOLOX target detection framework, the MS COCO (Microsoft Common Objects in Context) public dataset (with 17 key points) and the labeled long jump scene human frame data (with 30 key points) can be used for training. The trained model can detect the test person in the image and give a rectangular box of its position.

[0068] For example, the 30 key points mentioned above are: top of the head, nose, left ear, right ear, chin, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left palm, right palm, left middle finger, right middle finger, left hip, right hip, left knee, right knee, left ankle, right ankle, left heel, right heel, left toe, right toe, left eye, right eye, left thumb, right thumb.

[0069] S31, using a preset single target tracking model, a preset human body tracking data set and a labeled long jump scene motion data set for training to obtain a trained target tracking model.

[0070] For example, based on the OS-Track single target tracking model, the public human tracking datasets LaSOT (Large-scale Single Object Tracking, a large-scale single target tracking dataset) and GOT-10k (Generic Object Tracking Benchmark, a general target tracking benchmark dataset) are used for training, as well as the collected and labeled long jump scene motion dataset. The trained model can continuously track the position of the test person in the video sequence.

[0071] S32, based on a preset single-person posture estimation framework, use a data set containing a first number of key points and a labeled data set containing a second number of key points to perform mixed training to obtain a trained human posture estimation model.

[0072] For example, the SimDR (Similarity-based Domain Randomization, a technology used to enhance the generalization ability of machine learning models) single-person pose estimation framework based on the one-dimensional heat map paradigm uses the MS COCO public dataset (containing 17 key points) and the labeled long jump scene human skeleton point data (containing 30 key points) for mixed training. The trained model can output the human pose estimation results of 30 key points, which helps to analyze the test person's movements in more detail.

[0073] S33, using a preset key point detection framework and using the labeled heel key point data of the long jump scene for training to obtain a trained key point detection model.

[0074] For example, based on the RLEPose (a method that uses RLE encoding technology to represent or estimate the posture of objects (especially human bodies)) key point detection framework, the heel key point data of the long jump scene labeled by the experiment can be used for training. The trained key point detection model can detect the position of the heel of the test person when he lands.

[0075] Therefore, in the above steps S30 to S33, a series of models are trained by using different data sets and algorithm frameworks, which can detect, track and analyze the movements of standing long jump testers, thereby realizing automated sports evaluation. Each model is optimized for a specific task to improve the accuracy and efficiency of the evaluation.

[0076] In standing long jump, the landing frame refers to the video frame at the moment when the tester's feet touch the ground. The accurate capture of this video frame is crucial to calculating the tester's long jump score, because the score is calculated based on the distance between the take-off point and the landing point. If the image resolution of the camera is 1080 pixels vertically and 1920 pixels horizontally, that is, 1080×1920 resolution, and the frame rate is 25FPS (Frames Per Second), that is, 25 frames of images are taken per second. According to actual statistics, if there is a one-frame deviation in the positioning of the landing frame, it may cause an error of about 3 centimeters in the score calculation. This is because when 25 frames are taken in one second, the time interval represented by each frame is about 0.04 seconds, and the time that the tester is in the air is usually very short, so the deviation of each frame may cause a large distance error. Therefore, in order to find a stable and reliable key frame, the present application proposes an improved key frame positioning model, which is different from the existing solution that only relies on the movement speed of the human skeleton point to determine the take-off and landing. It adopts a multi-level cascade and multi-modal method, which is divided into two stages: coarse positioning stage (using the first key frame positioning model) and fine positioning stage (using the second key frame positioning model), as follows:

[0077] The goal of the coarse localization stage is high recall, that is, capturing as many possible key frames as possible. In order to speed up the inference, the first key frame localization model uses a human key point sequence (also known as a skeleton point sequence) as input. The key point sequence is obtained by analyzing the data of the changes in the position of human joints over time in the video. During the evaluation process, the coarse localization stage will be used multiple times to ensure that no key frames are missed.

[0078] The goal of the fine positioning stage is high accuracy, that is, to accurately locate the position of the key frame. The simple skeleton point input may be affected by the quality of skeleton point extraction and is not robust enough in some scenes (such as shadows, uneven lighting, etc.). Therefore, the second key frame positioning model uses multimodal input of key point sequence and image sequence, combining the motion information of skeleton points and the visual information of images to improve the robustness and accuracy of the model.

[0079] The first key frame positioning model and the second key frame positioning model are described below.

[0080] In some embodiments, the method further comprises a reasoning stage of a key frame positioning model, wherein the key frame positioning model comprises a first key frame positioning model and a second key frame positioning model, and the reasoning steps thereof comprise S40 and S41:

[0081] S40, the first key frame positioning model is used to identify whether a key frame is included; it uses the normalized key point sequence as input and predicts the action category through posture encoding, time sequence fusion and action classification.

[0082] Specifically, key gestures such as take-off and landing are considered micro-motions because they last very short periods of time in the video, usually only 2-3 frames. For example, the landing action from the toes touching the ground to the soles of the feet flattening out may only take about 3 frames in a 25FPS (25 frames per second) video. The first key frame localization model abstracts the key frame localization problem into a micro-motion recognition problem, that is, determining what micro-motion category the input posture estimation sequence corresponds to. The input of the first key frame localization model is a normalized sequence of key points, which represent the postures of the human body in different frames of the video.

[0083] Please refer to Figure 4 , Figure 4: is a schematic diagram of the first key frame positioning model provided by an embodiment of the present application. The first key frame positioning model includes a posture encoding module, a temporal fusion network and an action classification head. First, the posture encoding module encodes the key points of each frame into a high-dimensional vector (called a token), i.e., a key point vector, which helps to capture the spatial features of the key points. The posture encoding module is a multi-layer perceptron. Secondly, the input high-dimensional vector sequence (tokens) of each frame, together with a learnable category vector token, are input into a temporal fusion network. The temporal fusion network is a multi-layer Transformer (based on self-attention mechanism) encoder. The temporal fusion network is responsible for fusing the temporal information, i.e., the dynamic changes between different frames, and outputting the category vector token. Finally, the category vector token is input into an action classification head. The action classification head is also a multi-layer perceptron, and the action categories can include take-off, landing and others. The action classification head is responsible for predicting the final action category, i.e., the action category of the key posture. For example, the action classification head can determine whether the input sequence contains micro-actions such as take-off and landing.

[0084] Exemplarily, the keypoints of each frame are normalized as follows:

[0085] x min =min(x 1，1 , x 1，2 , …, x 1，n , …, x 1，30 );

[0086] y min =min(y 1，1 ,y 1，2 , …, y 1，n , …, y 1，30 );

[0087]

[0088]

[0089] Among them, x 1,n and 1,n Respectively represent the horizontal and vertical coordinates of the nth key point in the first frame; and Respectively represent the horizontal and vertical coordinate sets of all key points in the tth frame; W and H represent the horizontal and vertical resolutions of the original image respectively; and Represents the normalized horizontal and vertical coordinate sets of all key points in the tth frame.

[0090] S41, the second key frame positioning model is used to locate the position of the key frame; it uses multimodal input of key point sequence and image sequence to predict the position of the key frame through posture coding, image coding, modal fusion, temporal fusion and key frame multi-classification.

[0091] Specifically, the first key frame positioning model can roughly determine whether the current video clip contains a key frame (such as a take-off or landing frame), but cannot accurately locate the specific position of the key frame. Since it only relies on key point segments as input, when the quality of key points is poor (for example, in insufficient lighting or occlusion), it is difficult for the model to accurately locate the position of the landing frame.

[0092] In order to improve the accuracy of positioning, the fine positioning stage requires more information, including venue information and detailed descriptions of human movements. Therefore, in addition to key point information, image modality input is also required. However, relying solely on image sequences as input is challenging because there may be a lack of effective feature alignment between images, which makes it difficult to quickly locate motion information. Therefore, key point information is used as an aid to help the model better understand motion patterns.

[0093] The key frame positioning in the fine positioning stage is abstracted into a multi-classification problem. Specifically, the second key frame positioning model receives a time series information containing T frames (including key point sequence and image sequence) and outputs the position t of the frame where the key frame is located, where t is an integer between 0 and T-1. The goal of this multi-classification problem is to determine the specific position of the key frame in the input sequence, thereby improving the accuracy of the evaluation.

[0094] Please refer to Figure 5 , Figure 5 : is a schematic diagram of the second key frame positioning model provided in an embodiment of the present application. The second key frame positioning model includes a posture encoding module, an image encoding module, a modal fusion module, a temporal fusion network and a key frame classification head. First, the posture encoding module is used to encode the key point sequence into a high-dimensional vector, namely the key point vector, denoted as skeleton token, representing the skeleton point modality. At the same time, the model uses the image encoding module to encode the image sequence into a high-dimensional vector, namely the image vector, denoted as image token, representing the image modality.

[0095] Next, the model uses a modal fusion module to cross-modally fuse the key point vector (skeleton token) with the image vector (image token). In order to better align the different modal information of each frame input during the fusion process, the modal fusion module performs a splicing operation, namely: multi-modal token = concat (skeleton token, image token). This means that the skeleton point information and the image information are spliced ​​into a single high-dimensional vector, namely a multi-modal vector, denoted as multi-modal token. The multi-modal vector contains information from two modalities.

[0096] Then, the multi-modal vector multi-modal token and a learnable category vector token are input into the time series fusion network for time series feature fusion. In this process, the category vector token fully integrates the multi-modal information and time series information through the attention mechanism to capture the features of the key frames and output the category vector token.

[0097] Finally, the category vector token is input to the key frame multi-classification head, and the final key frame position is output. In this application, the number of multi-classification categories is equal to the input time series length, indicating the frame number of the key frame in the input sequence. For example, if the input sequence has T frames, then the second key frame positioning model will predict the specific position of the key frame in frames 0 to T-1.

[0098] It can be seen that the first key frame localization model abstracts the key frame localization problem into a classification problem, that is, judging whether the current video clip contains a key frame (such as a take-off or landing frame). By simplifying the problem into a classification problem, the model reduces the difficulty of learning, thereby being able to obtain a classification model with high recall rate (that is, detecting as many key frames as possible) and high accuracy. The first key frame localization model is used for preliminary screening to avoid missing key frames.

[0099] The second keyframe positioning model focuses more on the subtle differences between frames, especially those features that can indicate keyframes (such as landing frames). By leveraging the multimodal information of skeleton points and images, the model can more accurately identify and locate landing frames. The second keyframe positioning model aims to provide more accurate keyframe locations, thereby providing a more reliable and accurate basis for subsequent score calculations. The two models work together to ensure the accuracy and reliability of the entire evaluation process.

[0100] In some embodiments, the motion evaluation method uses a state machine to perform logic determination on the preparation phase, take-off phase, flight phase, and landing phase, and the logic determination steps include:

[0101] S50, in the preparation stage, the human detection model is called at regular time intervals to identify the human body in the video frame sequence and screen out the human body rectangular frame located in the take-off area; if it is detected that there is only one test person in the take-off area for multiple consecutive times, it is determined that the test person enters the preparation state.

[0102] Please refer to Figure 6 , Figure 6 : is a schematic diagram of the state machine judgment logic provided by an embodiment of the present application. The system is in a waiting state at the beginning, that is, waiting for the test person to be ready for long jump. For example, the system calls the human detection model at a detection interval of 25 frames at a certain time interval, and the model is used to identify the human body in the video. According to the output results of the human detection model, the system filters out the rectangular boxes of people in the take-off area, that is, determines which people are in the take-off area. When three consecutive detection results show that there is only one person in the take-off area, the system determines that it has entered the ready state, that is, the test person is ready to take off.

[0103] S51, in the take-off stage, when the tester is ready to take off, the first key frame positioning model is used to identify whether a take-off action occurs; if a take-off action is identified, the second key frame positioning model is switched to locate the take-off frame.

[0104] Specifically, the take-off state is first determined using the first key frame positioning model in the coarse positioning stage. When the classification result of the first key frame positioning model is the take-off category, the system switches to the second key frame positioning model in the fine positioning stage to accurately locate the take-off frame. The take-off frame is used to determine whether the tester violates the rules when taking off. For example, the positioning of the take-off frame is used to determine whether there are violations such as stepping on the line during the evaluation process.

[0105] S53, in the flight phase and the landing phase, using the first key frame positioning model to identify whether a landing action occurs; if a landing action is identified, switching to the second key frame positioning model to locate the landing frame.

[0106] Specifically, the judgment rules for the landing state are similar to those for the take-off state, and both use a cascade model for judgment. The first key frame positioning model in the coarse positioning stage is used to preliminarily judge the landing state. The second key frame positioning model in the fine positioning stage is used for accurate landing frame positioning. The landing frame is used to determine whether the tester violates the rules when landing.

[0107] In some embodiments, in step S140, the steps of the personnel tracking logic include:

[0108] S141, when the test person is in the take-off area, capture his initial image and set it as a tracking template; continuously obtain the current frame from the video stream as the test area, and apply the template matching algorithm in the current frame, and perform position retrieval based on the tracking template; and locate the position of the test person in the current frame according to the output result of the template matching algorithm, and output the human body frame.

[0109] Specifically, personnel tracking is critical to ensuring the accuracy of the test results. If the tracking is wrong, it may lead to incorrect landing frame and landing point detection, thereby calculating an incorrect test result. After the system determines that the tester has reached the ready state, the personnel tracking logic will be turned on to ensure that the tester is accurately tracked throughout the long jump process.

[0110] Traditional tracking solutions are usually based on multi-person pose estimation. Each frame requires pose estimation of all people in the picture, and then post-processing (such as field matching and frame matching) is performed to track the test person. Traditional solutions are prone to mistracking when there are a large number of onlookers because they need to process a large amount of data and perform complex matching.

[0111] Please refer to Figure 7a , Figure 7b , Figure 7a is a schematic diagram of a target tracking model provided in an embodiment of the present application, Figure 7b is a schematic diagram of a human frame and key points of a human body provided in an embodiment of the present application. The target tracking model of the present application adopts a single target tracking model to improve the reliability of tracking. The use of a single target tracking model can simplify the tracking process because it only needs to focus on a single target, rather than all the people in the picture, such as Figure 7b Specifically, Figure 7a As described, when the test person is located in the take-off area, the system captures its initial image, and the target tracking model uses a template as a tracking module, and the tracking template will be used as a reference for retrieving the position of the test person in subsequent frames. The system continuously obtains the current frame from the video stream, and these frames will be used as the test area (search image) to retrieve the position of the test person in each frame. The template matching algorithm is applied in the current frame, and the position retrieval is performed based on the tracking template. The template matching algorithm compares the image in the current frame with the tracking template to determine the position of the test person in the current frame. According to the output result of the template matching algorithm, the system locates the position of the test person in the current frame and outputs a bounding box, that is, a rectangular box surrounding the test person.

[0112] It can be seen that the use of a single target tracking model ensures the accuracy of tester tracking, simplifies the complexity of the entire system, and improves the operating efficiency of the system.

[0113] In some embodiments, in step S130, the step of determining the landing point of the tester according to the landing frame and calculating the standing long jump score includes:

[0114] S131, obtaining all scale points on the sports field.

[0115] For example, based on the field calibration algorithm, the system can obtain a scale distribution with an interval of 5 cm in the range of 0-300 cm. These scales are used to determine the specific location of the test person's landing point on the field.

[0116] S132, in the landing frame, determine the initial score interval according to whether the tester's foot contour point is located within a certain scale interval.

[0117] Specifically, in the ground frame, the system uses the tester's foot contour points to determine whether it is within a large score range. This is the initial score range determination, which is used to determine the approximate score range.

[0118] S133, according to the distance between the foot contour point and the left and right scale lines of the interval, the score within the small interval is calculated according to the strategy of equal proportion division; and the final score is determined by accumulating the scores of the large interval and the small interval.

[0119] That is, the score is calculated proportionally based on the relative position of the landing point between the scale lines.

[0120] The following example illustrates how to calculate the standing long jump score.

[0121] Suppose we have a standing long jump field with scale lines from 0 cm to 300 cm, with a scale every 5 cm. After the tester long jumps, the system needs to calculate his final score.

[0122] Assume that the coordinates of the test person's landing point are 123 cm.

[0123] In the landing frame, the system determines the initial score range based on whether the tester's foot contour points are within a certain scale range. For example, the landing point of 123 cm is between the scale lines 120 cm (x1) and 125 cm (x2).

[0124] According to the distance between the foot contour point and the left and right scale lines of the interval, the score within the small interval is calculated according to the strategy of equal proportion division. For example, the score calculation formula is: 120+(123-120) / (125-120)*5=120+3 / 5*5=120+3=123 cm.

[0125] The final score is determined by adding up the scores of the large interval and the small interval. The large interval refers to the range between the two scale lines where the coordinates of the landing point are located.

[0126] In this example, the large interval is the range between 120 cm (x1) and 125 cm (x2). The system first determines the initial score interval based on whether the coordinates of the tester's landing point are within this large interval. Specifically, when the tester's landing point coordinates are 123 cm, this coordinate is between the two scale lines of 120 cm and 125 cm, so the large interval is 120 cm to 125 cm. Within this large interval, the system will calculate the score within the small interval according to the strategy of equal proportion based on the distance between the tester's foot contour point and the left and right scale lines of the interval. In the end, the score for the large interval is 120 cm, and the score for the small interval is 3 cm. By adding the scores of the large interval and the small interval, the final score is determined to be 123 cm.

[0127] Therefore, the above steps S131 to S133 provide a general method for calculating the score, that is, to determine the final score by accumulating the scores of the large interval and the small interval. However, the specific calculation method, that is, how to calculate the score of each landing point according to the coordinates of the landing point and the position of the scale line, includes:

[0128] S134, determining between which two scale lines on the field calibration scale the coordinates of the landing points of the tester's left and right feet fall, and checking whether the landing points are beyond the test area.

[0129] For more information, see Figure 8 , Figure 8 : is a schematic diagram of calculating the standing long jump score provided by the embodiment of the present application. The system uses the human posture estimation and landing point detection model to locate the landing point, that is, the key point of the tester's heel, according to the landing state judgment strategy. For example, the system determines between which two scale lines [x1, x2] the coordinates of the left and right foot landing points fall, and checks whether they exceed the test area.

[0130] S135, for each landing point, a preset score calculation formula is applied to determine the score of the landing point, and the scores corresponding to all landing points are calculated, and the smallest score is selected as the final standing long jump score of the current assessment.

[0131] For example, the score calculation formula for each landing point is:

[0132] The scale line is x1+(x-x1) / (x2-x1)*s.

[0133] Among them, x1 represents the starting scale line of the interval where the landing point is located, x represents the coordinates of the actual landing point of the tester, x2-x1 represents the length of the interval where the landing point is located, and s represents the length of each scale interval (for example, 5 cm). The calculation formula means starting from the starting scale x1 of the interval, adding the offset of the landing point x relative to the starting scale x1. The offset is calculated in proportion to the relative position of the landing point in the interval, and the proportion is multiplied by the length s of each scale interval.

[0134] Finally, the system calculates the scores corresponding to all possible landing points, and selects the smallest score from these scores as the final standing long jump score for the test.

[0135] The following example illustrates how to calculate the standing long jump score.

[0136] For example, in a standing long jump field, the scale lines range from 0 cm to 300 cm, with a scale every 5 cm. After the tester jumps long, the system needs to calculate his final score.

[0137] Assume that the coordinates of the tester's left foot landing point are 123 cm and the coordinates of the right foot landing point are 127 cm. The system uses the human posture estimation model and the key frame positioning model to locate these two landing points.

[0138] For left foot landing point (123 cm):

[0139] It falls between the marks 120 cm (x1) and 125 cm (x2).

[0140] The formula for calculating the score is: 120+(123-120) / (125-120)*5=120+3 / 5*5=120+3=123 cm.

[0141] For right foot landing point (127 cm):

[0142] It falls between the marks 125 cm (x1) and 130 cm (x2).

[0143] The formula for calculating the score is: 125+(127-125) / (130-125)*5=125+2 / 5*5=125+2=127 cm.

[0144] The system takes the minimum of the two landing point scores as the final score, which is 123 cm.

[0145] In summary, the key frame positioning model provided by the present application improves the key frame recognition accuracy of key actions (such as take-off and landing) through a multi-level cascade. Among them, the coarse positioning stage uses the human key point sequence to quickly identify the key frame, the purpose is to have a high recall rate, that is, to identify all possible key frames as much as possible. The fine positioning stage combines the multimodal input of the key point sequence and the image sequence to further improve the accuracy and robustness of the key frame positioning, that is, it can work stably under various conditions. Moreover, in the fine positioning stage, the modal fusion module is used to fuse the skeleton points and the image information to overcome the limitations of the single modal input in specific situations, such as shadows, uneven lighting, etc. In addition, the single target tracking model is applied to improve the reliability of the system and avoid abnormal system termination or score errors caused by irrelevant personnel interference.

[0146] The motion evaluation device provided in the present application is described below. The motion evaluation device and the motion evaluation apparatus described below and the motion evaluation method described above can be referenced to each other.

[0147] Please refer to Fig. 9 , Fig. 9 A sports evaluation device 900 includes a tester positioning module 910 , a take-off detection module 920 , a landing detection module 930 , and a score calculation module 940 .

[0148] Exemplarily, the tester positioning module 910 is used to call a preset human shape detection model to detect the input image frame in the preparation stage to locate the human body frame, and determine the tester located in the take-off area according to the human body frame.

[0149] Exemplarily, the take-off detection module 920 is used to call a preset human posture estimation model and a preset key frame positioning model during the take-off phase to determine whether the test person has started to jump, so as to determine the take-off frame.

[0150] Exemplarily, the landing detection module 930 is used to call the human posture estimation model and the key frame positioning model based on the take-off frame during the flight stage and the landing stage to determine whether the test person has started to land, so as to determine the landing frame.

[0151] Exemplarily, the performance calculation module 940 is used to determine the landing point of the tester according to the landing frame and calculate the standing long jump performance.

[0152] Exemplarily, the motion assessment device 900 further includes a target tracking module, which is used to call a preset target tracking model after determining that the tester is located in the take-off area, and start the person tracking logic after the tester reaches a ready state.

[0153] Exemplarily, the motion evaluation device 900 is further used for:

[0154] Obtain standing long jump images, and use the preset annotation tool to annotate each scale point in each image to form dense annotations;

[0155] Training a preset first detection model to detect four vertices of the sports field, wherein the vertices are key points for determining the location of the field;

[0156] cropping the image according to the detected four vertices to extract the sports field;

[0157] The preset second detection model is trained to perform dense annotation detection on the cropped image and identify all scale points to complete the training of the target detection model.

[0158] Exemplarily, the motion evaluation device 900 is further used for:

[0159] According to the detection result of the second detection model, the detected scale point frames are sorted according to their positions in the image, and the sorted frames are divided into upper and lower rows of frames by identifying distance mutation points;

[0160] For the upper and lower row boxes, count the width of each box and the interval between boxes, and traverse from left to right to determine whether there are any missed boxes based on the box width, interval size, and the distance between two boxes; if there are any missed boxes, perform linear interpolation based on the known box information to complete the missing boxes;

[0161] Determine whether the number of completed boxes meets the preset requirements.

[0162] Exemplarily, the motion evaluation device 900 is further used for:

[0163] Using a preset target detection framework, training using a data set containing a first number of key points and labeled long jump scene human frame data to obtain a trained human figure detection model;

[0164] Use the preset single target tracking model, the preset human body tracking dataset and the labeled long jump scene motion dataset for training to obtain a trained target tracking model;

[0165] Based on a preset single-person pose estimation framework, a data set including a first number of key points and a labeled data set including a second number of key points are used for mixed training to obtain a trained human pose estimation model;

[0166] Use the preset key point detection framework and the labeled heel key point data of the long jump scene for training to obtain a trained key point detection model.

[0167] Exemplarily, the motion evaluation device 900 is further used for:

[0168] The first key frame positioning model is used to identify whether a key frame is included; it uses a normalized key point sequence as input and predicts the action category through posture encoding, time sequence fusion and action classification;

[0169] The second key frame positioning model is used to locate the position of the key frame; it uses multimodal input of key point sequence and image sequence to predict the position of the key frame through posture coding, image coding, modality fusion, time fusion and key frame multi-classification.

[0170] Exemplarily, the first key frame positioning model includes a posture encoding module, a timing fusion network and an action classification head; the posture encoding module encodes a key point sequence into a key point vector, and inputs the high-dimensional vector and the learnable category vector into the timing fusion network to fuse the timing information and inputs the output category vector into the action classification head to predict the final action category.

[0171] Exemplarily, the second key frame positioning model includes a posture encoding module, an image encoding module, a modal fusion module, a timing fusion network and a key frame classification head; the posture encoding module encodes a key point sequence into a key point vector, and the image encoding module encodes an image sequence into an image vector; the modal fusion module concatenates the key point vector and the image vector into a multimodal vector that fuses multimodal information, and inputs it and a learnable category vector into a timing fusion network to output a category vector after fusing multimodal information and timing information and input it into a key frame multi-classification head to predict the position of the key frame.

[0172] Exemplarily, the motion evaluation device 900 is further used for:

[0173] In the preparation stage, the human detection model is called at regular intervals to identify the human body in the video frame sequence and select the human body rectangular frame located in the take-off area; if only one test person is detected in the take-off area for multiple consecutive times, the test person is determined to be in the preparation state;

[0174] In the take-off phase, when the tester is ready to take off, the first key frame positioning model is used to identify whether a take-off action occurs; if a take-off action is identified, the second key frame positioning model is switched to locate the take-off frame;

[0175] In the air phase and the landing phase, the first key frame positioning model is used to identify whether a landing action occurs; if a landing action is identified, the second key frame positioning model is switched to locate the landing frame;

[0176] Among them, the take-off frame is used to determine whether the tester violates the rules when taking off, and the landing frame is used to determine whether the tester violates the rules when landing.

[0177] Exemplarily, the target tracking module is further used for:

[0178] When the test person is in the take-off area, capture his / her initial image and set it as the tracking template;

[0179] Continuously acquiring a current frame from a video stream as a test area, and applying a template matching algorithm in the current frame to perform position retrieval based on the tracking template;

[0180] According to the output results of the template matching algorithm, the position of the test person in the current frame is located and the human body frame is output.

[0181] Exemplarily, the score calculation module 940 is further used for:

[0182] Get all the scale points on the sports field;

[0183] In the landing frame, the initial score interval is determined based on whether the tester's foot contour points are within a certain scale interval;

[0184] According to the distance between the foot contour point and the left and right scale lines of the interval, the score within the small interval is calculated according to the strategy of equal proportion division;

[0185] The final score is determined by adding the scores of the large interval and the small interval.

[0186] Exemplarily, the score calculation module 940 is further used for:

[0187] Determine between which two scale lines on the field calibration scale the coordinates of the tester's left and right feet's landing points fall, and check whether the landing points are beyond the test area;

[0188] For each landing point, the preset score calculation formula is applied to determine the score of the landing point, and the scores corresponding to all landing points are calculated, and the smallest score is selected as the final standing long jump score of the test.

[0189] It should be noted here that the above-mentioned motion assessment device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be described in detail here.

[0190] Fig.10 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Fig.10As shown, the electronic device may include: a processor (Processor) 1010, a communication interface (Communications Interface) 1020, a memory (Memory) 1030 and a communication bus 1040, wherein the processor 1010, the communication interface 1020, and the memory 1030 communicate with each other through the communication bus 1040. The processor 1010 may call the logic instructions in the memory 1030 to execute the motion evaluation method.

[0191] In addition, the logic instructions in the above-mentioned memory 1030 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including several instructions to ensure that a computer device (which can be a personal computer, a server, or a network device, etc.) executes all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0192] On the other hand, the present application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the motion assessment method provided by the above methods.

[0193] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the motion evaluation methods provided above.

[0194] The embodiments of the present application provide an electronic device, a computer program product, and a processor-readable storage medium, on which the computer program stored enables the processor to implement all the method steps implemented in the above-mentioned method embodiments and to achieve the same technical effects. The parts and beneficial effects of this embodiment that are the same as those in the method embodiments will not be described in detail herein.

[0195] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0196] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions to ensure that a computer device (which can be a personal computer, a server, or a network device, etc.) executes the methods described in each embodiment or some parts of the embodiments.

[0197] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A motion evaluation method, characterized in that: Applied to standing long jump, covering the preparation stage, take-off stage, flight stage and landing stage, the sports evaluation method includes: In the preparation stage, a preset human figure detection model is called to detect the input image frame to locate the human body frame, and the test person located in the take-off area is determined according to the human body frame; In the take-off phase, the preset human posture estimation model and the preset key frame positioning model are called to determine whether the test person has started to jump, so as to determine the take-off frame; In the flight phase and the landing phase, based on the take-off frame, the human body posture estimation model and the key frame positioning model are called to determine whether the test person has started to land, so as to determine the landing frame, and the landing point of the test person is determined according to the landing frame to calculate the standing long jump result.

2. The motion evaluation method according to claim 1, characterized in that: The motion evaluation method further comprises: After determining the tester in the take-off area, call the preset target tracking model and start the personnel tracking logic after the tester reaches the ready state.

3. The motion evaluation method according to claim 1, characterized in that: The method further includes training a target detection model to calibrate the sports field, the target detection model including a first detection model and a second detection model, and the training steps include: Obtain standing long jump images, and use the preset annotation tool to annotate each scale point in each image to form dense annotations; Training a preset first detection model to detect four vertices of the sports field, wherein the vertices are key points for determining the location of the field; cropping the image according to the detected four vertices to extract the sports field; The preset second detection model is trained to perform dense annotation detection on the cropped image and identify all scale points to complete the training of the target detection model.

4. The motion evaluation method according to claim 3, characterized in that: The method also includes an inference phase of the target detection model, wherein the inference steps include: According to the detection result of the second detection model, the detected scale point frames are sorted according to their positions in the image, and the sorted frames are divided into upper and lower rows of frames by identifying distance mutation points; For the upper and lower row boxes, count the width of each box and the interval between boxes, and traverse from left to right to determine whether there are any missed boxes based on the box width, interval size, and the distance between two boxes; if there are any missed boxes, perform linear interpolation based on the known box information to complete the missing boxes; Determine whether the number of completed boxes meets the preset requirements.

5. The motion evaluation method according to claim 1, characterized in that: The method further includes training a human shape detection model, a target tracking model, a human posture estimation model, and a landing point detection model, wherein: Using a preset target detection framework, training using a data set containing a first number of key points and labeled long jump scene human frame data to obtain a trained human figure detection model; Use the preset single target tracking model, the preset human body tracking dataset and the labeled long jump scene motion dataset for training to obtain a trained target tracking model; Based on a preset single-person pose estimation framework, a data set including a first number of key points and a labeled data set including a second number of key points are used for mixed training to obtain a trained human pose estimation model; Use the preset key point detection framework and the labeled heel key point data of the long jump scene for training to obtain a trained key point detection model.

6. The motion evaluation method according to claim 1, characterized in that: The method further comprises a reasoning stage of a key frame positioning model, wherein the key frame positioning model comprises a first key frame positioning model and a second key frame positioning model, and the reasoning step comprises: The first key frame positioning model is used to identify whether a key frame is included; it uses a normalized key point sequence as input and predicts the action category through posture encoding, time sequence fusion and action classification; The second key frame positioning model is used to locate the position of the key frame; it uses multimodal input of key point sequence and image sequence to predict the position of the key frame through posture coding, image coding, modality fusion, time fusion and key frame multi-classification.

7. The motion evaluation method according to claim 6, characterized in that: The first key frame positioning model includes a posture encoding module, a timing fusion network and an action classification head; the posture encoding module encodes a key point sequence into a key point vector, and inputs the key point vector and the learnable category vector into the timing fusion network to fuse the timing information and inputs the output category vector into the action classification head to predict the final action category.

8. The motion evaluation method according to claim 6, characterized in that: The second key frame positioning model includes a posture encoding module, a picture encoding module, a modality fusion module, a temporal fusion network and a key frame classification head; The posture encoding module encodes the key point sequence into a key point vector, and the image encoding module encodes the image sequence into an image vector; the modal fusion module concatenates the key point vector and the image vector into a multimodal vector that fuses multimodal information, and inputs the multimodal vector and the learnable category vector into a temporal fusion network to output a category vector after fusing the multimodal information and the temporal information and input it into a key frame multi-classification head to predict the position of the key frame.

9. The motion evaluation method according to claim 1, characterized in that: The sports evaluation method uses a state machine to perform logic judgment on the preparation phase, take-off phase, flight phase and landing phase, and the logic judgment steps include: In the preparation stage, the human detection model is called at regular intervals to identify the human body in the video frame sequence and select the human body rectangular frame located in the take-off area; if only one test person is detected in the take-off area for multiple consecutive times, the test person is determined to be in the preparation state; In the take-off phase, when the tester is ready to take off, the first key frame positioning model is used to identify whether a take-off action occurs; if a take-off action is identified, the second key frame positioning model is switched to locate the take-off frame; In the flight phase and the landing phase, the first key frame positioning model is used to identify whether a landing action occurs; if a landing action is identified, the second key frame positioning model is switched to locate the landing frame; Among them, the take-off frame is used to determine whether the tester violates the rules when taking off, and the landing frame is used to determine whether the tester violates the rules when landing.

10. The motion evaluation method according to claim 2, characterized in that: The steps of the personnel tracking logic include: When the test person is in the take-off area, capture his / her initial image and set it as the tracking template; Continuously acquiring a current frame from a video stream as a test area, and applying a template matching algorithm in the current frame to perform position retrieval based on the tracking template; According to the output results of the template matching algorithm, the position of the test person in the current frame is located and the human body frame is output.

11. The motion evaluation method according to claim 1, characterized in that: The step of determining the landing point of the tester according to the landing frame and calculating the standing long jump score comprises: Get all the scale points on the sports field; In the landing frame, the initial score interval is determined based on whether the tester's foot contour points are within a certain scale interval; According to the distance between the foot contour point and the left and right scale lines of the interval, the score within the small interval is calculated according to the strategy of equal proportion division; The final score is determined by adding the scores of the large interval and the small interval.

12. The motion evaluation method according to claim 1, characterized in that: The step of determining the landing point of the tester according to the landing frame and calculating the standing long jump score also includes: Determine between which two scale lines on the field calibration scale the coordinates of the tester's left and right feet's landing points fall, and check whether the landing points are beyond the test area; For each landing point, the preset score calculation formula is applied to determine the score of the landing point, and the scores corresponding to all landing points are calculated, and the smallest score is selected as the final standing long jump score of the test.

13. A sports evaluation device, characterized in that: Applied to standing long jump, covering the preparation stage, take-off stage, flight stage and landing stage, the sports evaluation device includes: A tester positioning module is used to call a preset human shape detection model to detect the input image frame in the preparation stage to locate the human body frame, and determine the tester located in the take-off area according to the human body frame; The take-off detection module is used to call a preset human posture estimation model and a preset key frame positioning model during the take-off phase to determine whether the test person has started to take off, so as to determine the take-off frame; A landing detection module is used to call the human posture estimation model and the key frame positioning model based on the take-off frame during the flight stage and the landing stage to determine whether the test person has started to land, so as to determine the landing frame; The score calculation module is used to determine the landing point of the tester according to the landing frame and calculate the standing long jump score.

14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the motion evaluation method according to any one of claims 1 to 12 are implemented.

15. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the motion evaluation method according to any one of claims 1 to 12 are implemented.

Images