Sit-up action scoring method, device and system based on key point detection

Abstract

The invention relates to a sit-up action scoring method, device and system based on key point detection, and belongs to the technical field of computer vision, and the method comprises the following steps: collecting and marking a sit-up data set and a standard action data set; the former is used for key point detection model training, and the latter is used for an action comparison network; building a key point detection module based on RTMPose, and carrying out key point detection on the sit-up video; an action normative scoring network is constructed, the action normative scoring network comprises foul action detection, movement speed curve analysis and abnormal force generation mode recognition modules, and real-time action scoring and voice broadcasting are carried out according to training / testing requirements; and constructing an action comparison network, comparing the user action with the standard action, and visualizing the result. The device comprises all the local software and hardware modules, and the system comprises a terminal and a cloud server. The method is comprehensive in scoring dimension, comparative analysis is performed on the user and the standard action in combination with space and time sequence features, and the application scene is wide.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology and relates to a method, device and system for scoring sit-up movements based on key point detection. Background Technology

[0002] Sit-ups, a classic exercise for measuring core strength and abdominal muscle endurance, are widely used in physical education, physical fitness testing, fitness training, and rehabilitation assessment. However, traditional sit-up quality assessment relies primarily on manual judgment, which is affected by the assessor's experience and perspective, and is inefficient, especially in large-scale physical tests in schools. Furthermore, the lack of quantitative criteria means that existing scoring systems can only provide qualitative conclusions of whether a movement is acceptable, making it difficult to conduct detailed quantitative analysis of proper form. Users cannot make timely adjustments based on existing performance evaluation results, and prolonged improper posture can easily lead to neck injuries, lumbar disc herniation, and other problems.

[0003] Besides manual judgment, other methods include using sensors or computer vision to determine whether a sit-up is performed correctly. Sensor-based methods require wearing sensors, which can be restrictive and have limitations that can affect the final score. Computer vision-based methods, on the other hand, are more convenient to use, do not require complex equipment, and are less affected by lighting conditions.

[0004] With the development of computer vision and deep learning technologies, action quality assessment techniques based on keypoint detection have gradually become a research hotspot. However, they still have many limitations in sit-up scenarios, including inaccurate detection results due to occlusion, high model computational load, and poor real-time performance. Existing sit-up motion standardization scoring schemes also have many shortcomings, such as: CN113255622A compares and identifies the completion status of action posture by using key angles. Although this simplifies the algorithm deployment, it only relies on the changes in key angles between adjacent frames to judge the action, and focuses on recognizing the action stage of sit-ups. The scoring depth of the action standardization is not deep enough, and users cannot adjust their actions based on the score, thus limiting its practicality for assistance.

[0005] CN119904463A uses skeletal point data and a standard sit-up model to count and judge whether a sit-up is qualified. However, the scoring dimension is singular, and the standard model is only used to help define the range of qualified parameters. It does not establish a dynamic comparison mechanism with professional standard movements, and cannot make a detailed comparison between user movements and standard movements in multiple dimensions such as angle, trajectory, and speed.

[0006] CN118968629A improves the action pose detection capability by modifying the YOLOv8-Pose network and achieves weighted scoring based on key angles. However, it only focuses on 6 skeletal key points, does not distinguish between the left and right sides, ignores the movement on the other side of the body, and is not good at detecting common incorrect movements in sit-ups. In addition, the method only scores the quality based on spatial features and does not involve the evaluation and analysis of temporal features or the dynamic comparison of standard movements. Summary of the Invention

[0007] In view of this, the purpose of the present invention is to provide a method, device and system for scoring sit-up movements based on key point detection.

[0008] To achieve the above objectives, the present invention provides the following technical solution: On one hand, the present invention provides a method for scoring sit-up movements based on key point detection, comprising the following steps: S1: Collect sit-up image data and standard sit-up video data, annotate the human body key points in the sit-up image data, and construct a sit-up key point dataset for training the key point detection model; collect standard sit-up video data and extract its standard key point sequence and kinematic features for constructing an action comparison network. S2: Build a key point detection model based on the RTMPose model, and train and optimize it using the sit-up key point dataset built in S1; acquire user sit-up video data in real time through video capture equipment, input the video data into the trained key point detection model, and detect and output key point coordinate data, confidence scores, and video data with human skeleton prediction results in real time. S3: Construct a motion standardization scoring network and input the key point detection data obtained in S2 into the motion standardization scoring network in real time; the motion standardization scoring network first performs motion phase recognition based on the key point sequence and completes the effective count accordingly; then, according to the preset rules and models, it detects foul actions and performs standardization scoring. S4: Construct an action comparison network to compare and analyze the key point data of the user's action sequence with the standard sit-up video data collected in S1, compare the angle data and action speed curves, and visualize the differences obtained from the comparison.

[0009] Furthermore, step S1 includes the following steps: S11: Collect sit-up image data in multiple scenarios, including sit-up action images from the side angle of users with different lighting conditions and body types; S12: The collected sit-up image data is annotated with human key points. The annotated key points include 14 core joint points, including the head, neck, shoulders, elbows, wrists, hips, knees, and ankles, forming a sit-up image dataset. S13: Collect sit-up video data from professional coaches at different standards, including physical testing standards, fitness crunch standards, and rehabilitation standards, and extract key movement trajectory templates for the standards.

[0010] Furthermore, step S2, the keypoint detection model building step based on the RTMPose model, includes: The RTMPose model is improved to be lightweight by removing the limb key point detection branches that are not related to sit-up motion detection and retaining only 14 core key point detection branches for head, neck, shoulders, elbows, wrists, hips, knees and ankles. The spatial attention mechanism (SAM) is introduced to enhance the features of the core region of the sit-up movement. During model training, Mosaic data augmentation, random horizontal flipping, angle rotation, and random scaling were used to augment and expand the sit-up image dataset. S24: Input the video stream obtained from the video acquisition device into the trained RTMPose model in real time, frame by frame, and output the human key point coordinate data for each frame. The coordinate data includes the coordinate values ​​and confidence scores of the key points.

[0011] Furthermore, the scoring logic of the action standardization scoring network described in step S3 is as follows: S31: Based on key point coordinate data, calculate multiple key angles in real time, including the shoulder angle from elbow to shoulder to hip, the elbow angle from shoulder to elbow to wrist, the hip angle from shoulder to hip to knee, the knee angle from hip to knee to ankle, and the neck angle along the line connecting the head to the neck and shoulders. S32: Record timestamps to segment each complete sit-up movement, dividing the complete sit-up movement into the rising phase and the falling phase; the three timestamps are the starting mark T0 when lying flat, the intermediate mark T1 when the elbow touches or passes the knee, and the ending mark T2 when returning to a flat position; set standard threshold ranges for each key angle, with the standard range for the elbow angle being... The standard range for the body folding angle at the hip is: ; S33: Calculate the deviation of the angle of each part from the standard threshold, identify common fouls based on the magnitude of the deviation, and record the motion velocity curve of each part. Identify common incorrect force patterns based on the temporal characteristics of the neck forward tilt angle and hip key points. The motion standardization scoring network is divided into a test mode and a training mode. The scoring components of the test mode only include foul detection, while the scoring components of the training mode include foul detection, motion velocity curve analysis, and abnormal force pattern identification. The score is then calculated based on the weight of the three components. S34: The scoring results will be broadcast in real time. In test mode, the scoring results will only be broadcast for sit-up movements that have passed the test. In training mode, the broadcast content will include the score of the current movement.

[0012] Furthermore, the foul detection steps include: Let point A Point B is the hip joint. Point C is the knee joint. For the elbow joint, the shoulder-hip offset angle on the side closer to the image acquisition device, i.e., the angle between the hip-shoulder line and the horizontal x-axis, is... The angle between the hip, knee, and ankle is The angle between the shoulder, elbow, and wrist is The Euclidean distance between the wrist and the head is The horizontal distance from the knee joint to the elbow joint is Based on spatial features and time series analysis and judgment, start flag T0, middle flag T1 and end flag T2 are set. The angle values ​​of each part are calculated according to the coordinate results of the key point detection model. Then, the angle value and coordinate value of each frame are used to detect auxiliary flags to determine the action stage of the current sit-up cycle through the state machine. Then, corresponding foul action detection is performed in different action stages. The shoulder-hip offset angle is used to determine three auxiliary indicators. If the user's right shoulder bone is closer to the image acquisition device, the right shoulder-right hip offset angle is calculated; otherwise, the left shoulder bone angle is calculated. Time-series parameters are added, and a sliding window of length M is added to each frame to linearly fit the angle sequence. A threshold value for the slope k of the fitted curve is set. To robustly determine the trend of angle changes; During initialization, T0, T1, and T2 are all set to Null; subsequently, the hip-shoulder offset angle is calculated for each processed frame. The trend and coordinate values ​​are used for iterative judgment: if the current frame is the first frame, if... The absolute value of the angle is less than or equal to If T0 is not within the specified range, then T0 is set as the current frame, marking the start of a new action cycle. If it is not within the specified range, the current frame is not within the action cycle, and the system waits for the judgment of subsequent frames. When the angle change trend changes from decreasing to increasing, the current frame is set to the T0 flag; when the detected... When the trend of change changes from increasing to decreasing, T1 is set as the current frame, marking the peak of the body's movement; when it is detected again... When the trend of change changes from decreasing to increasing, T2 is set as the current frame, marking the end of the cycle and returning to the initial posture. At the same time, the frame number where the T2 flag is located is directly set as the frame where the T0 flag of the next action cycle is located. Within this state machine framework, the detection of fouls is embedded in the corresponding stages. Flag4 "Knee angle not up to standard" and flag2 "Hands not behind head" are continuously monitored and checked in real-time throughout the entire movement cycle from T0 to T2. Once a violation is detected, a corresponding flag is immediately set. Flag3 "Elbow not touching knee" is a key posture check item, triggered and judged once at time T1, when the body fold is at its minimum. For the judgment of flag1 "Shoulder not touching mat," at the end of the cycle, the shoulder-hip offset angle during the T1-T2 state is detected. Has the set threshold been reached? If not, set flag1 "Shoulder not touching the pad" to TRUE. The specific criteria for the foul "knee angle not meeting the standard" (flag 1) are: the hip-knee-ankle angle of the left and right knees must be within the specified range. All If it is inside, otherwise it is judged as a foul; The specific criteria for the foul "hands not covering head" in flag 2 are: the shoulder-elbow-wrist angle of the left and right elbows must be within the specified range. All satisfied Meanwhile, the Euclidean distance between the wrist and the head satisfies Otherwise, it is a foul; among them, l is the torso length of the user at the first time T0, which is the length of the line connecting the midpoint of the left and right shoulders to the midpoint of the left and right hips. The specific criteria for the foul "elbow not touching knee" in flag3 are as follows: at time T1, the horizontal Euclidean distance between the elbow and knee joints on both sides must meet the following conditions. The unit is pixels; otherwise, it is considered a foul. The specific determination of the foul "shoulder not touching the mat" in flag4 is as follows: it requires the shoulder-hip offset angle on the side closest to the image acquisition device during the T1 to T2 state. At least one frame satisfies Otherwise, it is judged as a foul; among them, For the first T0 time point If the user starts moving before the system prompt sound, resulting in storage... Then set to ; The above parameters involve the calculation of the angle between two vectors and the Euclidean distance. Assuming vectors... and The coordinates are respectively , and , Then the included angle The calculation formula is:

[0013] Suppose the coordinates of the two points for which the Euclidean distance is to be calculated are... and Then the formula for calculating the Euclidean distance D is:

[0014] When a complete cycle ends with the T2 flag, the cycle is comprehensively scored based on the flags of the foul action types stored in the current cycle: the states of the four foul flags flag1 to flag4 are retrieved; in test mode, if all flags are False, the movement in this cycle is considered qualified; if any foul flag exists, it is considered unqualified; in training mode, scoring is performed on a 100-point scale, with the four foul action types equally weighted. Flags with a TRUE foul flag are counted, and points are deducted according to their corresponding weights. When all four flags are TRUE, that portion of the score is... 0; then, before T2 is set to Null, the current motion cycle is broadcast with voice: in test mode, when there is no foul flag that meets the conditions, the count is effectively counted through voice broadcast; if there is a foul flag, the specific foul content or multiple fouls are selectively broadcast according to preset rules; in training mode, the voice broadcast content is combined with the analysis of the motion speed curve and the identification of abnormal force patterns; after the adjudication and broadcast are completed, the system resets T1 and T2 to Null, retains the T0 state of the current frame, clears all foul flag bits, and then enters the detection process for the next motion cycle.

[0015] Furthermore, the motion velocity curve analysis process is as follows: After obtaining a complete motion cycle defined by T0, T1, and T2, the user's motion speed curve is extracted and analyzed. The vertical displacement sequence of the key points of the shoulder joint in each complete motion cycle is extracted, and the speed sequence is obtained by differential calculation and smoothing filtering. Then, four core indicators are decomposed and calculated according to the following table, and the scores of each dimension are obtained according to the scoring method. Finally, the total score of this action in terms of speed control is obtained by weighted summation. The four core indicators include the standard deviation of speed in the rising phase, the standard deviation of speed in the falling phase, the maximum speed in the falling phase, and the rising / falling time ratio.

[0016] Furthermore, the abnormal force application pattern recognition process is as follows: In training mode, firstly, within a complete movement cycle determined by the state machine, the coordinate sequences of key points of the hip, neck and head are extracted; then, based on the key point detection data, biomechanical analysis is performed on the user's force exertion pattern to identify abnormal patterns of iliopsoas muscle-dominated compensation and neck muscle group compensation, as well as movement coordination. For iliopsoas muscle compensation, the rate of change of pelvic tilt angle is calculated. The pelvic tilt angle is defined as the angle between the line connecting the two hip joints and the horizontal line. The hip axis vector is calculated using the key points of the left and right hips, and then the angle between this vector and the horizontal axis (1,0) is calculated. For neck compensation, calculate the forward extension angle of the head and neck, and calculate the maximum value of the angle between the perpendicular line connecting the head key point and the neck key point and the line connecting the two shoulders. To assess the coordination of movements, the consistency of the head-shoulder-pelvis movement sequence is calculated. Specifically, this is determined by the displacement sequence of the head, the midpoint of the shoulder, and the midpoint of the pelvis in the Y direction. The vertical movement velocities of the three are calculated, the peak positions of each velocity curve during the rising phase are found, and the matching degree between the actual peak sequence and the ideal sequence is calculated. The ideal peak sequence is hip-shoulder-head. After calculating all indicators, scores for each dimension are obtained according to the scoring method. Finally, the total score of this action in terms of force exertion mode is obtained by weighted summation. The comprehensive score and the performance of each item are combined to generate voice feedback for the item with the largest deduction, which directly guides the user to carry out corrective training. In training mode, the formula for the comprehensive movement quality score for each exercise cycle is:

[0017] Where w1 is the weighting coefficient of the foul action detection module, w2 is the weighting coefficient of the motion speed curve analysis module, and w3 is the weighting coefficient of the abnormal force application pattern recognition module. This represents the total number of foul categories in the foul detection module. The number of detected foul categories; the score of the motion speed curve analysis module. The formula is:

[0018] The standard deviation of the vertical velocity of the shoulder joint during the ascending phase T0~T1 , The vertical velocity of the shoulder joint during the upward phase. The average velocity of the vertical velocity of the shoulder joint during the rising phase; The standard deviation of the vertical velocity of the shoulder joint during the descent phase T1~T2 is given. This represents the maximum vertical velocity of the shoulder joint during the descent phase. Score for abnormal force pattern recognition The formula is:

[0019] in, Pre-exercise period Hip anteversion angle at this stage This represents the maximum forward head and neck deflection angle during the entire movement cycle. The score is obtained by the order in which the movement velocities of different head-shoulder-hip joints peak.

[0020] Furthermore, in step S4, the visualization step for the difference results of the action comparison network is as follows: S41: Extract key angle data sequences and motion speed curve data of user actions and standard actions during the rising and falling phases, and use dynamic time warping algorithm combined with action phase constraints to segment and align motion sequences at different speeds. S42: Calculate the differences between user actions and standard actions in terms of angle data, velocity curves, and skeletal point trajectory dimensions, generate a difference heatmap, and highlight the difference areas with color coding. S43: Generate a visual report, including side-by-side display of user action videos and standard action videos, angle-time curve comparison charts, and pie charts of deduction statistics. Users can intuitively view their shortcomings in action through the report and make targeted improvements.

[0021] On the other hand, the present invention provides a sit-up motion scoring device based on key point detection, comprising: The image acquisition module is used to acquire video data of the user performing sit-ups and divide the acquired video into several frames. The key point extraction module, based on the RTMPose model, uses a human key point detection model to detect the coordinate information of the user's key skeletal points in each frame of the video. The standard storage module is used to store motion templates of different standards. The motion templates of different standards include sit-up videos in different usage scenarios, as well as the coordinate sequence of key points of human skeleton in the videos, angle curves and speed curves. The action scoring module includes an action standardization scoring unit and an action standard comparison unit. The action standardization scoring unit is used to calculate the comprehensive action standardization score using a weighted scoring algorithm across multiple dimensions and to identify foul actions. The action standard comparison unit is used to compare the user's video with the action templates in the standard storage module. The results output module includes a display unit and a voice broadcast unit. The voice broadcast unit is used to output the comprehensive score and feedback information on non-standard items in real time in voice form. The display unit is used to output various results such as analysis and comparison of user videos and standard videos. The model update module is used to iteratively train the human key detection model based on new action sample data and continuously optimize the detection accuracy. The identity recognition module is used to identify the user's identity information and facilitate the storage of the user's historical exercise records to provide more detailed and targeted training suggestions.

[0022] Thirdly, the present invention provides a sit-up motion scoring system based on key point detection, comprising: The terminal device integrates the sit-up motion scoring device based on key point detection as described above, and realizes local real-time scoring. The cloud server communicates with the terminal device via wired or wireless means, receives the exercise data and scoring results uploaded by the terminal device, stores the user's exercise history and provides statistical analysis.

[0023] The beneficial effects of this invention are as follows: This invention, through a real-time keypoint detection and scoring network, can provide immediate feedback when the user performs an action, achieving "evaluation while performing," resulting in an excellent interactive experience, especially suitable for training modes; This invention offers comprehensive scoring dimensions, not only performing basic action counting and amplitude judgment, detecting violations such as whether the shoulder touches the mat or the foot is lifted, but also extending to the evaluation of action rhythm and stability in training modes, and detecting common incorrect force application patterns, making the scoring system more scientific and comprehensive; This invention, through an action comparison network, transforms abstract action differences into intuitive angle comparison diagrams and speed curves, and generates visual comparison videos, enabling users to clearly understand their own action problems, facilitating targeted improvements, and is applicable to multiple standards in various application scenarios. Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0024] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a flowchart of the sit-up motion scoring method based on key point detection according to the present invention. Figure 2 This is a schematic diagram of key human body points detected by the method of the present invention. Figure 3 This is a network structure diagram of the action standardization scoring method of the present invention; Figure 4 This is a simplified structural diagram of the device of the present invention. Detailed Implementation

[0025] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0026] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0027] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0028] Example 1: like Figure 1 As shown, this invention provides a sit-up motion scoring method based on key point detection, including: S1. Collect sit-up image data and standard sit-up video data. Annotate key human body points in the sit-up image data, such as... Figure 2 As shown, a sit-up keypoint dataset was constructed to train the keypoint detection model. Simultaneously, standard sit-up video data performed by professionals was collected, and its standard keypoint sequences and kinematic features were extracted. This data was used in a subsequent motion comparison network to perform comparative analysis with users' sit-up movements. S2. Build a keypoint detection module based on the RTMPose model and train and optimize it using the dataset constructed in S1. Acquire video streams of users performing sit-ups in real time through video capture devices, and feed the video streams frame by frame into the trained RTMPose model to detect and output the two-dimensional coordinate sequence of 14 key points of the user's body (including head, neck, shoulder, elbow, hip, knee, and ankle) in real time. S3. Construct a motion standardization scoring network, such as Figure 3As shown, the system includes a foul action detection module, a motion speed curve analysis module, and an abnormal force application pattern recognition module. The keypoint detection data obtained from S2 is input into this network in real time. Depending on the training or testing needs, different modules are selected for real-time action scoring, and the scoring results are broadcast aloud in real time. The foul action detection module identifies foul actions based on the angle values ​​of joints such as the shoulder, elbow, and hand according to the key point detection results, and then scores the module based on the detected foul action identification results. The motion velocity curve analysis module analyzes the velocity in the vertical direction of the shoulder joint, calculates relevant core indicators, and obtains the module's score result by combining a weighted sum with the scoring method. The abnormal force pattern recognition module judges the range of relevant indicators for key point sequences of the hip, neck and head, and then obtains the score of each indicator based on the judgment result. Finally, the module's score result is obtained by weighted summation. The scoring network first performs action phase recognition (divided into rising and falling phases) based on keypoint sequences, and then counts the number of valid actions accordingly. Subsequently, the scoring network detects fouls and provides real-time standardized scoring based on preset rules and models. In test mode, the system provides a count prompt with a "beep" sound after each valid action; in training mode, the system announces the detailed scoring results for that action. S4. Construct an action comparison network containing a standard sit-up video database. After the user finishes sit-ups, the system sends the saved complete video and corresponding key point data to the action comparison network. This network uses the FASTDTW algorithm combined with action phase constraints to align the user video and the standard video using the key point data of the user's entire action sequence. It then analyzes the spatial and temporal features of both, comparing angle data and velocity curves, and visualizes the differences obtained from the comparison.

[0029] As an optimization, the specific steps of S1 are as follows: The sit-up keypoint dataset contains sit-up images of different body types, genders, clothing, and lighting conditions, with fine annotations on 14 keypoints. Standard sit-up video data should be demonstrated by sports experts to ensure absolutely correct movement, and standard keypoint motion trajectory templates should be extracted from them.

[0030] As an optimization, the construction process of the RTMPose keypoint detection model in S2 is as follows: The model is an optimization and lightweight improvement based on RTMPose-m, which strikes a relatively good balance between accuracy and speed. The improved RTMPose model includes a backbone network, a neck network, and a head network. The backbone network is responsible for extracting multi-level features from the input image. CSPNEXt is selected as the backbone network. Before the features are fed into the neck network, a SAM spatial attention module is inserted to apply spatial attention to the feature maps of each scale output by the backbone network, improving the robustness of keypoint detection. The neck network uses a Feature Pyramid Network (FPN) to fuse features extracted from different scales by the backbone network. The head network uses the SimCC paradigm for keypoint localization, transforming the keypoint coordinate prediction problem into two independent classification tasks: predicting the horizontal and vertical coordinates separately, which is computationally efficient and easy to deploy.

[0031] The keypoint selection process removes limb keypoint detection branches irrelevant to sit-up motion detection, reducing the model's computational load. Ultimately, 14 core keypoint detection branches are retained: head, neck, shoulders, elbows, wrists, hips, knees, and ankles. Simultaneously, image augmentation is performed on the sit-up image dataset before model training, including Mosaic data augmentation, random horizontal flipping, angle rotation, and random scaling enhancement, improving the model's adaptability to different scenarios.

[0032] As an optimization, the scoring logic of the action normativity scoring network described in S3 is as follows: 1) Based on key point coordinate data, calculate multiple key angles in real time, including the shoulder angle from elbow to shoulder to hip, the elbow angle from shoulder to elbow to wrist, the hip angle from shoulder to hip to knee, the knee angle from hip to knee to ankle, and the neck angle along the line connecting the head to neck to both shoulders. 2) Record timestamps to segment each complete sit-up movement, dividing the entire sit-up into two phases: the rising phase and the falling phase. Three timestamps are used: the starting point (T0) when lying flat, the intermediate point (T1) when the elbows touch or pass the knees, and the ending point (T2) when returning to a flat position. Set standard threshold ranges for each key angle; the standard range for the elbow angle is... The standard range for the body folding angle at the hip is: ; 3) Calculate the deviation of the angle of each body part from the standard threshold, and identify common fouls based on the magnitude of the deviation. Simultaneously, record the motion velocity curves of parts such as the hip, and identify common incorrect force application patterns based on the forward tilt angle of the neck and the temporal characteristics of key hip points. The motion standardization scoring network has two modes: a testing mode and a training mode. The scoring component of the testing mode only includes foul detection, while the scoring component of the training mode includes foul detection, motion velocity curves, and abnormal force application pattern identification. The scoring uses a 100-point scale. 4) The scoring results will be broadcast in real time using speech synthesis technology. In the test mode, the prompt will only be broadcast for sit-ups that have passed the score. In the training mode, the broadcast will include the score of the current movement.

[0033] The construction process of the S3 action normativity evaluation network is as follows: The system comprises a foul action detection module, a motion velocity curve analysis module, and an abnormal force application pattern recognition module. Based on the keypoint coordinates and confidence scores output by the keypoint detection model, the keypoint angles for each frame are calculated. The calculation results are then fed into different detection modules depending on the testing or training requirements. For testing purposes, only the foul action detection module is used, assigning weighted scores based on angle values ​​and keypoint coordinate changes to determine whether the action is acceptable. For training purposes, the angle values ​​and keypoint coordinate values ​​need to be fed into all three modules to detect foul actions, analyze the overall motion velocity curve, and detect abnormal force application patterns, then scoring is performed based on the weights of the three modules.

[0034] As a preferred option, the specific process of the foul detection module is as follows: According to the National Student Physical Fitness and Health Standards, the sit-up test requires the knees to be bent at approximately a 90-degree angle, the shoulder blades to touch the mat while lying supine, and the elbows to touch the knees or thighs during the sit-up, with both hands crossed behind the head throughout the entire exercise. Based on these standards, a violation detection system for sit-ups is implemented, focusing on the shoulders, elbows, and hands, identifying violations such as: flag 1 "knee angle not up to standard", flag 2 "hands not behind head", flag 3 "elbow not touching knee", and flag 4 "shoulder not touching mat".

[0035] For the sake of convenience in the following description of the solution, such as Figure 2 As shown, point A Point B is the hip joint. Point C is the knee joint. For the elbow joint, the shoulder-hip offset angle on the side closer to the image acquisition device, i.e., the angle between the hip-shoulder line and the horizontal x-axis, is... The angle between the hip, knee, and ankle is The angle between the shoulder, elbow, and wrist is The Euclidean distance between the wrist and the head is The horizontal distance from the knee joint to the elbow joint is .

[0036] This foul detection module analyzes and judges based on spatial features and time series, setting three auxiliary markers T0 (start marker), T1 (intermediate marker), and T2 (end marker) to accurately divide a complete sit-up cycle. Specifically, the system calculates the angle values ​​of each part based on the coordinate results of the key point detection module, and then uses a state machine to detect the auxiliary markers on the angle and coordinate values ​​of each frame to determine the action stage of the current sit-up cycle, and then performs corresponding foul detection at different action stages.

[0037] Considering the differences in body posture, movement habits, and the impact of knee joint keypoint detection accuracy on different users, the judgment of the three auxiliary markers is not based on the shoulder-hip-knee body folding angle, but on the shoulder-hip offset angle. This angle is differentiated based on the user's lateral skeleton; that is, when the user's right skeleton is closer to the camera, the right shoulder-right hip offset angle is calculated, and vice versa. Meanwhile, since the shoulder-hip offset angle is affected by the accuracy of keypoint detection, coordinate shifts inevitably occur in some frames, resulting in an unstable angle. Therefore, relying solely on the absolute angle of spatial features to determine the auxiliary marker segmentation of the action cycle is too inaccurate. Thus, a time series parameter is added to the system. A sliding window of length M=7 is added to each frame to linearly fit the angle sequence, and a threshold for the slope k of the fitted curve is set. To robustly determine the trend of angle change (increasing, decreasing, or remaining stable).

[0038] During system initialization, T0, T1, and T2 are all set to null. Subsequently, the hip-shoulder offset angle is calculated for each processed frame. The trend and coordinate values ​​are used for iterative judgment: if the current frame is the first frame, if... The absolute value of the angle is less than or equal to If T0 is not within the specified range, then T0 is directly set as the current frame, marking the start of a new action cycle. If it is not within the specified range, the current frame is not within the action cycle, and it needs to wait for the judgment of subsequent frames. When the angle change trend changes from decreasing to increasing, the current frame is set to the T0 flag; when the detected... When the trend of change changes from increasing to decreasing, T1 is set as the current frame, marking the peak of the body's movement; when it is detected again... When the trend of change changes from decreasing to increasing, T2 is set as the current frame, marking the end of the cycle and returning to the initial posture. At the same time, the frame number where the T2 flag is located is directly set as the frame where the T0 flag of the next action cycle is located.

[0039] Within this state machine framework, the detection of foul actions is embedded in the corresponding stages. Flag4 "Knee angle not up to standard" and flag2 "Hands not behind head" are continuously monitored and checked in real-time throughout the entire action cycle from T0 to T2. Once a violation is detected, the corresponding flag is immediately set. Flag3 "Elbow not touching knee" is a critical posture check item, triggered and judged once at time T1, when the body fold is at its minimum. For the judgment of flag1 "Shoulder not touching mat," the system detects the shoulder-hip offset angle during the transition from T1 to T2 at the end of the cycle (when T2 is set). Check if the set threshold has been reached. If not, set flag1 "Shoulder not touching the pad" to TRUE.

[0040] The specific criteria for the foul "knee angle not meeting the standard" (flag 1) are: the hip-knee-ankle angle of the left and right knees must be within the specified range. All If it is inside, otherwise it is judged as a foul.

[0041] The specific criteria for the foul "hands not covering head" in flag 2 are: the shoulder-elbow-wrist angle of the left and right elbows. All satisfied Meanwhile, the Euclidean distance between the wrist and the head satisfies Otherwise, it is a foul. 'l' represents the user's torso length at the first time point T0, which is the length of the line connecting the midpoints of the left and right shoulders to the midpoints of the left and right hips. Considering individual differences among users and the standard requirements for flexible movements such as hands behind the head and hands placed beside the ears, The threshold is not set as an absolute angle value, but is dynamically set according to the proportions of the human body.

[0042] The specific criteria for the foul "elbow not touching knee" in flag3 are as follows: at time T1, the horizontal Euclidean distance between the elbow and knee joints on both sides must meet the following conditions. The unit is pixels; otherwise, it is considered a foul. The specific determination of the foul "shoulder not touching the mat" in flag4 is as follows: it requires the shoulder-hip offset angle on the side closest to the image acquisition device during the T1 to T2 state. At least one frame satisfies Otherwise, it is judged as a foul; among them, For the first T0 time point If the user starts moving before the system prompt sound, resulting in storage... Then set to ; The above parameters involve the calculation of the angle between two vectors and the Euclidean distance. Assuming vectors... and The coordinates are respectively , and , Then the included angle The calculation formula is:

[0043] Suppose the coordinates of the two points for which the Euclidean distance is to be calculated are... and Then the formula for calculating the Euclidean distance D is:

[0044] When a complete cycle ends with the T2 flag, the system performs a comprehensive score for the current cycle based on the flags of the foul action types stored in the current cycle: It retrieves the status of the four foul flags (flag1 to flag4); in test mode, if all flags are False, the current cycle is considered successful; if any foul flag exists, it is considered unsuccessful; in training mode, the module is scored on a 100-point scale, with the four foul action types equally weighted. Flags with TRUE foul flags are counted, and points are deducted according to their corresponding weights. When all four flags are TRUE, the score for that part is 0. Then, before T2 is set to Null, the current cycle is broadcast aloud: in test mode, if no valid foul flags are found, a "beep" sound is broadcast and a valid count is performed; if foul flags are found, specific foul content or "multiple fouls" are selectively broadcast according to preset rules; in training mode, the broadcast content needs to be combined with the latter two modules. After the adjudication and broadcast are completed, the system resets T1 and T2 to null, retains the T0 state of the current frame (as it has been used as the starting point of the next cycle), clears all foul flags, and then enters the detection process for the next action cycle, ensuring the smoothness of system state transitions and the continuity of counting logic under continuous actions.

[0045] As a preferred option, the specific workflow of the motion velocity curve analysis module is as follows: After obtaining a complete motion cycle defined by T0, T1, and T2, the user's motion velocity curve is extracted and analyzed. Since the shoulder joint, as a representative point of the upper torso, most directly and significantly reflects the motion trajectory of the entire torso, the vertical displacement sequence of key shoulder joint points for each complete motion cycle is extracted. The velocity sequence is then calculated using differential calculation and smoothed. Subsequently, four core indicators are decomposed and calculated according to the following table, and scores for each dimension are obtained based on the scoring method. Finally, a weighted sum is used to obtain the total percentage score for speed control in this action.

[0046] As shown in Table 1, the standard deviation of the speed during the ascent and descent phases primarily reflects whether the user exerts force in a controlled and uniform manner during exercise; a lower standard deviation indicates better training results. The maximum speed during the descent phase is an indicator for assessing the risk of lower back injury. A higher maximum speed indicates insufficient eccentric force, making it difficult to control the descent speed, and the lumbar spine is more susceptible to significant impact, leading to lumbar strain and herniated discs. The ascent / descent time ratio reflects training rhythm and balance; the closer the ratio is to 1, the more balanced the user's control development.

[0047] Table 1

[0048] As a preferred option, the specific process of the abnormal force exertion pattern recognition module is as follows: In training mode, this module performs biomechanical analysis on the user's force application patterns based on key point detection data. Its aim is to identify deep-seated movement errors, warn of sports injury risks, and provide targeted improvement suggestions. This module focuses on two of the most common and high-risk abnormal force application patterns in sit-ups: the primary problem is "iliopsoas-dominant compensation," where insufficient rectus abdominis strength leads to excessive force exertion by the iliopsoas muscle, which crosses the hip joint, compensating for the movement with forceful anterior pelvic rotation and hip flexion. This pattern incorrectly transfers the training load to the lower back and hips, easily leading to lumbar strain and hip discomfort over time. The secondary problem is "neck muscle compensation," where when the user has difficulty curling up, they attempt to assist by excessive forward flexion of the neck and chin retraction, resulting in excessive stretching of the neck muscles and ligaments, increasing the risk of cervical spine injury.

[0049] To quantify the aforementioned abnormal patterns, this module defines the following core biomechanical indicators and incorporates them into the scoring system: for iliopsoas compensation, the "rate of change of pelvic tilt angle" is calculated; for cervical compensation, the "angle of head and neck extension" is calculated. The specific scoring scheme is shown in the table below. This scheme uses a percentage system, with a weighted calculation to arrive at the final score. Semantic improvement suggestions are generated for each abnormal pattern; for example, if iliopsoas compensation is detected, the suggestion is "Please focus on abdominal engagement," and if cervical compensation is detected, the suggestion is "Please tuck your chin and relax your neck."

[0050] Table 2

[0051] As shown in Table 2, the calculation and judgment of the rate of change of the pelvic tilt angle are defined as the angle between the line connecting the two hip joints (hip axis) and the horizontal line (X-axis). Specifically, the hip axis vector is calculated using the key points of the left and right hips, and then the angle between this vector and the horizontal axis (1,0) is calculated. Neck compensation is achieved by calculating the maximum value of the angle between the perpendicular line connecting the head key point and the neck key point and the line connecting the two shoulders. The consistency of the head-shoulder-pelvis movement sequence is judged by the displacement sequence of the head, shoulder midpoint, and pelvis midpoint in the Y direction. The vertical movement velocities of the three are calculated, the peak position (frame number) of each velocity curve in the rising phase is found, and then the matching degree between the actual peak sequence and the ideal sequence is calculated. The ideal peak sequence is hip-shoulder-head.

[0052] The specific process of this module is as follows: First, within a complete action cycle determined by the state machine, the key point coordinate sequences of the hip, neck, and head are extracted; then, the various indicators listed in the table above are calculated, and scores for each dimension are obtained according to the scoring method; finally, a weighted sum is used to obtain the total score of the action in terms of force application mode. The system will combine the overall score with the performance of each sub-item, and generate voice feedback for the sub-item with the largest deduction, directly guiding the user to conduct corrective training, thereby realizing an intelligent training closed loop from "identifying errors" to "guiding correctness".

[0053] In addition, the motion standardization scoring network also integrates a simple fatigue monitoring mechanism. The principle is to analyze the average speed of the rising and falling phases of each movement in a series of continuous movements. If the speed difference between the two phases reaches a threshold, it indicates that the user's eccentric control ability decreases with the increase of repetitions and the core muscle groups are fatigued. Based on this, the system will generate a voice prompt such as "It is recommended to take a short rest" to remind the user to rest, and provide the user with personalized training intensity suggestions.

[0054] In training mode, the formula for the comprehensive movement quality score for each exercise cycle is:

[0055] Among them, w1 is the weight coefficient of the foul action detection module, w2 is the weight coefficient of the motion speed curve analysis module, and w3 is the weight coefficient of the abnormal force pattern recognition module. The weight ratio of each module can be adjusted according to different training needs. The default weight coefficients are w1=0.4, w2=0.3, and w3=0.3. This represents the total number of foul categories in the foul detection module. The number of detected foul categories. Score from the motion speed curve analysis module. The formula is:

[0056] The standard deviation of the vertical velocity of the shoulder joint during the ascending phase T0~T1 , The vertical velocity of the shoulder joint during the upward phase. The average velocity of the shoulder joint's vertical velocity during the upward phase. The standard deviation of the vertical velocity of the shoulder joint during the descent phase T1~T2 is given. This represents the maximum vertical velocity of the shoulder joint during the descent phase.

[0057] Score of the abnormal force pattern recognition module The formula is:

[0058] in, Pre-exercise period Hip anteversion angle at this stage This represents the maximum forward head and neck deflection angle during the entire movement cycle. The score is obtained by the order in which the movement velocities of different head-shoulder-hip joints peak.

[0059] As an optimization, the construction process of the S4 action alignment network is as follows: As an optimization of the system of the present invention, the action comparison network is an offline, deep analysis module that is activated after the user completes a set of training or testing. It is designed to make up for the limitations of real-time scoring networks and provide users with intuitive and actionable adjustment suggestions through multi-dimensional fine comparison and visualization.

[0060] Considering the limitations of motion standardization scoring networks, which only provide rough scores, lack data visualization, and have incomplete application scenarios, this motion comparison network performs statistical analysis on data from the scoring network and compares user videos with standard videos. These standard videos are not limited to physical testing and practice scenarios but can also be applied to other scenarios such as abdominal crunches and rehabilitation training. The network acquires the keypoint coordinates and angle values ​​of each frame stored in the motion standardization network, as well as the frame location, count, number of frames where the violation flag (flag) is false, and the shoulder joint's velocity sequence in the vertical direction for each motion cycle. It then performs statistical analysis on the above data and the standard data, outputting a comparison of the differences in spatial and temporal features between the user video and the corresponding standard video, along with relevant statistical graphs.

[0061] The overall process of the action matching network is as follows: From the standard database built by S1, the corresponding standard motion template is called according to the mode selected by the user (such as physical test mode, fitness mode, rehabilitation mode). The template contains data such as standard key point sequence, standard angle curve, and standard speed curve. At the same time, the data of the user in each motion cycle in the whole video is retrieved through the scoring network.

[0062] To reduce the impact of differences in device distance and image size between user video and standard video during the comparison process, spatial normalization needs to be performed before the temporal alignment of the videos. After normalizing the coordinate sequence of the standard video, the standard skeleton is scaled according to the length of the user's torso stored in the T0 frame of the first motion cycle of the user video.

[0063] Timing alignment is performed between user videos and standard videos. Considering the different speeds and rhythms of user and standard actions, directly comparing the two videos frame-by-frame is meaningless. This network employs a Dynamic Time Warping (DTW) algorithm incorporating action phase constraints. First, three timestamps are extracted from each sit-up movement in both the user and standard videos: T0 (lying up), T1 (elbow to knee), and T2 (falling back to lying down). Then, T0→T1 (ascending phase) and T1→T2 (falling phase) are locally aligned using FastDTW. Alignment using only linear interpolation is not very effective; therefore, the matching method in DTW combines FastDTW with linear interpolation. The keypoint coordinates of each frame are flattened into a one-dimensional vector as frame features. Frames are matched by measuring the similarity of keypoints in each frame of the two videos, i.e., the shortest distance between keypoints. The mapping relationship between user frames and standard frames is output as the matching result. For user frames not appearing in the DTW path, the nearest matching points before and after the user frame are found using linear interpolation to calculate the standard frame index. After time alignment, it becomes more intuitive to compare the spatial feature differences between user videos and standard videos at the same stage, namely the differences in the coordinates and angle values ​​of each key point.

[0064] The system generates a visual report, including side-by-side displays of the user's action video and the standard action video, an angle-time curve comparison chart, and a pie chart summarizing deductions. Users can intuitively view their shortcomings and make targeted improvements through the report. In addition to showing the connection between the user and the standard skeleton and the angle values ​​at each key point, the comparison video also displays the current number of qualified sit-ups, the number of exercise cycles, and whether any violations were detected in the current frame.

[0065] Example 2: The sit-up motion scoring method based on key point detection in this embodiment includes the following steps: I. Collecting and constructing a sit-up dataset.

[0066] Sit-up Key Point Dataset: In this embodiment, sit-up videos were collected in various environments, including indoor venues and outdoor playgrounds. During collection, the camera was positioned to the side of the volunteer, 1.5m away and 0.8m above the horizontal ground. The subjects included individuals of different body types, genders, and clothing. The collected videos were saved in MP4 format, saved as frame images at a fixed frame rate of 30fps, and then labeled with 14 key points (head, neck, left / right shoulder, left / right elbow, left / right hip, left / right knee, left / right ankle) in each image. The dataset was divided into training, validation, and test sets in an 8:1:1 ratio.

[0067] Standard Movement Template Library: We invited students and relevant sports experts to record high-definition videos of standard sit-up movements strictly following the "National Student Physical Fitness and Health Standards." Additionally, to expand application scenarios, we invited fitness coaches and relevant experts to record standard sit-up videos tailored to fitness shaping and rehabilitation training needs.

[0068] II. Constructing a key point detection model.

[0069] This model improves and optimizes upon the RTMPose model, removing facial features such as eyes, nose, and ears that are irrelevant to sit-up motion detection, retaining only 14 core keypoint detection branches for the torso, upper limbs, and lower limbs, thus reducing computational load. A SAM spatial attention mechanism is introduced to enhance features in the core sit-up motion region, the torso, improving the robustness of keypoint detection. During model training, methods such as Mosaic data augmentation, random horizontal flipping, angle rotation, and random scaling are used to expand the sit-up image dataset, improving the model's adaptability to different scenarios.

[0070] The system continuously captures RGB video streams of a user performing sit-ups at 30fps using a high-definition camera at a fixed position and angle. This video is fed in real-time into a specially optimized pose estimation model based on the RTMPose-m architecture. The model infers from each input frame and outputs keypoint coordinate data, including the two-dimensional coordinates of 14 keypoints and their confidence levels.

[0071] Third, construct a motion standardization scoring network to score sit-up movements in two modes: testing and training.

[0072] This system component, acting as the core decision-making engine, runs continuously in a background thread. To provide standardized scoring for each action, it's necessary to segment each complete sit-up cycle, and a state machine is set up to record the movement phases. Initially, the key sit-up state flags T0, T1, and T2 of the state machine are set to null.

[0073] Subsequently, the scoring network receives keypoint data streams from the keypoint detection model and calculates the shoulder-hip offset angle, representing the torso posture, in real time. A sliding window of length M=7 is added to each frame to linearly fit the angle sequence, allowing the system to robustly determine the trend of angle changes (increasing, decreasing, or stabilizing), greatly enhancing the anti-interference capability of state discrimination.

[0074] The state machine switches states based on this trend: when the current frame is the first frame, it determines... Is the absolute value of the angle less than or equal to If so, T0 is set to the current frame, marking the start of a new action cycle; otherwise, the current frame is determined to be outside the action cycle, and the process waits for subsequent frames. When the angle change trend changes from decreasing to increasing, the current frame is marked with a T0 flag; when the trend changes from increasing to decreasing, the current frame is marked with a T1 flag, indicating that the body has reached the peak of the standing up position, and at this moment, an instantaneous judgment is triggered on whether the elbow touches the knee; when the trend changes from decreasing to increasing again, the T2 flag is set, the current cycle ends, and the data from the T1 to T2 phases is reviewed to determine whether the shoulder touches the mat. After the violation judgment of this cycle ends, the T1 flag is set again, and the detection of the next movement cycle begins. At the same time, the system performs continuous violation monitoring in parallel throughout the entire cycle, including whether the knee angle deviates from the standard based on the knee key point calculation and whether the hands leave the head based on the change in hand-head distance.

[0075] Finally, at time T2, the system makes a judgment based on all foul flags generated during the cycle. In test mode, if there is no foul, a "beep" sound is played and a valid count is recorded; if a foul occurs, the specific foul is announced. In training mode, the system calls a multi-dimensional scoring algorithm that includes movement amplitude, rhythm control, and posture stability, and instantly announces the score and main deductions for this movement, realizing a real-time coaching function. The movement velocity curve in training mode requires extracting the vertical displacement sequence of key shoulder joint points for each complete movement cycle. The velocity sequence is obtained through differential calculation and smoothed. The weighted scores of each indicator in this module are calculated based on the filtered shoulder joint vertical velocity to calculate the total score. The abnormal force module calculates the score based on the range of the hip lordosis angle and the maximum head and neck extension deviation angle.

[0076] IV. Construct an action comparison network to generate visual comparison videos and curve analysis charts.

[0077] This module performs offline in-depth analysis after the user completes the entire training or testing phase. The system first retrieves the user's complete motion video and corresponding keypoint sequence from the storage of the previous phase, and then calls a standard template that matches the current pattern from a pre-set standard motion template library.

[0078] To ensure the accuracy of the comparison, the system employs a dynamic time warping algorithm to non-linearly align the user's hip-shoulder angle sequence for a complete movement cycle (from T0 to T2) with the sequence of the standard template, thereby eliminating differences in movement speed. Based on this, the comparison network performs multi-dimensional analysis: on the posture trajectory, it generates a dynamic side-by-side comparison animation of the user and the standard skeleton throughout the cycle; on the data curve, it plots the time-series data of the user and the standard's hip-shoulder angle, anterior pelvic tilt angle, and shoulder joint vertical velocity on the same chart, clearly indicating differences with different colors. All these visualization elements, supplemented by targeted text analysis, such as "the descent speed is significantly faster than the standard in the middle, indicating insufficient eccentric control," are automatically integrated to generate a personalized PDF analysis report with rich graphics and text. This report not only points out the problems but also provides specific and actionable training suggestions tailored to different scenarios such as physical testing, fitness, or rehabilitation, thus completing a fully intelligent service from "problem identification" to "problem guidance."

[0079] Example 3: This embodiment provides a sit-up motion scoring device based on key point detection, such as... Figure 4 As shown, it includes: The image acquisition module, which is a hardware module or a device component integrating the hardware, is used to acquire video data of the user performing sit-ups and divide the acquired video into several frames. The hardware module supports an acquisition frame rate of ≥30fps, and the installation height and angle of the camera are adjustable to ensure that the user's full-body movements during sit-ups can be captured completely as required. In this embodiment, the hardware module includes an adjustable camera with a resolution of 1080P and a computer host for displaying images.

[0080] The key point extraction module, a software module based on the RTMPose model, is used to detect the coordinate information of the user's key skeletal points in each frame of the video using a human key point detection model; in this embodiment, the module is integrated into the computer host.

[0081] A standard storage module is used to store motion templates of different standards. This module includes sit-up videos for different usage scenarios, as well as the coordinate sequences of key points of the human skeleton in the videos, angle curves, and velocity curves. In this embodiment, this module is integrated into the computer host.

[0082] The action scoring module includes software modules for action standardization scoring and action standard comparison, which run on the terminal CPU / GPU. The scoring unit is used to calculate the comprehensive score of action standardization using a weighted scoring algorithm across multiple dimensions and to identify foul actions. The action standard comparison unit is used to compare the user's video with the action templates in the standard storage module. In this embodiment, this module is integrated into the computer host.

[0083] The results output module includes hardware modules for a display unit and a voice broadcast unit. The voice broadcast unit outputs the overall score and feedback information on non-standard items in real time via voice. The display unit outputs various results, such as analysis and comparison of the user's video and the standard video. In this embodiment, the display unit is a touch screen connected to the computer host, and the voice broadcast unit is a speaker integrated into the computer host.

[0084] As a preferred option, it also includes: The model update module is used to iteratively train the human key detection model based on new action sample data and continuously optimize the detection accuracy. The identity recognition module is used to identify the user's identity information, facilitating the storage of the user's historical exercise records to provide more detailed and targeted training suggestions. In this embodiment, it is an ID card reader.

[0085] The device's workflow is as follows: 1. The user starts the device, logs in via the ID card reader or manually logs in to their personal account on the display screen, and then selects the training mode or test mode on the touch screen; then the image acquisition module automatically focuses and prompts the user to lie down on the test mat and prepare for action. When the system detects that the user is ready for action, that is, when the start mark T0 is detected, the system will announce the countdown through the built-in speaker on the display screen. 2. The key point extraction module starts model inference and splits the video stream from the camera into video frames and extracts key point coordinate data in real time. At this time, the display screen can show the sit-up video with human skeleton after being processed by the key point extraction module. 3. After each complete sit-up, the action scoring module calculates the action parameters and updates the overall score. After the score is calculated, the voice broadcast unit in the result output module will broadcast the output result of the scoring module. If the current mode is test mode, a "beep" prompt will be broadcast for qualified actions; otherwise, the detected foul actions will be broadcast. If there are multiple foul actions, "unqualified" will be broadcast directly. If the current mode is training mode, the specific overall score will be broadcast, and the deduction items with the largest weight in the score will be broadcast, such as "lowering speed too fast", "head excessive forward extension", "elbow not touching knee", etc. When the average speed difference between the rising and falling phases of the user's action reaches the threshold, the voice broadcast unit will suggest that the user take a short rest. 4. After the process is complete, the user manually stops the device. The display unit of the result output module will show the difference between the user's action and the standard action on the screen. The content includes video comparison, graph comparison, violation action detection results, counting results and comprehensive score results at each stage. The default standard action for comparison is the standard action in the physical test mode. The user can manually switch to the standard action comparison in the exercise mode or rehabilitation mode.

[0086] Example 4: This embodiment provides a sit-up motion scoring system based on key point detection, including: The terminal device integrates all the hardware devices of the scoring device described above, including all the modules of the device, and is responsible for local real-time scoring. The cloud server communicates with the terminal devices via wired or wireless means, receives the exercise data and scoring results uploaded by the terminal devices, uses a MySQL database to store the user's exercise history and user identity information, supports multi-device synchronization of training data, and does not participate in real-time scoring calculation.

[0087] The specific scoring system process is as follows: The terminal verifies the user's identity, collects videos of the user's sit-up movements, calculates the movement score and parameters locally in real time, and synchronizes them to the cloud when connected to the network. The cloud server stores the user's identity information and historical exercise records. The terminal then generates a user training profile based on the records stored in the cloud, generates a weekly training report based on weekly data, and summarizes the correction suggestions. At the same time, the cloud server can update the key point detection model and scoring model parameters quarterly based on massive user data and push them to the terminal's model update module to improve detection accuracy and scoring accuracy.

[0088] Example 5: An electronic device, comprising a memory and a processor; The memory is used to store computer programs; The processor is configured to implement the method described in Embodiment 1 when executing the computer program.

[0089] Example 6: A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in Embodiment 1.

[0090] Example 7: A computer program product includes a computer program that, when executed by a processor, implements the method described in Example 1.

[0091] In the above embodiments, the reference to "this embodiment" in the specification indicates that a specific feature, structure, or characteristic described in connection with the embodiment is included in at least some embodiments, but not necessarily all embodiments. Multiple appearances of "this embodiment" do not necessarily refer to the same embodiment.

[0092] In the above embodiments, although the invention has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory structures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed. The embodiments of the invention are intended to cover all such substitutions, modifications, and variations falling within the broad scope of the appended claims.

[0093] As will be understood by those skilled in the art, the computer-readable storage medium described in this embodiment allows for the implementation of all or part of the steps in the above method embodiments by computer program-related hardware. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0094] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication between them. The memory is used to store computer programs, the communication interface is used to perform communication, and the processor and the transceiver are used to run the computer programs, so that the electronic terminal performs the steps of the above method.

[0095] In this embodiment, the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0096] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0097] This invention can be used in a wide range of general-purpose or special-purpose computing system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0098] This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for scoring sit-up movements based on key point detection, characterized in that: Includes the following steps: S1: Collect sit-up image data and standard sit-up video data, annotate the human body key points in the sit-up image data, and construct a sit-up key point dataset for training the key point detection model; collect standard sit-up video data and extract its standard key point sequence and kinematic features for constructing an action comparison network. S2: Build a key point detection model based on the RTMPose model, and train and optimize it using the sit-up key point dataset built in S1; acquire user sit-up video data in real time through video capture equipment, input the video data into the trained key point detection model, and detect and output key point coordinate data, confidence scores, and video data with human skeleton prediction results in real time. S3: Construct a motion standardization scoring network and input the key point detection data obtained in S2 into the motion standardization scoring network in real time; the motion standardization scoring network first performs motion phase recognition based on the key point sequence and completes the effective count accordingly; then, according to the preset rules and models, it detects foul actions and performs standardization scoring. S4: Construct an action comparison network to compare and analyze the key point data of the user's action sequence with the standard sit-up video data collected in S1, compare the angle data and action speed curves, and visualize the differences obtained from the comparison.

2. The sit-up motion scoring method based on key point detection according to claim 1, characterized in that: Step S1 includes the following steps: S11: Collect sit-up image data in multiple scenarios, including sit-up action images from the side angle of users with different lighting conditions and body types; S12: The collected sit-up image data is annotated with human key points. The annotated key points include 14 core joint points, including the head, neck, shoulders, elbows, wrists, hips, knees, and ankles, forming a sit-up image dataset. S13: Collect sit-up video data from professional coaches at different standards, including physical testing standards, fitness crunch standards, and rehabilitation standards, and extract key movement trajectory templates for the standards.

3. The sit-up motion scoring method based on key point detection according to claim 1, characterized in that: Step S2, the keypoint detection model building steps based on the RTMPose model, includes: The RTMPose model is improved to be lightweight by removing the limb key point detection branches that are not related to sit-up motion detection and retaining only 14 core key point detection branches for head, neck, shoulders, elbows, wrists, hips, knees and ankles. The spatial attention mechanism SAM is introduced to enhance the features of the core region of the sit-up movement. During model training, Mosaic data augmentation, random horizontal flipping, angle rotation, and random scaling were used to augment and expand the sit-up image dataset. S24: Input the video stream obtained from the video acquisition device into the trained RTMPose model in real time, frame by frame, and output the human key point coordinate data for each frame. The coordinate data includes the coordinate values ​​and confidence scores of the key points.

4. The sit-up motion scoring method based on key point detection according to claim 1, characterized in that: The scoring logic of the action standardization scoring network mentioned in step S3 is as follows: S31: Based on key point coordinate data, calculate multiple key angles in real time, including the shoulder angle from elbow to shoulder to hip, the elbow angle from shoulder to elbow to wrist, the hip angle from shoulder to hip to knee, the knee angle from hip to knee to ankle, and the neck angle along the line connecting the head to the neck and shoulders. S32: Record timestamps to segment each complete sit-up movement, dividing the complete sit-up movement into the rising phase and the falling phase; the three timestamps are the starting mark T0 when lying flat, the intermediate mark T1 when the elbow touches or passes the knee, and the ending mark T2 when returning to a flat position; set standard threshold ranges for each key angle, with the standard range for the elbow angle being... The standard range for the body folding angle at the hip is: ; S33: Calculate the deviation of the angle of each part from the standard threshold, identify common fouls based on the magnitude of the deviation, and record the motion velocity curve of each part. Identify common incorrect force patterns based on the temporal characteristics of the neck forward tilt angle and hip key points. The motion standardization scoring network is divided into a test mode and a training mode. The scoring components of the test mode only include foul detection, while the scoring components of the training mode include foul detection, motion velocity curve analysis, and abnormal force pattern identification. The score is then calculated based on the weight of the three components. S34: The scoring results will be broadcast in real time. In test mode, the scoring results will only be broadcast for sit-up movements that have passed the test. In training mode, the broadcast content will include the score of the current movement.

5. The sit-up motion scoring method based on key point detection according to claim 4, characterized in that: The foul detection steps include: Let point A Point B is the hip joint. Point C is the knee joint. For the elbow joint, the shoulder-hip offset angle on the side closer to the image acquisition device, i.e., the angle between the hip-shoulder line and the horizontal x-axis, is... The angle between the hip, knee, and ankle is The angle between the shoulder, elbow, and wrist is The Euclidean distance between the wrist and the head is The horizontal distance from the knee joint to the elbow joint is Based on spatial features and time series analysis and judgment, start flag T0, middle flag T1 and end flag T2 are set. The angle values ​​of each part are calculated according to the coordinate results of the key point detection model. Then, the angle value and coordinate value of each frame are used to detect auxiliary flags to determine the action stage of the current sit-up cycle through the state machine. Then, corresponding foul action detection is performed in different action stages. The shoulder-hip offset angle is used to determine three auxiliary indicators. If the user's right shoulder bone is closer to the image acquisition device, the right shoulder-right hip offset angle is calculated; otherwise, the left shoulder bone angle is calculated. Time-series parameters are added, and a sliding window of length M is added to each frame to linearly fit the angle sequence. A threshold value for the slope k of the fitted curve is set. To robustly determine the trend of angle changes; During initialization, T0, T1, and T2 are all set to Null; subsequently, the hip-shoulder offset angle is calculated for each processed frame. The trend and coordinate values ​​are used for iterative judgment: if the current frame is the first frame, if... The absolute value of the angle is less than or equal to If T0 is not within the specified range, then T0 is set as the current frame, marking the start of a new action cycle. If it is not within the specified range, the current frame is not within the action cycle, and the system waits for the judgment of subsequent frames. When the angle change trend changes from decreasing to increasing, the current frame is set to the T0 flag; when the detected... When the trend of change changes from increasing to decreasing, T1 is set as the current frame, marking the peak of the body's movement; when it is detected again... When the trend of change changes from decreasing to increasing, T2 is set as the current frame, marking the end of the cycle and returning to the initial posture. At the same time, the frame number where the T2 flag is located is directly set as the frame where the T0 flag of the next action cycle is located. Within this state machine framework, the detection of foul actions is embedded in the corresponding stages. Flag4 "Knee angle not up to standard" and flag2 "Hands not behind head" are continuously monitored and checked in real-time throughout the entire action cycle from T0 to T2. Once a violation is detected, a corresponding flag is immediately set. Flag3 "Elbow not touching knee" is a key posture check item, triggered and judged once at time T1, when the body fold is at its minimum. For the judgment of flag1 "Shoulder not touching mat," at the end of the cycle, the shoulder-hip offset angle during the T1-T2 state is detected. Has the set threshold been reached? If not, set flag1 "Shoulder not touching the pad" to TRUE. The specific criteria for the foul "knee angle not meeting the standard" (flag 1) are: the hip-knee-ankle angle of the left and right knees must be within the specified range. All If it is inside, otherwise it is judged as a foul; The specific criteria for the "hands not covering head" foul in flag 2 are: the shoulder-elbow-wrist angle of the left and right elbows must be within the specified range. All satisfied Meanwhile, the Euclidean distance between the wrist and the head satisfies Otherwise, it is a foul; among them, l is the torso length of the user at the first time T0, which is the length of the line connecting the midpoint of the left and right shoulders to the midpoint of the left and right hips. The specific criteria for the "elbow not touching knee" foul in flag3 are as follows: at time T1, the horizontal Euclidean distance between the elbow and knee joints on both sides must satisfy... The unit is pixels; otherwise, it is considered a foul. The specific determination of the foul "shoulder not touching the mat" in flag4 is: the shoulder-hip offset angle on the side closest to the image acquisition device during the T1 to T2 state. At least one frame satisfies Otherwise, it is judged as a foul; among them, For the first T0 time point If the user starts moving before the system prompt sound, resulting in storage... Then set to ; The above parameters involve the calculation of the angle between two vectors and the Euclidean distance. Assuming vectors... and The coordinates are respectively , and , Then the included angle The calculation formula is: Suppose the coordinates of the two points for which the Euclidean distance is to be calculated are... and Then the formula for calculating the Euclidean distance D is: When a complete cycle ends with the T2 flag, the cycle is comprehensively scored based on the flags of the foul action types stored in the current cycle: the states of the four foul flags flag1 to flag4 are retrieved; in test mode, if all flags are False, the movement in this cycle is considered qualified; if any foul flag exists, it is considered unqualified; in training mode, scoring is performed on a 100-point scale, with the four foul action types equally weighted. Flags with a TRUE foul flag are counted, and points are deducted according to their corresponding weights. When all four flags are TRUE, that portion of the score is... 0; then, before T2 is set to Null, the current motion cycle is broadcast with voice: in test mode, when there is no foul flag that meets the conditions, the count is effectively counted through voice broadcast; if there is a foul flag, the specific foul content or multiple fouls are selectively broadcast according to preset rules; in training mode, the voice broadcast content is combined with the analysis of the motion speed curve and the identification of abnormal force patterns; after the adjudication and broadcast are completed, the system resets T1 and T2 to Null, retains the T0 state of the current frame, clears all foul flag bits, and then enters the detection process for the next motion cycle.

6. The sit-up motion scoring method based on key point detection according to claim 4, characterized in that: The motion velocity curve analysis process is as follows: After obtaining a complete motion cycle defined by T0, T1, and T2, the user's motion velocity curve is extracted and analyzed; the vertical displacement sequence of the key points of the shoulder joint in each complete motion cycle is extracted, the velocity sequence is obtained through differential calculation, and then smoothed and filtered. The four core indicators are then broken down and calculated according to the following table, and scores for each dimension are obtained according to the scoring method. Finally, the total score for speed control in this action is obtained by weighted summation. The four core indicators include the standard deviation of speed during the ascent phase, the standard deviation of speed during the descent phase, the maximum speed during the descent phase, and the ascent / descent time ratio.

7. The sit-up motion scoring method based on key point detection according to claim 4, characterized in that: The abnormal force application pattern recognition process is as follows: In training mode, firstly, within a complete movement cycle determined by the state machine, the coordinate sequences of key points of the hip, neck and head are extracted; then, based on the key point detection data, biomechanical analysis is performed on the user's force exertion pattern to identify abnormal patterns of iliopsoas muscle-dominated compensation and neck muscle group compensation, as well as movement coordination. For iliopsoas muscle compensation, the rate of change of pelvic tilt angle is calculated. The pelvic tilt angle is defined as the angle between the line connecting the two hip joints and the horizontal line. The hip axis vector is calculated using the key points of the left and right hips, and then the angle between this vector and the horizontal axis (1,0) is calculated. For neck compensation, calculate the forward extension angle of the head and neck, and calculate the maximum value of the angle between the perpendicular line connecting the head key point and the neck key point and the line connecting the two shoulders. To assess the coordination of movements, the consistency of the head-shoulder-pelvis movement sequence is calculated. Specifically, this is determined by the displacement sequence of the head, the midpoint of the shoulder, and the midpoint of the pelvis in the Y direction. The vertical movement velocities of the three are calculated, the peak positions of each velocity curve during the rising phase are found, and the matching degree between the actual peak sequence and the ideal sequence is calculated. The ideal peak sequence is hip-shoulder-head. After calculating all indicators, scores for each dimension are obtained according to the scoring method. Finally, the total score of this action in terms of force exertion mode is obtained by weighted summation. The comprehensive score and the performance of each item are combined to generate voice feedback for the item with the largest deduction, which directly guides the user to carry out corrective training. In training mode, the formula for the comprehensive movement quality score for each exercise cycle is: Where w1 is the weighting coefficient of the foul action detection module, w2 is the weighting coefficient of the motion speed curve analysis module, and w3 is the weighting coefficient of the abnormal force application pattern recognition module. This represents the total number of foul categories in the foul detection module. The number of detected foul categories; the score of the motion speed curve analysis module. The formula is: The standard deviation of the vertical velocity of the shoulder joint during the ascending phase T0~T1 , The vertical velocity of the shoulder joint during the upward phase. The average velocity of the vertical velocity of the shoulder joint during the rising phase; The standard deviation of the vertical velocity of the shoulder joint during the descent phase T1~T2 is given. This represents the maximum vertical velocity of the shoulder joint during the descent phase. Score for abnormal force pattern recognition The formula is: in, Pre-exercise period Hip anteversion angle at this stage This represents the maximum forward head and neck deflection angle during the entire movement cycle. The score is obtained by the order in which the movement velocities of different head-shoulder-hip joints peak.

8. The sit-up motion scoring method based on key point detection according to claim 1, characterized in that: In step S4, the visualization of the difference results from the action alignment network is as follows: S41: Extract key angle data sequences and motion speed curve data of user actions and standard actions during the rising and falling phases, and use dynamic time warping algorithm combined with action phase constraints to segment and align motion sequences at different speeds. S42: Calculate the differences between user actions and standard actions in terms of angle data, velocity curves, and skeletal point trajectory dimensions, generate a difference heatmap, and highlight the difference areas with color coding. S43: Generate a visual report, including side-by-side display of user action videos and standard action videos, angle-time curve comparison charts, and pie charts of deduction statistics. Users can intuitively view their shortcomings in action through the report and make targeted improvements.

9. A sit-up motion scoring device based on key point detection, characterized in that: The apparatus applicable to the sit-up motion scoring method based on key point detection as described in any one of claims 1-8 includes: The image acquisition module is used to acquire video data of the user performing sit-ups and divide the acquired video into several frames. The key point extraction module, based on the RTMPose model, uses a human key point detection model to detect the coordinate information of the user's key skeletal points in each frame of the video. The standard storage module is used to store motion templates of different standards. The motion templates of different standards include sit-up videos in different usage scenarios, as well as the coordinate sequence of key points of human skeleton in the videos, angle curves and speed curves. The action scoring module includes an action standardization scoring unit and an action standard comparison unit. The action standardization scoring unit is used to calculate the comprehensive action standardization score using a weighted scoring algorithm across multiple dimensions and to identify foul actions. The action standard comparison unit is used to compare the user's video with the action templates in the standard storage module. The results output module includes a display unit and a voice broadcast unit. The voice broadcast unit is used to output the comprehensive score and feedback information on non-standard items in real time in voice form. The display unit is used to output various results such as analysis and comparison of user videos and standard videos. The model update module is used to iteratively train the human key detection model based on new action sample data and continuously optimize the detection accuracy. The identity recognition module is used to identify the user's identity information and facilitate the storage of the user's historical exercise records to provide more detailed and targeted training suggestions.

10. A sit-up motion scoring system based on key point detection, characterized in that: include: The terminal device integrates the sit-up motion scoring device based on key point detection as described in claim 9, and realizes local real-time scoring. The cloud server communicates with the terminal device via wired or wireless means, receives the exercise data and scoring results uploaded by the terminal device, stores the user's exercise history and provides statistical analysis.

Images