Pull-up process action decomposition and key frame extraction method and computer equipment

By extracting key points of the human body and calculating the motion trajectory, combined with the standard action model library, the accuracy and personalization of action analysis in pull-up teaching is solved, and the automatic decomposition and keyframe extraction of actions are realized, which improves teaching efficiency and analysis accuracy.

CN119068555AActive Publication Date: 2024-12-03ZHUHAI XINWEI TECHNOLOGY CO LTD

Patent Information

Application Number
CN202411484752.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-23
Publication Date
2024-12-03
Estimated Expiration
2044-10-23

AI Technical Summary

Technical Problem

In the process of pull-up teaching, it is difficult for students to accurately grasp the key points of the movements. Teachers need to frequently pause and watch videos to correct mistakes. The existing action analysis methods lack accurate and objective evaluation, and it is difficult to accurately capture the critical moments of the movements when keyframe extraction is difficult. It is impossible to fully reflect the motion characteristics in three-dimensional space.

Method used

By collecting video data, extracting key points of the human body, calculating motion trajectory and velocity curves, comparing with the preset standard action model library, dividing the action stages and selecting keyframes, using low-pass filters and differential methods to process the data, and matching the dynamic time regularization algorithm Model, dynamically adjust the division standards and frame selection to ensure the representativeness and personalization of frames.

Benefits of technology

It realizes automatic capture of the action process, accurately divide the action stages, extracts representative keyframes, generates detailed static breakdown diagrams of the action key points, provides personalized improvement suggestions, and improves teaching efficiency and the accuracy of action analysis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068555A_ABST
    Figure CN119068555A_ABST
Patent Text Reader

Abstract

The invention relates to a pull-up process action decomposition and key frame extraction method and computer equipment. The method comprises the following steps: acquiring a key point coordinate sequence; calculating a motion trail and a speed curve of the human body key points; selecting a most matched standard model from a preset standard action model library; determining a key stage division standard of the pull-up action of the student; dividing the pull-up action of the student into N key stages; selecting a candidate key frame corresponding to the key stage; performing optimization screening on the candidate key frames to obtain final key frames; and comparing the static decomposition graph with the best matching standard model, and giving an improvement suggestion based on a comparison result. According to the method, the complete action process can be automatically captured, the action stages can be accurately divided, the most representative key frame can be extracted from each stage, the detailed action main point static decomposition graph can be generated, the individual characteristics of students are considered, and personalized improvement suggestions are provided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to a method and computer device for decomposing and extracting key frames during a pull-up process. Background Art

[0002] In the process of teaching pull-ups, teachers need to demonstrate and explain the students' movements, but due to the continuity and rapidity of the movement process, students often find it difficult to accurately grasp the key movements. At the same time, when correcting students' movements, teachers also need to frequently pause and replay the demonstration video to find the key mistakes, which greatly reduces the teaching efficiency.

[0003] Existing motion analysis methods mainly rely on manual observation and subjective judgment, which makes it difficult to provide accurate and objective evaluation. Although some systems use a single camera to capture motion, they cannot fully reflect the motion characteristics in three-dimensional space. In addition, existing technologies often use fixed time intervals or simple motion thresholds in key frame extraction, which makes it difficult to accurately capture the key moments in the action.

[0004] In terms of action comparison and analysis, current methods are mostly limited to simple trajectory comparison or key point position comparison, lacking a comprehensive evaluation of the overall coordination and efficiency of the action. At the same time, when generating improvement suggestions, existing systems can only provide general guidance and it is difficult to give precise improvement plans based on individual differences of students. Summary of the invention

[0005] Based on this, it is necessary to provide a pull-up process action decomposition and key frame extraction method and computer equipment that can automatically capture the complete action process to address the above technical problems.

[0006] A method for decomposing and extracting key frames of a pull-up process comprises the following steps: Collect and preprocess the video data of students doing pull-ups, extract the key points of the human body from the preprocessed video data, and obtain the key point coordinate sequence; According to the key point coordinate sequence, calculate the motion trajectory and speed curve of the key points of the human body; Compare the motion trajectory and speed curve with the preset standard motion model library, and select the most matching standard model from the preset standard motion model library; According to the motion trajectory, speed curve and the most matching standard model, determine the key stage division criteria of students' pull-up movements; According to the key stage division criteria, key point coordinate sequence and speed curve, the students' pull-up movements are divided into N key stages; According to the key point coordinate sequence, motion trajectory and speed curve, select the candidate key frames corresponding to the key stage; The candidate key frames are optimized and screened to obtain final key frames; the key stage corresponds to at least one final key frame, and the number of final key frames is within a preset range.

[0007] In one embodiment, the step of calculating the motion trajectory and speed curve of the key points of the human body according to the key point coordinate sequence includes the steps of: The low-pass filter is used to slide in the time dimension to perform local polynomial fitting on the coordinate values ​​in the key point coordinate sequence to obtain the motion trajectory; The motion trajectory is processed using the difference method to obtain the preliminary velocity curve; The speed curve is obtained by sliding a low-pass filter in the time dimension and performing local polynomial fitting on the speed values ​​in the preliminary speed curve.

[0008] In one embodiment, the preset standard action model library includes N standard models of pull-up actions; the standard models are divided into high-level standard models, medium-level standard models and low-level standard models; The step of comparing the motion trajectory and the speed curve with a preset standard motion model library and selecting the most matching standard model from the preset standard motion model library includes the following steps: Preprocessing the motion trajectory, velocity curve and standard model; the preprocessing includes time normalization and space normalization; Using a dynamic time warping algorithm, a first dynamic time warping distance between the preprocessed motion trajectory and the standard motion trajectory in the preprocessed standard model is calculated, and a second dynamic time warping distance between the preprocessed velocity curve and the standard velocity curve in the preprocessed standard model is calculated; Obtaining a comprehensive distance index according to the first dynamic time warping distance, the second dynamic time warping distance, and the dynamic time warping weights corresponding to the key points of the human body; The standard model corresponding to the smallest comprehensive distance index is taken as the best matching standard model.

[0009] In one embodiment, the step of determining the key stage division criteria of the student's pull-up action according to the motion trajectory, the speed curve and the best matching standard model includes the steps of: Extract key parameters from the most matching standard model, including speed threshold, position threshold and time ratio corresponding to each stage of the pull-up action; The motion trajectory, speed curve, speed threshold, position threshold and time ratio are compared to obtain the key stage division criteria; the key stage division criteria include the start and end judgment conditions of the key stage, the time ratio range of the key stage in the entire pull-up action and the dynamic adjustment rules.

[0010] In one embodiment, the key stages include a preparation stage, a pull-up stage, a transition stage, and a lowering stage; According to the key stage division criteria, key point coordinate sequence and speed curve, the students' pull-up movements are divided into N key stages, including the following steps: Starting from the first frame of the key point coordinate sequence, the first vertical speed of the student's wrist and hip is continuously monitored according to the speed curve, and the frame where the first vertical speed meets the corresponding start and end judgment conditions is marked as the end point of the preparation phase and the start point of the pull-up phase; After identifying the end point of the preparation phase and the start point of the pull-up phase, the head position and the second vertical speed of the wrist of the student are continuously monitored according to the key point coordinate sequence, and the frames in which the head position and the second vertical speed meet the corresponding start and end judgment conditions are marked as the end point of the pull-up phase and the start point of the transition phase; After identifying the end point of the pull-up phase and the start point of the transition phase, the third vertical velocity of the student's hip is continuously monitored according to the velocity curve, and the frame in which the third vertical velocity meets the corresponding start and end judgment conditions is marked as the end point of the transition phase and the start point of the lowering phase; After identifying the end point of the transition phase and the start point of the lowering phase, the fourth vertical speed of the student's wrist and hip is continuously monitored according to the speed curve, and the frame where the fourth vertical speed meets the corresponding start and end judgment conditions is marked as the end point of the lowering phase; According to the time ratio range, check and adjust the end point of the preparation phase, the start point of the pull-up phase, the end point of the pull-up phase, the start point of the transition phase, the end point of the transition phase and the start point of the lowering phase to divide the students' pull-up movements into the preparation phase, the pull-up phase, the transition phase and the lowering phase; In the process of key stage division, the start and end judgment conditions and the time ratio range are dynamically adjusted according to the dynamic adjustment rules, key point coordinate sequence and speed curve.

[0011] In one embodiment, the step of selecting candidate key frames corresponding to the key stage according to the key point coordinate sequence, the motion trajectory and the speed curve includes the steps of: In the preparation stage, the average position of the student's hip and shoulder in the preparation stage is calculated according to the motion trajectory; the offset distance between the student's hip position and shoulder position of each frame in the preparation stage and the average position is calculated according to the motion trajectory, and the frame corresponding to the minimum offset distance is used as the candidate key frame corresponding to the preparation stage; In the pull-up stage, the average vertical speed in the pull-up stage is calculated according to the speed curve, and the frame closest to the average vertical speed is marked from the speed curve as the first feature frame; the change speed of the student's elbow angle is analyzed according to the motion trajectory, and the frame corresponding to the maximum change speed is marked as the second feature frame; the vertical relative position of the student's wrist and shoulder is analyzed according to the motion trajectory, and the frame corresponding to the vertical relative position of zero is taken as the third feature frame; the candidate key frame corresponding to the pull-up stage is selected from the first feature frame, the second feature frame and the third feature frame; In the transition stage, the frame with the highest head position of the student is marked as the fourth feature frame according to the motion trajectory; the angle of the student's elbow is calculated according to the motion trajectory, and the frame corresponding to the minimum elbow angle is marked as the fifth feature frame; the frame where the student's hip begins to move forward is marked as the sixth feature frame according to the motion trajectory; and the candidate key frame corresponding to the transition stage is selected from the fourth feature frame, the fifth feature frame, and the sixth feature frame; In the lowering stage, the average vertical speed of the lowering stage is calculated according to the speed curve, and the frame closest to the average vertical speed on the speed curve is marked as the seventh feature frame; the changing speed of the student's elbow angle is analyzed according to the motion trajectory data, and the frame corresponding to the maximum changing speed is marked as the eighth feature frame; the change in the angle between the student's hip and shoulder is calculated according to the motion trajectory, and the frame where the student starts to lean forward is marked according to the change in the angle and used as the ninth feature; the candidate key frames corresponding to the lowering stage are selected from the seventh feature frame, the eighth feature frame and the ninth feature frame.

[0012] In one embodiment, the step of optimizing and screening the candidate key frames to obtain the final key frame includes the following steps: Sort the candidate key frames in chronological order to obtain a preliminary sorting; Based on the preliminary sorting, the first candidate key frame in the preparation phase is identified as the final key frame; Traverse the remaining candidate key frames that are preliminarily sorted, and calculate the time interval between the current candidate key frame and the last selected final key frame during the traversal process; Based on the preliminary judgment condition that the time interval is greater than the preset time interval threshold, and combined with the key features contained in the candidate key frames, the remaining final key frames are screened, and the first final key frame and the remaining final key frames are combined into a final key frame sequence.

[0013] In one embodiment, the following steps are also included: Detecting time intervals between adjacent final key frames in a final key frame sequence; When it is detected that the time interval between two adjacent final key frames exceeds the preset maximum time interval threshold: obtain the speed values ​​of all skipped candidate key frames between the two adjacent final key frames whose time interval exceeds the preset maximum time interval threshold; for each candidate key frame, calculate the sum of the absolute value of the speed difference between the current candidate key frame and the previous candidate key frame and the absolute value of the speed difference between the current candidate key frame and the next candidate key frame as the speed change significance index of the current candidate key frame; select the candidate key frame with the largest speed change significance index as an additional key frame; add the additional key frame to the final key frame sequence.

[0014] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0015] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0016] One of the above technical solutions has the following advantages and beneficial effects: The present application comprises the following steps: collecting video data of students doing pull-ups and preprocessing them, extracting key points of the human body from the preprocessed video data, and obtaining a sequence of key point coordinates; calculating the motion trajectory and speed curve of the key points of the human body according to the sequence of key point coordinates; comparing the motion trajectory and speed curve with a preset standard action model library, and selecting the most matching standard model from the preset standard action model library; determining the key stage division criteria of the students' pull-ups according to the motion trajectory, speed curve and the most matching standard model; dividing the students' pull-ups into N key stages according to the key stage division criteria, the sequence of key point coordinates and the speed curve; selecting candidate key frames corresponding to the key stages according to the sequence of key point coordinates, the motion trajectory and the speed curve; optimizing and screening the candidate key frames to obtain the final key frames; the key stages correspond to at least one final key frame, and the number of final key frames is within a preset range. The present application can automatically capture the complete action process, accurately divide the action stages, extract the most representative key frames from each stage, generate a detailed static decomposition diagram of the action essentials, and provide personalized improvement suggestions by considering the individual characteristics of the students. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flowchart of the method for decomposing the pull-up process movements and extracting key frames in the embodiment of the present application.

[0018] Figure 2 Schematic diagram of the internal structure of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0020] In order to achieve the above object, in one embodiment, Figure 1 As shown, a method for decomposing and extracting key frames of a pull-up process is provided, comprising the following steps: Step S110, collecting and preprocessing the video data of the student doing pull-ups, extracting the key points of the human body from the preprocessed video data, and obtaining a key point coordinate sequence.

[0021] In the campus environment, fixed high-definition webcams are installed near the pull-up machines. Each machine is equipped with two cameras, one for front view and one for side view, to obtain a comprehensive view of the action. The system uses a 1080p resolution, 30fps frame rate webcam connected to the school's local area network via a wireless network to ensure stable data transmission.

[0022] When students use the system, they need to enter their student ID number on the touch screen terminal, and the system retrieves the student's basic information for subsequent analysis. When the student is ready, click the "Start" button, and the system automatically starts recording, with a preset duration of 30 seconds. After recording is completed, the system quickly checks the video to ensure that the student's entire body is in the frame.

[0023] The preprocessing phase begins immediately after video acquisition. First, noise removal is performed, using the Gaussian filter algorithm to remove noise from the image. Next, background separation is performed, using a mixed Gaussian model (MOG2) for background modeling, and then background subtraction is applied to obtain a foreground mask.

[0024] The motion detection phase uses the frame difference method to detect the motion area, and applies morphological operations to the detected area to remove small noise points. The system determines the minimum rectangular region (ROI) containing the student and crops the original video to this region to reduce the amount of data for subsequent processing.

[0025] To improve image quality, the system applies adaptive histogram equalization (CLAHE) to the ROI area. Then the frame rate is adjusted to ensure that the video frame rate is 30fps. If the frame rate is higher than 30fps, it is downsampled; if it is lower than 30fps, linear interpolation is used to fill in the frames.

[0026] The system uniformly adjusts the resolution of the ROI area to 640x480 and uses the bicubic interpolation algorithm for scaling. To facilitate subsequent analysis, a millisecond timestamp is added to the upper left corner of each frame.

[0027] Finally, the system converts the processed video into MP4 format encoded with H.264 and sets the bit rate to 2Mbps. After preprocessing, the system automatically checks the video quality, including brightness uniformity, student outline clarity, and timestamp correctness. Qualified video files are stored on the school's central server and the relevant metadata is recorded in the database.

[0028] In the process of extracting human key points from preprocessed video data and obtaining the key point coordinate sequence, the system first loads a pre-trained deep learning model based on the improved OpenPose framework, which has been trained on a large-scale human motion dataset and has high accuracy and robustness. This model has been specially optimized to effectively handle various changes in students' pull-up movements in school sports scenes. The model receives preprocessed 640x480 resolution video frames as input and outputs the two-dimensional coordinates of 8 human key points. These 8 key points include left and right wrists, left and right shoulders, left and right elbows, hip center points and head vertices. Their selection is based on the principles of human kinematics and the characteristics of pull-up movements to ensure that the key motion features in the movement can be accurately captured. For example, the key points of the wrist and elbow are crucial for analyzing the movement of the upper arm and forearm, while the hip center point helps to evaluate the overall posture and balance of the body.

[0029] The key point extraction process uses a sliding window technique, with 5 frames as a processing unit and a step size of 1 frame to gradually process the entire video sequence. This method utilizes timing information to improve the accuracy and stability of key point detection. By considering the relationship between adjacent frames, the system can better handle detection difficulties caused by changes in perspective, lighting changes, or partial occlusion. The selection of a window size of 5 frames is based on the analysis of the characteristics of the pull-up action. It strikes a balance between temporal resolution and computational efficiency, and can capture the continuity of the action without introducing too much delay.

[0030] The entire processing flow includes three core steps: feature extraction and heat map generation, time series integration, key point positioning and skeleton connection. In the feature extraction and heat map generation stage, the system uses the ResNet-50 network to extract features from each frame of 640x480 resolution images. ResNet-50 contains 50 convolutional layers and pooling layers, and outputs feature maps of 2048 channels. These feature maps are then input into a deconvolution network, which consists of 3 transposed convolutional layers, each with a convolution kernel size of 4x4 and a stride of 2. The last layer has 8 output channels, corresponding to 8 key points. Each channel generates a heat map of the same size as the input image, which represents the probability of the corresponding key point at each position in the image.

[0031] In the temporal integration stage, the system uses a 3D convolutional neural network to process the heat map of 5 consecutive frames. The network contains 3 layers of 3D convolutional layers, each with a convolution kernel size of 3x3x3 and a step size of (1,1,1). The first layer has 8 input channels (corresponding to 8 key points) and 16 output channels; the second layer has 32 output channels; and the third layer still has 8 output channels. Each 3D convolutional layer is followed by a ReLU activation function and a batch normalization layer. This process integrates the information of the time dimension into the heat map of each frame, further improving the accuracy of key point detection.

[0032] The final key point location and skeleton connection stage is divided into two sub-steps. First, the system applies the non-maximum suppression (NMS) algorithm to the heat map after temporal integration to locate the key points. NMS uses a 3x3 sliding window on each heat map to retain the local maximum and suppress the surrounding non-maxima. For each key point, the position with the highest probability value on the heat map is selected as the coordinate of the key point. Then, the system uses a predefined skeleton model to connect the detected key points. The skeleton model defines 7 connections: left shoulder to left elbow, left elbow to left wrist, right shoulder to right elbow, right elbow to right wrist, left shoulder to right shoulder, midpoint of two shoulders to hip center, hip center to head vertex. The system traverses these predefined connections and connects the corresponding key point coordinates to form a complete skeleton structure.

[0033] Through this processing flow, the system finally outputs the two-dimensional coordinates of 8 key points in each frame (i.e., the key point coordinate sequence). The structure of the output data is a three-dimensional tensor with the dimensions of number of frames × 8 × 2, where 8 represents 8 key points and 2 represents the x and y coordinates of each key point.

[0034] Step S120, calculating the motion trajectory and speed curve of the key points of the human body according to the key point coordinate sequence.

[0035] For each key point, its motion trajectory during the entire action can be expressed as a series of two-dimensional coordinate points. For example, the motion trajectory of the left wrist can be expressed as {(x_t, y_t)|t=1,2,...,T}, where T is the total number of frames in the video and (x_t, y_t) is the coordinate of the left wrist in the tth frame. The system performs the same processing on all 8 key points, obtaining 8 sets of key point coordinate sequences.

[0036] In one of the examples, the steps of calculating the motion trajectory and speed curve of key points of a human body according to a key point coordinate sequence include the following steps: sliding a low-pass filter on a time dimension, performing local polynomial fitting on the coordinate values ​​in the key point coordinate sequence, and obtaining a motion trajectory; processing the motion trajectory using a differential method to obtain a preliminary speed curve; sliding a low-pass filter on a time dimension, performing local polynomial fitting on the speed values ​​in the preliminary speed curve, and obtaining a speed curve.

[0037] In order to obtain a more accurate and smooth motion trajectory, the system applies the Savitzky-Golay filter to the human body key point coordinate sequence. This filtering method can effectively smooth the data while retaining the high-order moments of the signal. It is particularly suitable for processing motion data that may be noisy but needs to retain fast-changing features. The filter window size is set to 15 frames and the polynomial order is 3.

[0038] The filtering process is as follows: For each key point of the human body, the system applies the Savitzky-Golay filter to its x-coordinate sequence and y-coordinate sequence respectively. The filter slides in the time dimension and performs local polynomial fitting on the coordinate value of each time point. Specifically, for time point t, the filter considers a total of 15 points before and after it (if t is close to the beginning or end of the sequence, the number of points considered is reduced accordingly), fits these points with a third-order polynomial, and then uses the value of the fitting polynomial at point t as the filtered coordinate value. This process can be expressed as: x'_t=Σ(c_n*x_{t+n})(n from -7 to 7) y'_t=Σ(c_n*y_{t+n})(n from -7 to 7) Where c_n is the coefficient of the Savitzky-Golay filter, which is determined by the window size and the polynomial order. (x'_t,y'_t) is the coordinate after filtering.

[0039] Through this filtering process, the system obtains smooth motion trajectories that eliminate noise caused by detection errors or small jitters while retaining key features of the pull-up movement, such as the changes in the fast pull-up and slow lowering.

[0040] The motion trajectories are stored in the form of a two-dimensional coordinate sequence. Each key point k (k=1,2,...,8, corresponding to the left and right wrists, left and right shoulders, left and right elbows, hip center points and head vertices) corresponds to a sequence Tk={(x'_k,t,y'_k,t)|t=1,2,...,T}. T is the total number of frames in the video. These motion trajectories intuitively show the spatial position changes of each key point of the human body during the pull-up process.

[0041] Next, the system calculates the velocity curve based on the smoothed trajectory data. The velocity calculation uses the central difference method, which is more accurate than simple forward difference or backward difference and can better handle nonlinear changes. For any time point t, the velocity v_t is calculated as follows: v_x_t=(x'_{t+1}-x'_{t-1}) / (2*Δt) v_y_t=(y'_{t+1}-y'_{t-1}) / (2*Δt) Where Δt is the time interval between adjacent frames, usually 1 / 30 second (based on 30fps video). v_x_t and v_y_t are the velocity components in the x and y directions at time t, respectively.

[0042] For points near the beginning and end of the sequence, the system calculates the velocity using forward or backward differencing to ensure a complete velocity profile. Specifically: For t=1: v_x_1=(x'_2-x'_1) / Δt v_y_1=(y'_2-y'_1) / Δt For t=T: v_x_T=(x'_T-x'_{T-1}) / Δt v_y_T=(y'_T-y'_{T-1}) / Δt To further improve the quality of the velocity curve, the system applies the Savitzky-Golay filter again to the calculated velocity data. This time the filtering uses a smaller window size (7 frames) and a lower polynomial order (2nd order) to smooth out the small fluctuations that may exist in the velocity curve while retaining the main features of the velocity changes. The filtering process is similar to the filtering of the coordinate data, but with different parameters: v'_x_t=Σ(c'_n*v_x_{t+n})(n from -3 to 3) v'_y_t=Σ(c'_n*v_y_{t+n})(n from -3 to 3) Where c'_n is the new Savitzky-Golay filter coefficient and (v'_x_t,v'_y_t) is the velocity value after filtering.

[0043] Through these steps, the system obtains the smooth motion trajectory and corresponding speed curve of each key point of the human body during the entire pull-up process. The speed curve data is stored in the form of two one-dimensional arrays, corresponding to the speed in the x direction and the speed in the y direction respectively. Each key point k of the human body has a corresponding speed curve Vk={(v'_x_k,t,v'_y_k,t)|t=1,2,...,T}, where T is the total number of frames in the video.

[0044] These motion trajectories and speed curve data fully describe the motion state of the key points of the human body in space, including the changes in position and speed. The motion trajectory reflects the movement path of the key points of the human body in a two-dimensional plane, while the speed curve shows the changes in the speed of these key points in the x and y directions.

[0045] For example, by observing the movement trajectory of the wrist, you can intuitively see the rising height and movement path of the pull-up; and its speed curve reflects the speed changes during the pull-up and lowering process. The movement trajectory of the hip can show the movement of the body's center of gravity, and its speed curve reflects the rhythm of the overall movement. The movement trajectory and speed curve of the shoulder can be used to analyze the movement characteristics of the upper body.

[0046] Step S130, comparing the motion trajectory and speed curve with a preset standard motion model library, and selecting the most matching standard model from the preset standard motion model library.

[0047] Among them, the preset standard action model library includes N standard models of pull-up actions; the standard models are divided into high-level standard models, medium-level standard models and low-level standard models. The preset standard action model contains multiple standard models of pull-up actions at different levels. These models are divided into three levels: high, medium and low. The low-level standard model reflects the basic action form, the medium-level standard model reflects the standard action requirements, and the high-level standard model shows high-efficiency and high-stability action performance. This division enables the system to select the most appropriate reference standard for students of different levels, thereby improving the accuracy of analysis and guidance.

[0048] Each standard model m contains data in the same format: Motion trajectory Tm,k={(x_m,k,t,y_m,k,t)|t=1,2,...,T_m} and the velocity curve Vm,k={(v_x_m,k,t,v_y_m,k,t)|t=1,2,...,T_m}, where T_m is the frame number of the standard model.

[0049] In one example, the motion trajectory and speed curve are compared with a preset standard motion model library, and the step of selecting the best matching standard model from the preset standard motion model library includes the steps of: preprocessing the motion trajectory, speed curve and standard model; the preprocessing includes time normalization and space normalization; using a dynamic time warping algorithm to calculate the first dynamic time warping distance between the preprocessed motion trajectory and the standard motion trajectory in the preprocessed standard model, and calculate the second dynamic time warping distance between the preprocessed speed curve and the standard speed curve in the preprocessed standard model; obtaining a comprehensive distance index based on the first dynamic time warping distance, the second dynamic time warping distance and the dynamic time warping weights corresponding to the key points of the human body; and taking the standard model corresponding to the smallest comprehensive distance index as the best matching standard model.

[0050] 1. In order to make an effective comparison, the system first preprocesses the student data (including motion trajectory and speed curve) and the standard model, including time normalization and space normalization: (1) Time normalization: Since the time it takes for students to complete an action may be different from the standard model, all data need to be resampled to the same number of time points N (for example, N = 100). This process uses a linear interpolation method.

[0051] For the student data Tk and Vk, for each human key point k, the data of T time points are resampled to N time points: Tk_n={(x_k,τ,y_k,τ)|τ=1,2,...,N} Vk_n={(v_x_k,τ,v_y_k,τ)|τ=1,2,...,N} Among them, (x_k,τ,y_k,τ) and (v_x_k,τ,v_y_k,τ) are the position and velocity of the τth sampling point obtained by linear interpolation. τ (Greek letter tau) is used here to represent the normalized time point index to distinguish it from the 'n' in the sequence name.

[0052] For each human keypoint k in the standard model m: Tm,k_n={(x_m,k,τ,y_m,k,τ)|τ=1,2,...,N} Vm,k_n={(v_x_m,k,τ,v_y_m,k,τ)|τ=1,2,...,N} The n in Vk_n, Vm, k_n all stands for "normalized time" (time normalization) (2) Spatial normalization: To eliminate the effect of height differences, all spatial coordinates were divided by the corresponding height.

[0053] For the student data, assuming that the student's height is H: Tk_s={(x_k,τ / H,y_k,τ / H)|τ=1,2,...,N} For the standard model m, assume that the corresponding standard height is H_m: Tm,k_s={(x_m,k,τ / H_m,y_m,k,τ / H_m)|τ=1,2,...,N} The s in Tk_s and Tm,k_s stands for "spatiallynormalized".

[0054] Note: The velocity data (Vk_n and Vm,k_n) do not need to be spatially normalized because they are already relative quantities.

[0055] 2. After preprocessing, the system starts data matching. The matching process uses the dynamic time warping (DTW) algorithm. For each key point k of the human body, the system calculates the DTW distance between its motion trajectory and velocity curve and the corresponding data of each standard model m.

[0056] The DTW distance of the trajectory is calculated as follows: DTW_traj(k,m)=DTW(Tk_s,Tm,k_s) The DTW distance of the velocity curve is calculated as follows: DTW_vel(k,m)=DTW(Vk_n,Vm,k_n) The DTW function calculates the dynamic time warping distance between two time series. Specifically, for two sequences A={a_1,a_2,...,a_N} and B={b_1,b_2,...,b_N}, the DTW distance is defined as: DTW(A,B)=min(Σd(w_i)forallwarpingpathsW=w_1,w_2,...,w_L) Where d(w_i)=||a_i-b_j|| is the Euclidean distance and W is all possible regular paths.

[0057] 3. Next, the system calculates the comprehensive distance index for each standard model m: D(m)=Σ(w_traj,k*DTW_traj(k,m)+w_vel,k*DTW_vel(k,m))fork=1to8 Where w_traj,k and w_vel,k are the weights of the trajectory and velocity of each human key point. These weights can be adjusted according to the importance of different human key points in the pull-up action.

[0058] The system calculates the comprehensive distance index between the student data and each standard model in the model library, and then selects the model with the smallest distance as the most matching standard model: best_model=argmin(D(m))forallminmodel_library Step S140, determining the key stage division criteria of the student's pull-up action according to the motion trajectory, speed curve and the best matching standard model.

[0059] In one example, the steps of determining the key stage division criteria for students' pull-up movements based on the motion trajectory, speed curve and the best matching standard model include the following steps: extracting key parameters from the best matching standard model, the key parameters including the speed threshold, position threshold and time ratio corresponding to each stage of the pull-up movement; comparing the motion trajectory, speed curve with the speed threshold, position threshold and time ratio to obtain the key stage division criteria; the key stage division criteria include the start and end judgment conditions of the key stage, the time ratio range of the key stage in the entire pull-up movement and the dynamic adjustment rules.

[0060] The system first extracts key parameters from the selected best-matching standard model, including speed thresholds, position thresholds, and time proportions for each stage. For example, the standard model may define a speed threshold of 1.2 m / s at the beginning of the pull-up phase, a threshold of 20 cm between the wrist and the bar at the end of the pull-up phase, and a typical time proportion of 30%-45% of the total time for the pull-up phase. The key parameters will be used as initial reference values, and the system will adjust them based on the student's actual data.

[0061] Next, the system deeply analyzes the student's actual motion trajectory Tk and velocity curve Vk, and compares them with the parameters of the most matching standard model. This process conducts a detailed analysis of each stage of the pull-up action. In the preparation phase analysis, the system carefully examines the changes in wrist and hip velocity in Vk, looking for the time point when the velocity begins to increase significantly. The system pays special attention to the inflection point of the velocity curve, which usually marks the beginning of the student's intention to pull up. Assume that the standard model defines the velocity threshold for the end of the preparation phase as 1.2m / s, but after analyzing the student's Vk data, the system finds that the actual starting speed is 0.9m / s. This difference is recorded by the system as an important basis for adjusting the velocity threshold for the end of the preparation phase. The system not only considers this speed value, but also analyzes the slope of the speed increase to ensure that the true starting point of the action is captured. At the same time, the system also deeply analyzes the position data in Tk to check the student's posture changes in the preparation phase, including observing the relative positions of the joints of the body, especially the angle changes between the wrist, shoulder and hip.

[0062] In the analysis of the pull-up phase, the system will more comprehensively examine the velocity curve in Vk and the position change in Tk. First, the student's maximum pull-up velocity is extracted from Vk, assuming it is 1.8m / s, while the maximum pull-up velocity of the standard model is 2.2m / s. The system not only records this peak difference, but also analyzes the speed change pattern throughout the pull-up process, including changes in acceleration, the shape of the velocity curve (such as whether there are multiple peaks), and the time the speed is maintained. Next, the system conducts an in-depth analysis of the point when the speed begins to drop significantly. If it is found that the student's speed begins to drop rapidly when it reaches 70% of the maximum value (while the standard model may be 50%), the system will carefully consider the reasons behind this difference, which may reflect the student's ability to maintain a high point or a pattern of power decay.

[0063] In terms of position, the system analyzes in detail the position changes of the head or wrist when approaching the horizontal bar in Tk, including vertical and horizontal movements. Suppose the system finds that the student starts to slow down when he is 22cm away from the horizontal bar, while this distance is 20cm in the standard model. The system will carefully consider the impact of this 2cm difference on the overall movement quality, and analyze the student's body posture in this position, including the tilt angle of the torso, the degree of bending of the arms, etc.

[0064] In the analysis of the transition phase, the system focuses on the movement characteristics of the hips in Tk and Vk. The system mainly analyzes the change in hip velocity in Vk, paying special attention to the time point when the velocity changes from positive to negative. Suppose the system finds that the student's change point occurs at 58% of the total movement duration, while it may be 55% in the standard model. The system will carefully evaluate the impact of this 3% difference, considering whether the student has more time to stay at the high point, or whether there is unnecessary delay in the movement.

[0065] In the analysis of the lowering phase, the system will comprehensively examine the descent velocity curve in Vk and the position recovery in Tk. The system will analyze the changing pattern of the descent velocity in detail, including the initial falling velocity, the fluctuation of the velocity, and the deceleration control when approaching the end point. Suppose the system finds that the average lowering velocity of the student is 1.3m / s, while it may be 1.5m / s in the standard model. The system will carefully evaluate the impact of this speed difference on the quality and safety of the movement, and analyze the position recovery in Tk, including the order and time for various parts of the body to return to the initial position.

[0066] The system also determines the typical time ratio range of each stage in the entire movement. The preparation stage usually accounts for 10%-20% of the total time, the pull-up stage accounts for 30%-45%, the transition stage accounts for 10%-25%, and the lowering stage accounts for 25%-40%. The setting of these time ratio ranges not only refers to the standard model, but also takes into account the actual situation of ordinary students, providing a flexible standard for subsequent movement evaluation.

[0067] In order to adapt to the individual differences of different students, the system has also developed a set of dynamic adjustment rules and ranges. In terms of speed thresholds, if the student's maximum pull-up speed is detected to be lower than 60% of the standard model, the system will reduce all speed-related thresholds by 25%. Conversely, if the student's maximum pull-up speed exceeds 130% of the standard model, the system will increase the speed threshold by 15%. In terms of height thresholds, the system will adjust according to the student's height. If the student's height exceeds or falls below the standard model by 7%, the system will adjust all height-related thresholds by 5% accordingly. In terms of time proportion, if the student's overall completion time exceeds 30% of the standard model, the system allows the time proportion of each stage to fluctuate up and down by 7 percentage points.

[0068] Finally, the system determined the key phase division criteria for the pull-up movement, including the judgment conditions for the start and end of each phase (preparation, pull-up, transition, and lowering), the typical time ratio range of each phase in the entire movement, and the rules and range of dynamic adjustment. These division criteria and dynamic adjustment rules comprehensively consider the ideal characteristics of the standard model and the actual performance of students, and can adapt to students of different levels and body types, providing personalized movement analysis and evaluation.

[0069] Step S150, dividing the student's pull-up action into N key stages according to the key stage division standard, key point coordinate sequence and speed curve. The key stages include the preparation stage, the pull-up stage, the transition stage and the lowering stage; In one example, the steps of dividing the student's pull-up action into N key stages according to the key stage division standard, the key point coordinate sequence and the speed curve include the following steps: Starting from the first frame of the key point coordinate sequence, the first vertical speed of the student's wrist and hip is continuously monitored according to the speed curve, and the frame where the first vertical speed meets the corresponding start and end judgment conditions is marked as the end point of the preparation phase and the start point of the pull-up phase; After identifying the end point of the preparation phase and the start point of the pull-up phase, the head position and the second vertical speed of the wrist of the student are continuously monitored according to the key point coordinate sequence, and the frames in which the head position and the second vertical speed meet the corresponding start and end judgment conditions are marked as the end point of the pull-up phase and the start point of the transition phase; After identifying the end point of the pull-up phase and the start point of the transition phase, the third vertical velocity of the student's hip is continuously monitored according to the velocity curve, and the frame in which the third vertical velocity meets the corresponding start and end judgment conditions is marked as the end point of the transition phase and the start point of the lowering phase; After identifying the end point of the transition phase and the start point of the lowering phase, the fourth vertical speed of the student's wrist and hip is continuously monitored according to the speed curve, and the frame where the fourth vertical speed meets the corresponding start and end judgment conditions is marked as the end point of the lowering phase; According to the time ratio range, check and adjust the end point of the preparation phase, the start point of the pull-up phase, the end point of the pull-up phase, the start point of the transition phase, the end point of the transition phase and the start point of the lowering phase to divide the students' pull-up movements into the preparation phase, the pull-up phase, the transition phase and the lowering phase; In the process of key stage division, the start and end judgment conditions and the time ratio range are dynamically adjusted according to the dynamic adjustment rules, key point coordinate sequence and speed curve.

[0070] The system uses a 15-frame sliding window to analyze the entire action sequence frame by frame. In each window, the system uses the key point coordinate sequence and speed data to calculate key indicators. Specifically: First, the system identifies the preparation phase. Based on the start and end judgment conditions, the system starts detection from the first frame at the beginning of the action. Using the speed curve, the system continuously monitors the vertical speed of the wrist and hip. When it is detected that the vertical speed of these two parts exceeds 1% of the height per second at the same time, the system marks the frame as the end of the preparation phase and the beginning of the pull-up phase.

[0071] Next, the system begins to identify the pull-up phase. During this process, the system continuously monitors the vertical velocity of the wrist and the position of the head. The system first records the maximum vertical velocity during the pull-up process, and then uses the velocity curve to detect the speed drop. At the same time, the system uses the key point coordinate sequence to calculate the distance between the head and the crossbar. When the vertical velocity of the wrist drops below 50% of the peak velocity and the distance between the head and the crossbar is less than 20cm, the system marks the frame as the end of the pull-up phase and the beginning of the transition phase.

[0072] The identification of the transition phase depends mainly on the movement of the hip. The system continuously monitors the vertical velocity of the hip using the velocity curve. When the vertical velocity of the hip turns from positive to negative, the system marks the frame as the end of the transition phase and the beginning of the lowering phase.

[0073] Finally, the system identifies the lowering phase. During this process, the system monitors the vertical speed of the wrist and hip, as well as their positions. Using the speed curve, the system detects changes in speed; using the key point coordinate sequence, the system calculates the difference between the current position and the initial position. When the vertical speed of the wrist and hip is less than 1% of the height per second, and their positions are close to the initial position, the system marks the frame as the end of the lowering phase, which is also the end of the entire action.

[0074] Throughout the segmentation process, the system applies defined dynamic adjustment rules. For example, if during the pull-up phase it is detected that the student's maximum pull-up speed is less than 60% of the standard model, the system will reduce all speed-related thresholds by 25%. This means that the speed threshold for determining the end of the pull-up phase may be reduced from 50% of the peak speed to 37.5%. Similarly, if the student's height deviates significantly from the standard model, the system will adjust all height-related thresholds accordingly. For example, for a student whose height exceeds the standard model by 7%, the threshold for determining the distance to the bar may be increased from 20cm to 21cm.

[0075] After the initial division is completed, the system will check and adjust according to the specified typical time proportion range of each stage. The system calculates the time proportion of each stage and compares it with the expected range. If the time proportion of a stage deviates significantly from the expected range, the system will re-check the boundaries of the stage. For example, if the pull-up phase accounts for more than 45% of the total time (higher than the expected 30%-45%), the system will re-analyze the end point of the stage and may end the pull-up phase earlier, extending the start time of the transition phase accordingly.

[0076] When making these adjustments, the system repeatedly uses the key point coordinate sequence and speed curve. For example, when adjusting the end point of the pull-up phase, the system reanalyzes the vertical speed change (speed curve) and body posture change (motion trajectory) during this period to find a more appropriate dividing point. If the student's overall completion time exceeds 30% of the standard model, the system will allow a larger floating space for the time proportion of each stage, which can fluctuate up or down by 7 percentage points.

[0077] Through this comprehensive and detailed process, the system successfully divides the continuous action sequence into four key stages: preparation, pull-up, transition, and release. Finally, the system generates a concise data structure as output. This data structure contains the start and end frames of each stage, as well as the total number of frames of the entire action sequence. For example: {"Total frames":180,"Preparation phase":[1,30],"Pull-up phase":[31,90],"Transition phase":[91,105],"Lower phase":[106,180]} Step S160, selecting candidate key frames corresponding to the key stages according to the key point coordinate sequence, motion trajectory and speed curve.

[0078] In one example, the step of selecting candidate key frames corresponding to the key stage according to the key point coordinate sequence, the motion trajectory and the speed curve includes the following steps: In the preparation stage, the average position of the student's hip and shoulder in the preparation stage is calculated according to the motion trajectory; the offset distance between the student's hip position and shoulder position of each frame in the preparation stage and the average position is calculated according to the motion trajectory, and the frame corresponding to the minimum offset distance is used as the candidate key frame corresponding to the preparation stage; In the pull-up stage, the average vertical speed in the pull-up stage is calculated according to the speed curve, and the frame closest to the average vertical speed is marked from the speed curve as the first feature frame; the change speed of the student's elbow angle is analyzed according to the motion trajectory, and the frame corresponding to the maximum change speed is marked as the second feature frame; the vertical relative position of the student's wrist and shoulder is analyzed according to the motion trajectory, and the frame corresponding to the vertical relative position of zero is taken as the third feature frame; the candidate key frame corresponding to the pull-up stage is selected from the first feature frame, the second feature frame and the third feature frame; In the transition stage, the frame with the highest head position of the student is marked as the fourth feature frame according to the motion trajectory; the angle of the student's elbow is calculated according to the motion trajectory, and the frame corresponding to the minimum elbow angle is marked as the fifth feature frame; the frame where the student's hip begins to move forward is marked as the sixth feature frame according to the motion trajectory; and the candidate key frame corresponding to the transition stage is selected from the fourth feature frame, the fifth feature frame, and the sixth feature frame; In the lowering stage, the average vertical speed of the lowering stage is calculated according to the speed curve, and the frame closest to the average vertical speed on the speed curve is marked as the seventh feature frame; the changing speed of the student's elbow angle is analyzed according to the motion trajectory data, and the frame corresponding to the maximum changing speed is marked as the eighth feature frame; the change in the angle between the student's hip and shoulder is calculated according to the motion trajectory, and the frame where the student starts to lean forward is marked according to the change in the angle and used as the ninth feature; the candidate key frames corresponding to the lowering stage are selected from the seventh feature frame, the eighth feature frame and the ninth feature frame.

[0079] Specifically, in the preparation phase, the main goal of the system is to find the frame that best represents the stable preparation posture. The system first uses the motion trajectory to calculate the average position of the hip and shoulder throughout the preparation phase. Assuming that there are 30 frames in the preparation phase, the system will extract the hip and shoulder coordinates of these 30 frames from the motion trajectory data, accumulate their x-coordinates and y-coordinates, and then divide them by 30 to get the average position. This average position represents an ideal static state of the body throughout the preparation phase.

[0080] Next, the system calculates the distance between the hip and shoulder positions and this average position for each frame in the preparation phase. This calculation process uses the motion trajectory data directly. Specifically, the system calculates the difference in the hip x coordinate, the hip y coordinate, the shoulder x coordinate, and the shoulder y coordinate for each frame. Then, the squares of these four differences are added together and the square root is taken to obtain a comprehensive distance value. This distance value reflects the degree of deviation of the current frame from the ideal static state.

[0081] For example, if the hip position for a frame is 2 cm to the right and 1 cm up from the average position, and the shoulder position is 1 cm to the left and 2 cm down from the average position, then the total offset distance for that frame is the square root of the sum of the squares of these four offset values, which is approximately 3.16 cm. This offset distance is calculated for each frame in the preparation phase.

[0082] The system selects the frame with the smallest offset distance as the candidate keyframe. If there are multiple frames with very close offset distances, such as a difference of less than 0.5 cm, the system selects the frame that is closest in time to the midpoint of the preparation phase. For example, if the preparation phase is from frame 1 to frame 30, the midpoint is frame 15. If the offset distances of frames 14 and 16 are both small and not much different, the system selects frame 15 as the candidate keyframe.

[0083] During the pull-up phase, the system needs to capture multiple key features because this phase contains the most complex and critical part of the pull-up action. First, the system uses the speed curve data to calculate the average vertical speed of the entire pull-up phase. Assuming that there are 60 frames in the pull-up phase, the system will extract the vertical speed values ​​of these 60 frames from the speed curve, accumulate them, and divide them by 60 to get the average speed. This average speed represents the typical speed of the entire pull-up process.

[0084] The system then finds the point on the speed curve that is closest to this average and marks the corresponding frame as a feature frame representing the average speed. For example, if the average speed is 2 meters per second, and a frame has a speed of 1.98 meters per second, while other frames have a greater speed difference, then this frame may be selected as a feature frame representing the average speed. This frame usually appears in the middle of the pull-up phase, reflecting the average intensity of the action.

[0085] Next, the system uses the motion trajectory data to analyze the rate of change of the elbow angle. The system calculates the elbow angle of each frame based on the coordinates of the shoulder, elbow, and wrist in the motion trajectory, and then calculates the angle change between each two adjacent frames to find the frame with the largest change. This frame usually represents the moment when the force is most concentrated and the action is most intense. For example, if in a certain frame, the elbow angle decreases by 15 degrees compared to the previous frame, and the changes in other frames are less than 10 degrees, then this frame may be identified as the frame with the fastest angle change. This frame usually appears in the early or middle stages of the pull-up, reflecting the moment of major force application.

[0086] The system also uses the motion trajectory data to analyze the relative position of the wrist and shoulder to find the frame where the wrist is exactly flush with the shoulder. This is achieved by comparing the y coordinates of the wrist and shoulder in each frame. If the y coordinate of the wrist and the y coordinate of the shoulder in a certain frame do not differ by more than 2 cm, then this frame may be considered a frame where the wrist is flush with the shoulder. This frame represents a critical transition point in the pull-up action, usually marking that the upper body is about to cross the horizontal bar.

[0087] If the three feature frames (average speed frame, fastest angle change frame, wrist and shoulder level frame) are close in time, for example, the difference between their frame numbers does not exceed 20% of the total number of frames in the pull-up phase, then the system will select the frame in the middle as the candidate key frame. If the three frames are more dispersed, the system will select all three frames as candidate key frames to ensure that the different features of the pull-up phase can be captured.

[0088] During the transition phase, although this phase is usually short, it contains the key turning points of the pull-up movement. The system first uses the motion trajectory data to find the frame with the highest head position. This is done by comparing the y coordinate of the head in each frame. This frame represents the moment when the pull-up movement reaches its highest point, the peak of the movement.

[0089] The system then uses the motion trajectory data to calculate the elbow angle for each frame and finds the frame with the smallest angle, which represents the moment when the arm is most flexed. This frame reflects the extreme state of the movement, which usually corresponds to the maximum force output.

[0090] The system also analyzes the trajectory of the hip movement to find the frame where the hip begins to move forward. This is achieved by detecting the horizontal velocity of the hip. The system calculates the horizontal velocity of the hip for each frame, and when the horizontal velocity exceeds a small positive value (such as 10 cm per second) for several consecutive frames (such as 3 frames), it is considered that the hip begins to move forward. This frame marks the beginning of the body's preparation for lowering, and is the turning point from rising to descending.

[0091] If the three feature frames are close in time, for example, the difference between their frame numbers does not exceed 30% of the total number of frames in the transition phase, the system will select their middle frame as the candidate key frame. Otherwise, all three frames will be selected as candidate key frames to ensure that all key features of the transition phase are captured.

[0092] During the lowering phase, the system also relies on velocity curve and motion trajectory data for analysis. First, the system uses the velocity curve data to calculate the average vertical velocity for the entire lowering phase. This velocity is usually negative because the body is falling. Then, the system finds the point on the velocity curve with the velocity closest to this average value and marks the corresponding frame as a characteristic frame representing a typical lowering velocity. This frame reflects the athlete's overall performance in controlling the fall.

[0093] Next, the system uses the motion trajectory data to analyze the change in elbow angle and find the frame with the fastest angle change. This usually occurs at the beginning of the lowering, reflecting the rapid transition from the flexed arm state to the extended arm state. The system calculates the change in elbow angle between each two adjacent frames and finds the frame with the largest rate of change. This frame is very important for understanding the force control during the lowering process.

[0094] The system also uses the motion trajectory data to calculate the change in the angle between the hip and shoulder to find the frame where the body begins to lean forward. This can be achieved by comparing the angle of each frame with the average angle of the preparation phase. The system first calculates the hip-shoulder angle of all frames in the preparation phase and takes the average value as a reference. Then in the lowering phase, when the angle of a certain frame differs from this reference value by more than a certain degree (such as 10 degrees), it is considered that the body begins to lean forward. This forward leaning action is an important feature of the lowering process, reflecting the skill of maintaining balance and controlling the falling speed.

[0095] If the three feature frames (average speed frame, fastest angle change frame, and body forward leaning frame) are close in time, for example, the difference in their frame numbers does not exceed 25% of the total number of frames in the release phase, then the system will select their middle frame as a candidate key frame. Otherwise, all three frames will be selected as candidate key frames to ensure that all key features of the release phase are captured.

[0096] Step S170, optimizing and screening the candidate key frames to obtain final key frames; the key stage corresponds to at least one final key frame, and the number of final key frames is within a preset range.

[0097] In one example, the step of optimizing and screening candidate key frames to obtain a final key frame includes the following steps: sorting the candidate key frames in chronological order to obtain a preliminary sorting; based on the preliminary sorting, identifying the first candidate key frame in the preparation stage as the final key frame; traversing the remaining candidate key frames in the preliminary sorting, and calculating the time interval between the current candidate key frame and the final key frame most recently selected during the traversal process; based on the preliminary judgment condition that the time interval is greater than a preset time interval threshold, and in combination with judging the key features contained in the candidate key frames, screening the remaining final key frames.

[0098] The system first performs a preliminary sorting of all candidate key frames based on frame numbers to ensure that the system can process these frames in chronological order. After the sorting is completed, the system creates a new list to store the final selected key frames.

[0099] Next, the system will set a preset time interval threshold. The setting of this threshold needs to take into account the total duration of the pull-up action and the expected number of key frames. For example, if the entire action lasts for 3 seconds (assuming the video frame rate is 30fps, a total of 90 frames), and it is expected that there will be 8-10 key frames in the end, then the minimum time interval can be set to about 0.3 seconds (9 frames). This setting is not fixed, and the system will dynamically adjust it according to the specific situation to adapt to the movement characteristics and speed of different students.

[0100] The optimization screening process starts with selecting the first candidate keyframe. Usually, the system selects the first candidate frame in the preparation stage as the initial keyframe and adds it to the final keyframe list. This frame represents the standard preparation posture before the action begins and is crucial to understanding the entire action. Then, the system starts to traverse the remaining candidate keyframes, analyzing and judging each frame in detail.

[0101] During the traversal process, the system first calculates the time interval between the current candidate keyframe and the last final keyframe selected during the traversal process. If this interval is less than the preset time interval threshold, the system will usually temporarily skip this frame and continue to check the next candidate frame. The purpose of this is to avoid selecting too dense keyframes, which may cause redundant information in the final action decomposition graph. However, the system does not apply this rule completely mechanically. If the current frame represents a particularly important action feature, the system may choose to keep it even if it is less spaced from the previous keyframe.

[0102] If the time interval meets the requirements, the system will further evaluate the key features of the current frame. This includes checking whether the frame represents the important features of a specific stage, such as the maximum speed point in the pull-up stage, or the highest point in the transition stage. The system will refer to the selection reasons and feature data of each candidate key frame and comprehensively consider its importance. At the same time, the system will also compare the current frame with the selected key frames to check whether there is information redundancy. For example, if the current frame and the previous key frame both represent similar action features (such as force frames in the pull-up stage), the system will carefully compare the feature data of the two frames and select the frame that better represents the feature.

[0103] During this process, the system always keeps an eye on the overall action structure. It ensures that each key stage (preparation, pull-up, transition, and release) has at least one key frame. If a stage is found to have no key frame and the current frame belongs to that stage, the system will give priority to this frame, even if it may not be the most representative frame in that stage. This approach ensures that the final set of key frames can fully cover the entire action process.

[0104] Based on the above comprehensive judgment, if the current frame passes all checks, the system will add it to the final keyframe list. If it does not pass, the system will continue to check the next candidate frame. This process will continue until all candidate frames have been evaluated.

[0105] After completing the traversal of all candidate frames, the system will perform an overall check and adjustment. First, the system will reconfirm whether each key stage has at least one final key frame. If it is found that a certain stage does not have a final key frame, the system will go back to the candidate key frames of that stage and select the most representative one to add to the final list. This ensures that the integrity of the action will not be affected by the screening process.

[0106] Next, the system checks whether the total number of final keyframes is within the expected range. If the total number exceeds a preset upper limit (for example, 10 frames), the system will perform further screening. This may involve deleting some relatively unimportant keyframes while ensuring that the overall performance of the action is not affected. In this process, the system will give priority to retaining frames that represent turning points of key actions, such as the frame where the pull-up starts, the frame where the highest point is reached, the frame where the lowering begins, and so on. On the contrary, if the number of keyframes is insufficient, the system will re-examine the candidate frames that were previously skipped, and may relax some screening criteria to increase the number of keyframes. This dynamic adjustment ensures that the final set of keyframes can fully reflect the characteristics of the action without affecting the efficiency of subsequent processing due to excessive number.

[0107] In one example, the pull-up process action decomposition and key frame extraction method of the present application further includes the following steps: Detecting time intervals between adjacent final key frames in a final key frame sequence; When it is detected that the time interval between two adjacent final key frames exceeds the preset maximum time interval threshold: obtain the speed values ​​of all skipped candidate key frames between the two adjacent final key frames whose time interval exceeds the preset maximum time interval threshold; for each candidate key frame, calculate the sum of the absolute value of the speed difference between the current candidate key frame and the previous candidate key frame and the absolute value of the speed difference between the current candidate key frame and the next candidate key frame as the speed change significance index of the current candidate key frame; select the candidate key frame with the largest speed change significance index as an additional key frame; add the additional key frame to the final key frame sequence.

[0108] The system also pays special attention to the time interval between adjacent keyframes. If the interval between two adjacent keyframes is found to be too large (for example, more than 25% of the entire action duration), the system will try to add an additional keyframe in this interval. This additional frame is usually selected from the previously skipped candidate frames to better describe the continuity of the action and avoid the "jump" feeling of the action description. When selecting additional keyframes, the system mainly focuses on the speed change of the action. Specifically, the system first obtains the speed values ​​of all skipped candidate keyframes within a large time interval. For each candidate frame, the system calculates the speed difference v1 with the previous frame and the speed difference v2 with the next frame. Then, the system calculates the sum of the absolute values ​​of the two differences, that is, |v1| + |v2|, as the speed change significance index of the frame. This calculation method takes into account the speed change of the candidate frame and its previous and subsequent frames at the same time, and can effectively capture the mutation point of speed. The system traverses all candidate frames and records the significance index value of each frame. Finally, the system compares the significance indexes of all candidate frames and selects the frame with the largest index value as the frame with the most significant speed change. If there are multiple maximum values ​​of the same indicator, the system will select the candidate frame that is closest in time to the midpoint of the two original keyframes. After selection, the system will insert this frame as an additional keyframe into the final keyframe sequence.

[0109] After completing all these optimizations and adjustments, the system will sort the final selected key frame sequence to ensure that they are strictly arranged in chronological order. At the same time, the system will add additional information to each key frame, including the action stage it belongs to, its importance in that stage, and the specific action characteristics it represents. This additional information not only helps with subsequent action analysis and teaching guidance, but also provides important contextual information for generating a coherent action decomposition diagram.

[0110] For example, the final keyframe list may include: the stable posture frame in the preparation phase, the initial force frame at the beginning of the pull-up phase, the maximum speed frame in the middle of the pull-up phase, the wrist approaching the horizontal bar frame at the end of the pull-up phase, the head highest point frame in the transition phase, the hips start to move forward in the transition phase, the control of the lowering phase, the typical lowering speed frame in the middle of the lowering phase, and the frame at the end of the lowering phase when it is about to return to the starting position. Such a set of keyframes not only covers the key stages of the pull-up movement, but also captures the important movement details in each stage.

[0111] Through this meticulous optimization and screening process, the system ultimately obtains a set of key frames that fully and accurately reflects the entire pull-up action while maintaining a reasonable time distribution between key frames.

[0112] The method for decomposing and extracting key frames of the pull-up process in the present application also includes the steps of: visualizing the final key frames, generating a static decomposition diagram of the student's pull-up movements, comparing the static decomposition diagram with the most matching standard model, and providing improvement suggestions based on the comparison results.

[0113] In one example, the final key frame is visualized to generate a static decomposition diagram of the student's pull-up action, the static decomposition diagram is compared with the most matching standard model, and improvement suggestions are given based on the comparison results, including the steps of: preprocessing the final key frame, adding corresponding text information to the preprocessed final key frame, and generating a static decomposition diagram; generating a standard static decomposition diagram based on the most matching standard model; displaying the static decomposition diagram and the standard static decomposition diagram side by side, and highlighting the differences; comparing the static decomposition diagram and the standard static decomposition diagram, giving improvement suggestions, and displaying the improvement suggestions in the static decomposition diagram.

[0114] First, the system reads the final keyframe set. This set usually contains 8-10 final keyframes, each of which represents an important moment or posture in the pull-up movement. The system processes these frames in chronological order to ensure that the final decomposition diagram can clearly show the progress of the movement.

[0115] For each key frame, the system performs a series of image processing and enhancement operations. This includes improving the clarity of the original image, adjusting the contrast, and removing noise. These processes are designed to ensure that each key frame in the final decomposition diagram is clearly identifiable, making it easier for students and coaches to observe the details. At the same time, the system will mark key body parts such as wrists, elbows, shoulders, hips, and heads on each key frame. These annotations usually use eye-catching colors and appropriate graphic symbols, such as dots or small arrows, to highlight the location and movement trajectory of these key points.

[0116] Next, the system adds concise text descriptions to each keyframe. These descriptions include the action phase to which the frame belongs (such as "preparation phase", "early pull-up phase", etc.), as well as the specific action characteristics represented by the frame (such as "maximum speed point", "transition start", etc.). The text descriptions use clear, easy-to-read fonts and are placed in a position that does not obstruct the key action area. In addition, the system will add arrows or lines between adjacent keyframes to indicate the flow direction of the action and enhance the coherence of the decomposition diagram.

[0117] In order to better demonstrate the continuity of the action, the system may insert some semi-transparent transition frames between the main key frames. These transition frames will not contain detailed annotations, but can help viewers better understand the continuous change process of the action. At the same time, the system will add a timeline at the top or bottom of the entire decomposition diagram, indicating the relative time position of each key frame in the entire action, to help viewers understand the rhythm and time allocation of the action.

[0118] After the static decomposition diagram of the student's action is completed, the system will read the selected standard model that best matches it. The system will use the same visualization method to generate a corresponding standard static decomposition diagram for the best matching standard model. This standard static decomposition diagram will be displayed side by side with the student's static decomposition diagram for direct comparison. In order to highlight the difference between the two, the system will use different color schemes, such as blue for the student's static decomposition diagram and green for the standard static decomposition diagram.

[0119] Next, the system will conduct a detailed comparative analysis. This analysis process will take into account multiple aspects: first, posture comparison. The system will compare the body postures of the students and the standard model at each critical moment, including the relative positions and angles of key points (such as wrists, elbows, shoulders, and hips). Secondly, time allocation comparison. The system will analyze whether the time allocation of the students at each stage is reasonable and whether it is consistent with the standard model. Thirdly, motion trajectory comparison. The system will compare the key point motion trajectories of the students and the standard model to identify the parts with large deviations.

[0120] Based on these comparative analyses, the system will generate a series of improvement suggestions. These suggestions will be directly marked on the student's decomposition diagram, and presented in a way that is eye-catching but does not affect the overall appearance of the picture. For example, if it is found that the angle of the student's elbow in the pull-up phase is significantly different from that of the standard model, the system will add a suggestion mark to the corresponding keyframe, accompanied by a concise text description, such as "Note: When pulling up, you should tighten your elbow more to reduce the angle between the upper arm and the torso." If the student's time allocation at a certain stage is obviously unreasonable, the system will also mark it on the timeline to prompt the student to adjust the rhythm.

[0121] An example to illustrate the whole process: Xiao Wang, a high school student, is learning to do pull-ups, but has been unable to complete the standard movements. The physical education teacher decides to use the system of the present invention to help Xiao Wang improve his technique.

[0122] Step 1: When Xiao Wang stands in front of the pull-up machine, the system automatically starts recording with two high-definition cameras. One camera shoots from the front and the other from the side, ensuring that all angles of Xiao Wang's movements are captured. After recording, the system automatically pre-processes the video to remove background noise and highlight Xiao Wang's body outline.

[0123] Step 2: The system uses an improved human pose estimation algorithm to accurately extract eight key points of Xiao Wang's body from the preprocessed video, including left and right wrists, shoulders, elbows, hip center, and top of the head. Even during Xiao Wang's fast pull-up phase, the system can accurately track the positions of these key points.

[0124] Step 3: Based on the extracted key point data, the system calculates the motion trajectory and speed curve of each key part of Xiao Wang during the entire pull-up process. Through filtering, the system eliminates the data fluctuations caused by Xiao Wang's slight shaking of his body and obtains a smooth motion curve.

[0125] Step 4: The system compares Xiao Wang’s motion data with the pre-established standard motion model library. Considering Xiao Wang’s height (170cm) and weight (60kg), the system selects a standard intermediate-level model of similar body shape as a reference.

[0126] Step 5: Based on the selected standard model and Xiao Wang's actual data, the system developed a standard for the division of action phases suitable for Xiao Wang. Since Xiao Wang is a beginner, the system appropriately relaxed the speed requirement for the pull-up phase, reducing the standard from 1.5m / s to 1.2m / s.

[0127] Step 6: Using the criteria established in step 5, the system automatically divides Xiao Wang's entire sequence of movements into four phases: preparation, pull-up, transition, and lowering. The system accurately identifies the moment when Xiao Wang begins to pull up with force and marks it as the starting point of the pull-up phase.

[0128] Step 7: In each divided stage, the system selects the most representative frames as candidate key frames. For example, in the pull-up stage, the system selects the frame where Xiao Wang's speed reaches the average value, the frame where the elbow angle changes the fastest, and the frame where the wrist is aligned with the shoulder.

[0129] Step 8: The system optimizes and screens the candidate keyframes to ensure that the time intervals between them are appropriate and that there is at least one keyframe in each stage. Finally, the system selects 9 keyframes for Xiao Wang's action, which fully reflects his pull-up process.

[0130] Step 9: The system processes the filtered keyframes into a static decomposition diagram and compares it with the standard model. Through analysis, the system found that Xiao Wang's elbow angle was too large during the pull-up phase, resulting in reduced force transfer efficiency. Based on this finding, the system generated targeted improvement suggestions, such as "tighten the elbow more when pulling up to reduce the angle between the upper arm and the torso." Based on these suggestions, the physical education teacher developed a targeted training plan for Xiao Wang to help him improve his pull-up technique.

[0131] The present application comprises the following steps: collecting video data of students doing pull-ups and preprocessing it, extracting key points of the human body from the preprocessed video data, and obtaining a sequence of key point coordinates; calculating the motion trajectory and speed curve of the key points of the human body based on the sequence of key point coordinates; comparing the motion trajectory and speed curve with a preset standard action model library, and selecting the most matching standard model from the preset standard action model library; determining the key stage division criteria of the students' pull-up movements based on the motion trajectory, speed curve and the most matching standard model; dividing the students' pull-up movements into N key stages based on the key stage division criteria, the sequence of key point coordinates and the speed curve; selecting candidate key frames corresponding to the key stages based on the sequence of key point coordinates, the motion trajectory and the speed curve; optimizing and screening the candidate key frames to obtain the final key frames; the key stages correspond to at least one final key frame, and the number of final key frames is within a preset range; visualizing the final key frames to generate a static decomposition diagram of the students' pull-up movements, comparing the static decomposition diagram with the most matching standard model, and giving improvement suggestions based on the comparison results. This application can automatically capture the complete action process and accurately divide the action stages. It can extract the most representative key frames from each stage, generate detailed static decomposition diagrams of action essentials, and consider the individual characteristics of students to provide personalized improvement suggestions.

[0132] 1. This application realizes personalized movement evaluation through intelligent standard model matching and dynamic adjustment mechanism. The system has established a standard model library containing various body shapes and skill levels. The intelligent matching algorithm comprehensively considers the static parameters and dynamic factors of the students, and selects the most matching standard model from the model library. This method overcomes the limitations of the traditional "one-size-fits-all" evaluation method. The dynamic adjustment mechanism can adjust the evaluation criteria in real time according to the actual performance of the students, such as appropriately lowering the pull-up speed requirements for beginners and raising the movement stability standards for high-level students. This ensures that the evaluation criteria always match the actual abilities of the students and improves the accuracy and adaptability of movement evaluation.

[0133] 2. This application adopts automatic action segmentation and intelligent key frame extraction technology to provide a scientific and systematic action decomposition method. The automatic segmentation algorithm uses signal processing and pattern recognition technology to accurately divide the entire pull-up action into key stages such as preparation, pull-up, transition and lowering. Key frame extraction technology makes intelligent selections based on the importance of action features, rather than simply selecting at fixed time intervals. For example, in the pull-up stage, the system will give priority to frames at key moments such as when the speed reaches its peak, the elbow angle changes most dramatically, and the wrist is aligned with the shoulder. This method ensures that the selected key frames can reflect the key features of the action to the greatest extent, while improving the efficiency and consistency of the analysis.

[0134] 3. This application introduces keyframe optimization technology to further improve the accuracy and practicality of motion analysis. The optimization process balances the time intervals between keyframes through an intelligent algorithm to ensure that each action stage has sufficient keyframe descriptions while avoiding information redundancy. The system also takes into account the importance weights of different action stages and adjusts the keyframe distribution of each stage according to the level and characteristics of the learners. For example, for beginners, more keyframes may be selected in the pull-up stage, while for high-level athletes, the number of keyframes is increased in the transition stage. This intelligent optimization ensures that the final selected set of keyframes can comprehensively and accurately reflect the characteristics of the entire pull-up action, providing reliable data support for subsequent action evaluation and guidance.

[0135] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0136] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 2As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as video data, motion trajectories and speed curves. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for decomposing the pull-up process action and extracting key frames is implemented.

[0137] Those skilled in the art will understand that Figure 2 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0138] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented: Collect and preprocess the video data of students doing pull-ups, extract the key points of the human body from the preprocessed video data, and obtain the key point coordinate sequence; According to the key point coordinate sequence, calculate the motion trajectory and speed curve of the key points of the human body; Compare the motion trajectory and speed curve with the preset standard motion model library, and select the most matching standard model from the preset standard motion model library; According to the motion trajectory, speed curve and the most matching standard model, determine the key stage division criteria of students' pull-up movements; According to the key stage division criteria, key point coordinate sequence and speed curve, the students' pull-up movements are divided into N key stages; According to the key point coordinate sequence, motion trajectory and speed curve, select the candidate key frames corresponding to the key stage; Optimize and screen the candidate key frames to obtain the final key frames; the key stage corresponds to at least one final key frame, and the number of final key frames is within a preset range; The final keyframes are visualized to generate a static decomposition diagram of the student's pull-up action. The static decomposition diagram is compared with the most matching standard model, and improvement suggestions are given based on the comparison results.

[0139] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: Collect and preprocess the video data of students doing pull-ups, extract the key points of the human body from the preprocessed video data, and obtain the key point coordinate sequence; According to the key point coordinate sequence, calculate the motion trajectory and speed curve of the key points of the human body; Compare the motion trajectory and speed curve with the preset standard motion model library, and select the most matching standard model from the preset standard motion model library; According to the motion trajectory, speed curve and the most matching standard model, determine the key stage division criteria of students' pull-up movements; According to the key stage division criteria, key point coordinate sequence and speed curve, the students' pull-up movements are divided into N key stages; According to the key point coordinate sequence, motion trajectory and speed curve, select the candidate key frames corresponding to the key stage; Optimize and screen the candidate key frames to obtain the final key frames; the key stage corresponds to at least one final key frame, and the number of final key frames is within a preset range; The final keyframes are visualized to generate a static decomposition diagram of the student's pull-up action. The static decomposition diagram is compared with the most matching standard model, and improvement suggestions are given based on the comparison results.

[0140] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0141] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.

Claims

1. A method for decomposing and extracting key frames during a pull-up process, characterized in that: The following steps are involved: Collecting and preprocessing video data of students doing pull-ups, extracting key points of the human body from the preprocessed video data, and obtaining a key point coordinate sequence; Calculating the motion trajectory and speed curve of the key points of the human body according to the key point coordinate sequence; Comparing the motion trajectory and the speed curve with a preset standard motion model library, and selecting the most matching standard model from the preset standard motion model library; Determining the key stage division criteria of the student's pull-up action according to the motion trajectory, the speed curve and the best matching standard model; Dividing the student's pull-up action into N key stages according to the key stage division standard, the key point coordinate sequence and the speed curve; Selecting candidate key frames corresponding to the key stage according to the key point coordinate sequence, the motion trajectory and the speed curve; Optimizing and screening the candidate key frames to obtain the final key frames; The key stage corresponds to at least one final key frame, and the number of the final key frames is within a preset range.

2. The pull-up process action decomposition and key frame extraction method according to claim 1 is characterized in that: The step of calculating the motion trajectory and speed curve of the key points of the human body according to the key point coordinate sequence comprises the steps of: Using a low-pass filter to slide in the time dimension, local polynomial fitting is performed on the coordinate values ​​in the key point coordinate sequence to obtain the motion trajectory; Processing the motion trajectory using a differential method to obtain a preliminary velocity curve; The speed curve is obtained by sliding a low-pass filter in the time dimension and performing local polynomial fitting on the speed values ​​in the preliminary speed curve.

3. The pull-up process action decomposition and key frame extraction method according to claim 1 is characterized in that: The preset standard action model library includes N standard models of pull-up actions; the standard models are divided into high-level standard models, medium-level standard models and low-level standard models; The step of comparing the motion trajectory and the speed curve with a preset standard motion model library and selecting the most matching standard model from the preset standard motion model library comprises the steps of: Preprocessing the motion trajectory, the velocity curve and the standard model; the preprocessing includes time normalization and space normalization; Using a dynamic time warping algorithm, calculate a first dynamic time warping distance between the preprocessed motion trajectory and the preprocessed standard motion trajectory in the standard model, and calculate a second dynamic time warping distance between the preprocessed velocity curve and the preprocessed standard velocity curve in the standard model; Obtaining a comprehensive distance index according to the first dynamic time warping distance, the second dynamic time warping distance, and the dynamic time warping weights corresponding to the key points of the human body; The standard model corresponding to the smallest comprehensive distance index is used as the best matching standard model.

4. The pull-up process action decomposition and key frame extraction method according to claim 1, characterized in that: The step of determining the key stage division standard of the student's pull-up action according to the motion trajectory, the speed curve and the best matching standard model includes the steps of: Extracting key parameters from the best matching standard model, wherein the key parameters include a speed threshold, a position threshold, and a time ratio corresponding to each stage of the pull-up action; The motion trajectory, the speed curve, the speed threshold, the position threshold and the time ratio are compared to obtain the key stage division standard; the key stage division standard includes the start and end judgment conditions of the key stage, the time ratio range of the key stage in the entire pull-up action and the dynamic adjustment rules.

5. The pull-up process action decomposition and key frame extraction method according to claim 4 is characterized in that: The key stages include the preparation stage, the pull-up stage, the transition stage and the lowering stage; The step of dividing the student's pull-up action into N key stages according to the key stage division standard, the key point coordinate sequence and the speed curve includes the steps of: Starting from the first frame of the key point coordinate sequence, the first vertical speed of the student's wrist and hip is continuously monitored according to the speed curve, and the frame where the first vertical speed meets the corresponding start and end judgment conditions is marked as the end point of the preparation phase and the start point of the pull-up phase; After identifying the end point of the preparation phase and the start point of the pull-up phase, continuously monitoring the student's head position and the second vertical speed of the wrist according to the key point coordinate sequence, marking the frame where the head position and the second vertical speed meet the corresponding start and end judgment conditions as the end point of the pull-up phase and the start point of the transition phase; After identifying the end point of the pull-up phase and the start point of the transition phase, continuously monitoring the third vertical speed of the student's hip according to the speed curve, marking the frame where the third vertical speed meets the corresponding start and end judgment conditions as the end point of the transition phase and the start point of the lowering phase; After identifying the end point of the transition phase and the start point of the lowering phase, continuously monitoring the fourth vertical speed of the student's wrist and hip according to the speed curve, and marking the frame where the fourth vertical speed meets the corresponding start and end judgment conditions as the end point of the lowering phase; According to the time ratio range, the end point of the preparation phase, the start point of the pull-up phase, the end point of the pull-up phase, the start point of the transition phase, the end point of the transition phase and the start point of the lowering phase are checked and adjusted to divide the student's pull-up action into the preparation phase, the pull-up phase, the transition phase and the lowering phase; In the key stage division process, the start and end judgment conditions and the time ratio range are dynamically adjusted according to the dynamic adjustment rule, the key point coordinate sequence and the speed curve.

6. The pull-up process action decomposition and key frame extraction method according to claim 5, characterized in that: The step of selecting candidate key frames corresponding to the key stage according to the key point coordinate sequence, the motion trajectory and the speed curve comprises the steps of: In the preparation stage, according to the motion trajectory, the average position of the student's hip and shoulder in the preparation stage is calculated; according to the motion trajectory, the offset distance between the student's hip position and shoulder position of each frame in the preparation stage and the average position is calculated, and the frame corresponding to the smallest offset distance is used as the candidate key frame corresponding to the preparation stage; In the pull-up stage, an average vertical speed in the pull-up stage is calculated according to the speed curve, and a frame closest to the average vertical speed on the speed curve is marked as a first feature frame; Analyze the change speed of the student's elbow angle according to the motion trajectory, and mark the frame corresponding to the maximum change speed as the second feature frame; Analyze the vertical relative position of the student's wrist and shoulder according to the motion trajectory, and use the frame corresponding to the vertical relative position of zero as the third feature frame; select the candidate key frame corresponding to the pull-up stage from the first feature frame, the second feature frame and the third feature frame; In the transition phase, marking the frame with the highest position of the student's head according to the motion trajectory as the fourth feature frame; Calculating the student's elbow angle according to the motion trajectory, and marking the frame corresponding to the smallest elbow angle as the fifth feature frame; Marking the frame where the student's hip starts to move forward as the sixth feature frame according to the motion trajectory; Selecting a candidate key frame corresponding to the transition stage from the fourth feature frame, the fifth feature frame, and the sixth feature frame; In the lowering stage, the average vertical speed of the lowering stage is calculated according to the speed curve, and the frame closest to the average vertical speed on the speed curve is marked as the seventh feature frame; Analyze the changing speed of the student's elbow angle according to the motion trajectory data, and mark the frame corresponding to the maximum changing speed as the eighth feature frame; calculate the angle change between the student's hip and shoulder according to the motion trajectory, and mark the frame where the student starts to lean forward according to the angle change as the ninth feature; select candidate key frames corresponding to the lowering stage from the seventh feature frame, the eighth feature frame and the ninth feature frame.

7. The pull-up process action decomposition and key frame extraction method according to claim 5, characterized in that: The step of optimizing and screening the candidate key frames to obtain the final key frames includes the following steps: Sorting the candidate key frames in chronological order to obtain a preliminary sorting; Based on the preliminary sorting, identifying the first candidate key frame in the preparation stage as the final key frame; Traversing the remaining candidate key frames in the preliminary sorting, and calculating the time interval between the current candidate key frame and the last selected final key frame in the traversal process; Based on the preliminary judgment condition that the time interval is greater than the preset time interval threshold, and combined with the key features contained in the candidate key frames, the remaining final key frames are screened, and the first final key frame and the remaining final key frames are combined into a final key frame sequence.

8. The pull-up process action decomposition and key frame extraction method according to claim 7, characterized in that: The following steps are also included: Detecting the time intervals between adjacent final key frames in the final key frame sequence; When it is detected that the time interval between two adjacent final key frames exceeds a preset maximum time interval threshold: obtaining speed values ​​of all skipped candidate key frames between the two adjacent final key frames whose time interval exceeds the preset maximum time interval threshold; For each candidate key frame, calculate the sum of the absolute value of the speed difference between the current candidate key frame and the previous candidate key frame and the absolute value of the speed difference between the current candidate key frame and the next candidate key frame as the speed change significance index of the current candidate key frame; select the candidate key frame with the largest speed change significance index as an additional key frame; add the additional key frame to the final key frame sequence.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Citation Information

Patent Citations

  • Human body action standard judgment method and device based on real-time image processing

    CN117115926A

  • Sports test method and system, electronic equipment, chip and storage medium

    CN117292288A

  • Pull-up test counting and completion quality evaluation method based on visual inspection

    CN118609201A

  • Systems And Methods For Generating A Motion Performance Metric

    US20240331170A1

  • Counting method and apparatus, electronic device, storage medium, program, and program product

    WO2024104223A1

Cited By

  • Athlete action assessment method and system

    CN121074984A

  • Athlete movement assessment method and system

    CN121074984B