A human action quality evaluation method based on a large model

Abstract

The present application relates to a kind of human action quality evaluation methods based on large model, comprising the following steps: obtaining public human standard action video and standard action definition data set;Using the three-dimensional coordinates of the specified key point contained in the specified joint of each frame of pose estimation model extraction human, obtain the original key point sequence set of each human standard action;It is normalized to obtain the normalized standard action key point sequence set;By can obtain all standard action abstract file set Y;By the same processing method, obtain the action abstract file set X of the video A to be evaluated;Using X and Y calculate corresponding action angle sequence set Y' and X';Action evaluation template is constructed, and the evaluation result of all the action to be evaluated in A is obtained based on the evaluation template using visual large model.The present application is applicable to the human action standardization evaluation in production line operation specification, sports training, rehabilitation medical treatment and the like.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a method for evaluating the quality of human movements based on a large model. Background Technology

[0002] Existing human motion analysis technologies are mainly applied to motion recognition, motion detection, and motion occurrence determination. Their core objective is to determine whether a human has completed a specific motion, with relatively limited research on the quality assessment of the motion execution process itself. Existing motion quality assessment methods are usually based on human posture estimation results, extracting the positions of key points, joint angles, or their temporal features over time, and combining them with rule-based judgment, temporal matching, or statistical learning models for analysis.

[0003] In terms of specific implementation methods, one common approach is to set rule thresholds based on human experience, constraining and judging low-level geometric features such as key joint angles, inter-joint distances, or movement amplitudes to determine whether the action meets the expected standard. This type of method mainly relies on pre-set judgment rules and lacks an understanding of the overall semantics and execution rationality of the action. Another approach uses standard actions as templates, employing methods such as dynamic time warping, keyframe matching, or temporal similarity calculation to align and compare the action to be evaluated with the standard action at the time series level, thereby obtaining an overall matching degree or similarity score. However, this type of method usually only reflects numerical similarity and is difficult to express the semantic meaning of action deviations. In addition, some studies have introduced traditional machine learning methods such as support vector machines and hidden Markov models, or temporal network models based on deep learning, to classify or regress action quality. However, this type of method usually relies on large-scale labeled data for training, and the evaluation results are mostly output in numerical or categorical form, making it difficult to explain the source of specific action quality problems.

[0004] In summary, while existing technologies can analyze motion execution to some extent from the perspective of human posture, they generally lack effective modeling methods for the relationship between the temporal features of human posture and high-level motion semantics. This makes it difficult to establish a clear and interpretable correspondence between joint angle changes and motion standardization evaluation. Therefore, how to combine the temporal features of human posture with the semantic understanding capabilities of large-scale models to achieve interpretable evaluation of human motion quality has become a core technical problem that urgently needs to be solved in the current technological field. Summary of the Invention

[0005] In view of the above-mentioned problems in the existing technology, the technical problem to be solved by the present invention is: how to combine human posture temporal features with the semantic understanding ability of large models to improve the accuracy of interpretable evaluation of human movement quality.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] A method for assessing human motion quality based on a large model includes the following steps:

[0008] S1: Obtain publicly available videos of standard human body movements and standard action definition data set , Each standard human movement in the standard movement definition data set has a corresponding semantic description; each semantic description of the movement in the movement definition data set includes the movement name, start time, end time, movement description, and mirror mark;

[0009] Specify the key joints that need to be collected;

[0010] S2: Using the attitude estimation model to... Perform frame-by-frame analysis to extract the 3D coordinates of key points contained in all key joints on each video frame. Each key joint includes multiple key points. The original key point sequence set of each human body's standard movements ;

[0011] S3: Yes Normalization is performed to obtain a set of normalized standard action key point sequences. ;

[0012] S4: Based on the start and end times of each person's standard human movement, from Extract the key point sequence of each human standard movement within its corresponding start and end time intervals, and use these key point sequences as the standard movement summary file for the corresponding standard movement; all standard movement summary files constitute set Y;

[0013] S5: Select video A to be evaluated and use a large model based on... Analyze all actions to be evaluated in A and output the start and end times of each action;

[0014] S6: Using the methods in steps S2-S4, obtain the set of summary files X of the actions to be evaluated corresponding to A;

[0015] S7: Calculate the angle sequence set Y' of the key joints corresponding to the standard movements included in Y and the angle sequence set X' of the key joints corresponding to the movements to be evaluated included in X based on Y and X respectively;

[0016] S8: Construct an action evaluation template, input Y' and X' into the visual large model, and determine whether the total number of characters in Y' and X' exceeds the character count threshold of the visual large model. If it does not exceed the character count threshold, the visual large model outputs the evaluation results of all actions to be evaluated in A according to the action evaluation template; otherwise, adaptive optimization processing is performed on Y' and X' to make the total number of characters in Y' and X' less than the character count threshold. Then, the adaptively optimized X' and Y' are input into the visual large model again, and the visual large model outputs the evaluation results of all actions to be evaluated in A according to the action evaluation template.

[0017] Preferably, in S3, the... The normalization process is as follows:

[0018] S3-1: Select any video frame in x and define the hip center coordinates of that video frame. As the reference point for the human body, the calculation expression is as follows:

[0019]

[0020] in, Represents the three-dimensional coordinates of the key points on the left hip. Represents the three-dimensional coordinates of key points on the right hip;

[0021] S3-2: The normalized scaling parameter is defined by calculating the Euclidean distance between the left and right shoulder joints of the video frame. The calculation formula is as follows:

[0022]

[0023] in, and The x and y coordinates of the left shoulder joint are represented sequentially. and The x and y coordinates of the right shoulder joint are represented sequentially.

[0024] S3-3: Preset distance threshold ,Will and When comparing, hour, The value is 1.0; otherwise, the current value is used. The calculated value;

[0025] S3-4: Perform a normalization operation on the original keypoint sequence on the video frame to obtain the normalized 3D coordinates of the keypoints corresponding to the video frame. The calculation formula is as follows:

[0026]

[0027] in, Represents the original three-dimensional coordinates of the key points;

[0028] S3-5: Traverse all video frames on x, repeating S3-1 to S3-4 to obtain the normalized set of standard motion keypoint sequences. .

[0029] Preferably, in step S7, the calculation process for calculating the angle sequence set Y' of the key joints corresponding to the standard movements included in Y and the angle sequence set X' of the key joints corresponding to the movements to be evaluated included in X is as follows:

[0030] S7-1: Define each critical joint as containing three specified key points A, B, and C, wherein key points A, B, and C constitute the structure of the critical joint and are contained within... ;

[0031] Using point B as the vertex of the key joint, construct vectors respectively. and Calculated by vector dot product and magnitude and Angle between cosine value ,in," " represents the vector dot product operation. Indicates the magnitude of the vector; This represents a small constant that prevents division by zero.

[0032] S7-2: Using the inverse cosine function Converted to joint angle The formula is as follows:

[0033]

[0034] in, This means restricting the cosine value to 1 / 2. Within the range, Represents the inverse cosine function;

[0035] S7-3: The joint angles of all key joints in each video frame are calculated using the methods of S7-1 and S7-2; the joint angles of the same key joint in all video frames are arranged in the time sequence of the video frames to form the angle sequence of that key joint.

[0036] S7-4: Perform downsampling processing on the angle sequences corresponding to all key joints to obtain the set of angle sequences Y' of all key joints;

[0037] The downsampling sampling rule can be expressed as follows: ,in, Indicates the downsampling step size. Indicates the first A key joint and , Indicates the total number of specified key joints.

[0038] The angle sequence set X' corresponding to the key joints of the action to be evaluated was obtained using the same method as Y'.

[0039] Preferably, the adaptive optimization process in S8 is as follows:

[0040] First, count the total number of characters in Y' and X'. ,in, The number of characters representing Y' Indicates the number of characters in X';

[0041] Secondly, when Exceeding the preset character count threshold At that time, the data compression process is triggered;

[0042] The data compression process is as follows:

[0043] Set downsampling step size ,according to By uniformly sampling Y' and X' respectively, the angle sequences corresponding to Y' and X' are obtained after compression.

[0044] Next, for all angle values ​​in the angle sequences corresponding to the compressed Y' and X' respectively. All were obtained by performing precision compression processing. The expression is as follows:

[0045]

[0046] in, This indicates that the angle value is rounded. Indicates the number of decimal places to retain;

[0047] Finally, all The resulting set of angle sequences serves as the angle sequence after adaptive optimization.

[0048] Preferably, the process by which the visual large model in S8 outputs the evaluation results of all actions to be evaluated in A based on the action evaluation template is as follows:

[0049] First, according to The action semantic descriptions included in the text divide x and A into K action segments, where each action segment includes at least one standard human action;

[0050] For the The first action segment Calculate the angular deviation of each key joint during the motion segment. The calculation formula is as follows:

[0051]

[0052] in, This represents the peak value of the angle sequence within the action segment, and can be the maximum, minimum, or average value depending on the importance of the key points of the action. This represents the angle value of the j-th critical joint within the k-th motion segment to be evaluated. This represents the angle value of the j-th key joint within the k-th motion segment in the standard movement;

[0053] Secondly, utilize Calculate the overall deviation of the action segment and deviation of movement range The calculation formula is as follows:

[0054]

[0055]

[0056] in, This indicates the total number of key joints. and These represent taking the maximum and minimum values ​​of the angle sequence within the action segment, respectively. This represents the weight coefficient of the nth critical joint;

[0057] according to and Calculate the score of the action to be evaluated in the k-th action segment. The calculation formula is as follows:

[0058]

[0059] in, Indicates the maximum allowable critical angle deviation. This represents the amplitude change of the j-th key joint within the k-th movement segment of the standard movement;

[0060] Finally, the overall score for the action to be evaluated is calculated using the following formula:

[0061]

[0062] in, Indicates the weighting parameter;

[0063] The scoring range is 0–100, and the corresponding levels of action standardization are: 90–100 highly consistent, 70–89 basically correct, 50–69 need improvement, and 0–49 seriously non-standard or missing.

[0064] Compared with the prior art, the present invention has at least the following advantages:

[0065] The method of this invention not only effectively alleviates the problems of strong subjectivity, difficulty in quantification and poor interpretability in traditional collaborative assessment, but also shows significant advantages in terms of assessment accuracy, efficiency and cost control. It has good practicality and promotion value and can be widely applied to practical scenarios such as production line operation specification assessment, sports training guidance and rehabilitation medical monitoring.

[0066] 1. The method of this invention first utilizes computer vision technology to automate the processing of the input video. It employs the MediaPipe pose estimation model to extract the 3D skeleton coordinates of 33 key points of the human body frame by frame from the video, avoiding subjective biases caused by manual observation and experience-based judgment. In the skeleton feature extraction stage, this invention further introduces a normalization processing method based on human structure. Using the hip center as the coordinate origin and the left and right shoulder widths as the scale benchmark, the coordinates of all key points are converted into relative coordinates relative to the human body's own structure. This effectively eliminates differences in height, body shape, shooting distance, and camera angle among different subjects, ensuring good comparability of video data collected under different conditions and guaranteeing the objectivity and consistency of the evaluation results. This solves the problems of strong subjectivity, low efficiency, difficulty in quantification, single evaluation dimension, and insufficient interpretability in existing manual motion evaluation processes.

[0067] 2. This invention transforms the previously difficult-to-quantify issue of motion quality into a set of numerical features with clear physical meaning by calculating joint angles on a normalized human skeletal sequence. By calculating the angle changes of key human joints during motion execution, core indicators such as motion amplitude, posture standardization, and motion completion can be accurately characterized. Combined with data optimization strategies such as angle time-series downsampling and numerical precision control, this invention effectively reduces the data scale while ensuring the complete preservation of key motion information, providing an efficient and stable input representation for subsequent analysis based on large models.

[0068] 3. In the motion evaluation stage, this invention introduces a large-scale model as the evaluation engine. Leveraging its powerful semantic understanding and comprehensive analysis capabilities, it compares and analyzes standard motion angle data with the motion angle data to be evaluated. Unlike traditional methods that rely solely on numerical distance or threshold judgments, the large-scale model achieves multi-dimensional motion quality evaluation. It not only determines whether a motion occurs and whether the sequence of motions is correct, but also provides a comprehensive score from multiple perspectives, including motion standardization, fluency, and amplitude rationality. Furthermore, it outputs detailed textual explanations and improvement suggestions, thereby significantly enhancing the interpretability and practical guidance value of the evaluation results. Attached Figure Description

[0069] Figure 1 This is a schematic diagram of the evaluation steps of the method of the present invention.

[0070] Figure 2 This is a schematic diagram of the model framework of the present invention.

[0071] Figure 3 This is a schematic diagram of the normalization process in this method.

[0072] Figure 4 A visual diagram illustrating the process of assessing the quality of human movement.

[0073] Figure 5 This is a graph comparing the accuracy and consistency of our method with other methods. Detailed Implementation

[0074] The present invention will now be described in further detail.

[0075] This invention discloses a method for evaluating human motion quality based on a large model, thereby achieving automated, objective, and multi-dimensional evaluation of human motion quality. The method includes the following steps: extracting key points of the human skeleton from standard motion videos and motion videos to be evaluated using the MediaPipe pose estimation model; normalizing the key points, using the hip center as the origin and shoulder width as the scale, eliminating the influence of factors such as body size, shooting distance, and angle; calculating joint angle features from the normalized key point sequence, including key joint angles such as elbows, knees, and torsos; downsampling and precision optimization of the angle sequence to adapt to the input limitations of the large model; constructing evaluation prompts to clarify evaluation requirements and scoring criteria; and using a large visual model to compare and analyze the standard motion angle data and the motion angle data to be evaluated, outputting evaluation results including whether the motion occurred, whether the motion sequence was correct, motion quality score, key angle deviation evaluation, detailed text description, and improvement suggestions.

[0076] See Figures 1-5 A method for assessing human motion quality based on a large model includes the following steps:

[0077] S1: Obtain publicly available videos of standard human body movements and standard action definition data set , Each standard human movement in the standard movement definition data set has a corresponding semantic description; each semantic description of the movement in the movement definition data set includes the movement name, start time, end time, movement description, and mirror mark;

[0078] Specify the key joints to be collected; the action definition data is in JSON format. When using video data to analyze human movements, standard action videos are standard and standardized demonstrations of human movements and are publicly available datasets; the mirror flag in the action definition data is marked as yes or no.

[0079] S2: Using the attitude estimation model to... Perform frame-by-frame analysis to extract the 3D coordinates of key points contained in all key joints on each video frame. Each key joint includes multiple key points. The original key point sequence set of each human body's standard movements The pose estimation model uses MediaPipe as an existing technology. When used for human motion analysis, the number of key points is generally 33.

[0080] S3: Yes Normalization is performed to obtain a set of normalized standard action key point sequences. Normalization actually involves using the center of the hip as the origin and the shoulder width as the scale to normalize all key points. This can eliminate the influence of external factors such as body size, shooting distance, and shooting angle.

[0081] In S3, for The normalization process is as follows:

[0082] S3-1: Select any video frame in x and define the hip center coordinates of that video frame. As the reference point for the human body, the calculation expression is as follows:

[0083]

[0084] in, Represents the three-dimensional coordinates of the key points on the left hip. Represents the three-dimensional coordinates of key points on the right hip;

[0085] S3-2: The normalized scaling parameter is defined by calculating the Euclidean distance between the left and right shoulder joints of the video frame. The calculation formula is as follows:

[0086]

[0087] in, and The x and y coordinates of the left shoulder joint are represented sequentially. and The x and y coordinates of the right shoulder joint are represented sequentially.

[0088] S3-3: Preset distance threshold ,Will and When comparing, hour, The value is 1.0; otherwise, the current value is used. The calculated value; this plan is designed for implementation. The value is forcibly set to 1.0 for values ​​less than the threshold to avoid numerical instability or division by zero errors.

[0089] S3-4: Perform a normalization operation on the original keypoint sequence on the video frame to obtain the normalized 3D coordinates of the keypoints corresponding to the video frame. The calculation formula is as follows:

[0090]

[0091] in, Represents the original three-dimensional coordinates of the key points;

[0092] S3-5: Traverse all video frames on x, repeating S3-1 to S3-4 to obtain the normalized set of standard motion keypoint sequences. .

[0093] When the action definition includes a mirror flag, to unify the expression of left and right directions, the normalized keypoint coordinate values ​​are... Mirroring is performed in the horizontal direction, i.e. This ensures the consistency of left-right symmetrical movements in the feature space;

[0094] To eliminate the influence of scale and displacement caused by differences in height, camera distance, and shooting angle among different subjects, the key point sequence of the human skeleton is uniformly normalized based on the hip region, which is the center point of the human body structure, and the shoulder width as the human body scale benchmark.

[0095] S4: Based on the start and end times of each person's standard human movement, from Extract the key point sequence of each human standard movement within its corresponding start and end time intervals, and use these key point sequences as the standard movement summary file for the corresponding standard movement; all standard movement summary files constitute set Y;

[0096] S5: Select video A to be evaluated and use a large model based on... Analyze all actions to be evaluated in A and output the start and end times of each action; the large model here uses the Qwen3-VL-plus model, which is an existing technology;

[0097] S6: Using the methods in steps S2-S4, obtain the set of summary files X of the actions to be evaluated corresponding to A;

[0098] S7: Calculate the angle sequence set Y' of the key joints corresponding to the standard movements included in Y and the angle sequence set X' of the key joints corresponding to the movements to be evaluated included in X based on Y and X respectively;

[0099] In step S7, the calculation process for calculating the angle sequence set Y' of the key joints corresponding to the standard movements included in Y and the angle sequence set X' of the key joints corresponding to the movements to be evaluated included in X is as follows:

[0100] S7-1: Define each critical joint as containing three specified key points A, B, and C, wherein key points A, B, and C constitute the structure of the critical joint and are contained within... ;

[0101] Using point B as the vertex of the key joint, construct vectors respectively. and Calculated by vector dot product and magnitude and Angle between cosine value ,in," " represents the vector dot product operation. Indicates the magnitude of the vector; This represents a small constant to prevent division by zero; both vector dot product calculation and magnitude calculation are existing techniques.

[0102] For each frame of keypoint data for each movement, the following joint angles are specifically calculated: left elbow angle (the angle formed by the left shoulder, left elbow, and left wrist), right elbow angle (the angle formed by the right shoulder, right elbow, and right wrist), left knee angle (the angle formed by the left hip, left knee, and left ankle), right knee angle (the angle formed by the right hip, right knee, and right ankle), torso shoulder angle (the angle formed by the left shoulder, left hip, and right shoulder), hip rotation angle (the angle formed by the left hip, right hip, and right shoulder), and neck tilt angle (the angle formed by the tip of the nose, the midpoint of the neck, and the left shoulder).

[0103] S7-2: Using the inverse cosine function Converted to joint angle The formula is as follows:

[0104]

[0105] in, This means restricting the cosine value to 1 / 2. Within this range, the limitation serves to prevent numerical overflow; Represents the inverse cosine function;

[0106] S7-3: The joint angles of all key joints in each video frame are calculated using the methods of S7-1 and S7-2; the joint angles of the same key joint in all video frames are arranged in the time sequence of the video frames to form the angle sequence of that key joint.

[0107] S7-4: Perform downsampling processing on the angle sequences corresponding to all key joints to obtain the set of angle sequences Y' of all key joints;

[0108] The downsampling sampling rule can be expressed as follows: ,in, Indicates the downsampling step size. Indicates the first A key joint and , This indicates the total number of specified key joints; the purpose of downsampling is to reduce the data size while ensuring the integrity of the action sequence features, thereby reducing the amount of data while retaining key information. Finally, these angle data can be saved as a JSON file.

[0109] The angle sequence set X' corresponding to the key joints of the action to be evaluated was obtained using the same method as Y'.

[0110] Calculating joint angles on the normalized sequence of human joint key points can transform human movements from spatial coordinate representations into motion features that are more physically semantic and interpretable.

[0111] S8: Construct an action evaluation template, input Y' and X' into the visual large model, and determine whether the total number of characters in Y' and X' exceeds the character count threshold of the visual large model. If it does not exceed the character count threshold, the visual large model outputs the evaluation results of all actions to be evaluated in A according to the action evaluation template; otherwise, adaptive optimization processing is performed on Y' and X' to make the total number of characters in Y' and X' less than the character count threshold. Then, the adaptively optimized X' and Y' are input into the visual large model again, and the visual large model outputs the evaluation results of all actions to be evaluated in A according to the action evaluation template.

[0112] The evaluation results include whether the action occurred, whether the action sequence was correct, the action quality score (0-100 points), the assessment of key angle deviations, detailed written descriptions, and improvement suggestions; they are generally saved as a common JSON format file.

[0113] By calling a large model to automatically evaluate the quality of motion angle data, the standard motion and the motion angle data to be evaluated are serialized into compact JSON, and a multimodal input message is constructed. Each motion segment output contains a JSON object with fields including "action", "appeared" (whether the action occurred), "sequence_correct" (whether the action order is correct), "angle_deviation" (angle deviation), and "score" (motion quality score). The "angle_deviation" object can be directly filled in. The values ​​are then used to generate a “comment” description that automatically describes motion problems by combining deviation and amplitude difference. After parsing the results returned by the large model, the integrity of all motion segments is checked to ensure that each record contains predefined fields, thereby obtaining motion quality assessment results that can be displayed, stored, or analyzed later.

[0114] The evaluation template includes evaluation prompts, evaluation requirements, and scoring criteria. The evaluation prompts serve as natural language guidance information, specifically defining the evaluation role, evaluation objective, output structure, and focus of the visual large-scale model. The evaluation requirements refer to the judgment principles and constraints for evaluating the action to be evaluated. The scoring criteria refer to the rules for scoring the action to be evaluated. This system maps the comprehensive judgment result of the large-scale model on the action quality into a unified quantitative scoring system, specifying the range of action quality scores and the corresponding levels of action standardization for different score intervals.

[0115] The relationship between assessment prompts, assessment requirements, and scoring criteria is as follows: assessment prompts guide the reasoning direction and output format of the large model; assessment requirements constrain assessment behavior and judgment criteria; and scoring criteria unify the quantitative expression of assessment results, thereby jointly ensuring the stability, controllability, and interpretability of the action quality assessment results.

[0116] In this invention, the following motion evaluation template is constructed and input into the visual large model as an evaluation instruction: "You are a professional motion quality evaluation expert. Based on the input standard motion angle sequence and the motion angle sequence to be evaluated, please compare and analyze each motion segment and output a structured evaluation result."

[0117] The evaluation results are output in JSON format, and the evaluation results for each action segment include at least the following fields:

[0118] action is used to represent the action name;

[0119] "appeared" indicates whether the action appeared in the video to be evaluated;

[0120] sequence_correct is used to indicate whether the order of the action in the overall action sequence is correct;

[0121] The score is used to represent the quality rating of the action, with a rating range of 0–100.

[0122] angle_deviation is used to represent the evaluation result of the angle deviation of key joints. It is a dictionary structure containing the angle deviation values ​​of each key joint.

[0123] Comments are used to provide written descriptions of the quality of action execution, existing problems, and suggestions for improvement.

[0124] The following evaluation requirements should be met during the evaluation process:

[0125] (1) The input motion angle data has been normalized according to human body scale and shooting conditions, and is not judged based on absolute coordinate differences;

[0126] (2) Focus on the trend of joint angle changes, peak characteristics, range of motion, and degree of completion of the movement;

[0127] (3) Make a comprehensive judgment on whether the action occurred, the correctness of the action sequence, and the quality of the action;

[0128] (4) When an action does not occur, the appeared field is false and the score field is 0.

[0129] The scoring criteria for action quality are as follows:

[0130] A score of 90–100 indicates that the movement is performed correctly and that the changes in the angles of the key joints are highly consistent with the standard movement.

[0131] A score of 70–89 indicates that the movement was basically correct, but there were slight deviations in angle or amplitude.

[0132] A score of 50–69 indicates that there are obvious problems with the movement and it needs to be improved;

[0133] A score of 0–49 indicates severely improper or missing movements.

[0134] The adaptive optimization process in S8 is as follows:

[0135] First, count the total number of characters in Y' and X'. ,in, The number of characters representing Y' Indicates the number of characters in X';

[0136] Secondly, when Exceeding the preset character count threshold At that time, the data compression process is triggered;

[0137] The data compression process is as follows:

[0138] Set downsampling step size ,according to By uniformly sampling Y' and X' respectively, the angle sequences corresponding to Y' and X' are obtained after compression.

[0139] Next, for all angle values ​​in the angle sequences corresponding to the compressed Y' and X' respectively. All were obtained by performing precision compression processing. The expression is as follows:

[0140]

[0141] in, This indicates that the angle value is rounded. Indicates the number of decimal places to retain; for example, it can be set during implementation. ;

[0142] Finally, all The resulting set of angle sequences serves as the angle sequence after adaptive optimization.

[0143] Adaptive optimization is designed to ensure the complete transfer of standard motion angle data and motion angle data to be evaluated, even under conditions where the input length of a large model is limited.

[0144] The process by which the visual large model in S8 outputs the evaluation results of all actions to be evaluated in A based on the action evaluation template is as follows:

[0145] First, according to The action semantic descriptions included in the text divide x and A into K action segments, where each action segment includes at least one standard human action;

[0146] For the The first action segment Calculate the angular deviation of each key joint during the motion segment. The calculation formula is as follows:

[0147]

[0148] in, This represents the peak value of the angle sequence within the action segment, and can be the maximum, minimum, or average value depending on the importance of the key points of the action. This represents the angle value of the j-th critical joint within the k-th motion segment to be evaluated. This represents the angle value of the j-th key joint within the k-th motion segment in the standard movement;

[0149] Secondly, utilize Calculate the overall deviation of the action segment and deviation of movement range The calculation formula is as follows:

[0150]

[0151]

[0152] in, This indicates the total number of key joints. and These represent taking the maximum and minimum values ​​of the angle sequence within the action segment, respectively. This represents the weighting coefficient of the nth critical joint; the weighting coefficient is set according to the degree of motion participation or motion correlation of the critical joint in the corresponding action;

[0153] according to and Calculate the score of the action to be evaluated in the k-th action segment. The calculation formula is as follows:

[0154]

[0155] in, Indicates the maximum allowable critical angle deviation. This represents the amplitude change of the j-th key joint within the k-th movement segment of the standard movement;

[0156] Finally, the overall score for the action to be evaluated is calculated using the following formula:

[0157]

[0158] in, Indicates the weighting parameter; It is generally set based on the movement structure and the degree of joint involvement;

[0159] The scoring range is 0–100, and the corresponding levels of action standardization are: 90–100 highly consistent, 70–89 basically correct, 50–69 need improvement, and 0–49 seriously non-standard or missing. The purpose of constructing evaluation prompts is to guide the large model to conduct a stable, controllable and interpretable evaluation of action quality.

[0160] Experimental content and results

[0161] Example: To verify the feasibility of the method of the present invention, this example selects two single-person action demonstration videos, each approximately 30 seconds long, as test samples. The video resolution is 1920×1080, and the frame rate is 30fps. The first video is a standard action demonstration video. The first video segment is used to construct the standard movement angle sequence Y'; the second video segment is the video to be evaluated, used to verify the feasibility of the movement quality evaluation method of the present invention. Both the standard video and the video to be evaluated contain a "preparatory section", "first section stretching exercise" and "second section chest expansion exercise", and each section consists of multiple eight-beat movements forming a complete movement sequence.

[0162] First, the standard action demonstration video. Perform action time interval labeling and construct a standard action definition dataset. The aforementioned The data is stored using a JSON structure. Each record includes a section name, an action name, a start time, an end time, an action description, and a mirror flag. Each section contains multiple eight-beat actions, and each action segment is identified by its start and end times, which are used for subsequent action alignment and segment extraction of keypoint sequences.

[0163] Subsequently, a pose estimation model was used to analyze the standard video. Frame-by-frame keypoint extraction is performed, outputting the 3D coordinates of 33 keypoints in the human body for each frame, forming a standard keypoint sequence set y. Taking a single frame as an example, the coordinates of keypoints such as the shoulder and hip joints can be obtained, thus constructing complete skeletal structure data. Based on this, the keypoint sequence is normalized. Using the centers of the left and right hip joints as the origin and the distance between the left and right shoulder joints as the scale parameter, a normalization transformation is performed on all keypoints. If the mirror flag in the corresponding action definition is true, a mirror transformation is performed on the normalized horizontal coordinates to ensure the consistency of left-right symmetrical actions in the feature space. The processed result is a normalized standard keypoint sequence set. .

[0164] After normalization, the system generates a keypoint intermediate data file, `standard_keypoints`, to record the normalized 3D coordinates of human keypoints in each frame. The `standard_keypoints` file can be stored in a matrix or JSON structure, with each frame containing coordinate data for multiple keypoints. For example, in a frame of "Section 1 Stretching Exercise_First Eight Beats", the normalized partial keypoint data is represented as follows:

[0165] Left shoulder: (-0.0520, -3.5750, -2.8181)

[0166] Right shoulder: (0.0870, -3.5710, -2.8199)

[0167] Left elbow: (-0.5265, -2.5701, -0.9430)

[0168] Right elbow: (0.4702, -2.6506, -0.8574)

[0169] Left wrist: (-0.5716, -1.1293, -0.3550)

[0170] Right wrist: (0.5648, -1.2089, -0.3311)

[0171] Left palm: (-0.6095, 0.2241, -1.5011)

[0172] Right palm: (0.6213, 0.0965, -1.5122)

[0173] This intermediate result provides a clear visual verification that the normalization process has successfully completed coordinate translation and scale unification, providing a standardized skeleton data foundation for subsequent calculations of key joint angles. It can also be used to verify the reproducibility of motion segment extraction and motion alignment. The time interval information in the middle, from Extract the key point sequence corresponding to each action segment to form a standard action summary set Y.

[0174] For the video A to be evaluated, firstly, A is divided into action segments based on the visual big model, and the start and end times of each action segment are output. Then, the same key point extraction, normalization and mirroring methods as the standard video x are used to obtain the set of key point sequences X to be evaluated.

[0175] Subsequently, key joint angles are calculated for each movement segment in the standard movement set Y and the movement set X to be evaluated. These key joints include the elbow, knee, shoulder, and neck-related joints. Joint angles are calculated using a vector dot product and converted to angle values ​​via an inverse cosine function. Each key joint forms a corresponding angle sequence in the time dimension.

[0176] After completing the joint angle calculation, the system generates intermediate data files standard_step and eval_step for the key joint angle sequence, which are used to record the angle changes of key joints in each action segment.

[0177] For example, in certain frames of the "first section of extension movement" action segment, the angle data of some key joints can be represented as follows:

[0178] Table 1. Comparison of key joint angles in standard movements and key joint angles in movements to be evaluated.

[0179]

[0180] A uniform downsampling process is performed on all key joint angle sequences to obtain the standard motion angle sequence set Y' and the motion angle sequence set X' to be evaluated.

[0181] Before inputting the large visual model, the number of characters in the angle sequence data is counted. In this example, the number of characters far exceeds the 260,000-character limit of the large model. To ensure that the amount of data input to the large visual model is kept within the threshold, the system performs uniform downsampling on the standard action angle sequence set Y' and the action angle sequence set X' to be evaluated. Downsampling adopts a fixed frame interval strategy: 5 keyframes are selected for each action segment's angle sequence at a fixed step size. This method reduces the sequence length while preserving the amplitude and rhythmic features of the action as much as possible. After downsampling, the system adjusts the angle value precision to one decimal place, resulting in a new standard action angle sequence set Y'_ds and a new action angle sequence set X'_ds to be evaluated.

[0182] Subsequently, the system converts the downsampled angle sequence sets Y'_ds and X'_ds, along with the preset action evaluation templates, into an input format acceptable to the large visual model, forming the model input data package Input_data.

[0183] After receiving the Input_data, the visual big data model sequentially performs motion quality assessments on each motion segment, generating motion scores and analysis results of key joint angle deviations, and providing improvement suggestions based on the motion completion. The system ultimately generates a motion quality assessment result file, Eval_result, which records the overall score of each motion segment, key joint angle deviations, and specific motion improvement suggestions. For example, for the motion to be evaluated, "first section extension movement," the following results can be obtained:

[0184] Table 2 Evaluation results of the evaluation actions

[0185]

[0186] The processing results of this embodiment can intuitively verify the feasibility of the motion quality assessment method of the present invention in actual videos.

[0187] To further verify the effectiveness of this invention, it was compared with current mainstream human motion assessment methods, as follows:

[0188] Manual: This refers to a subjective assessment by professional evaluators who visually observe the video to judge the quality of the movements.

[0189] Raw Coordinates: This method uses the keypoint coordinates extracted by MediaPipe directly and evaluates the similarity of actions by calculating Euclidean distance.

[0190] Normalized Coordinates: This method normalizes the coordinates of key points and then evaluates the similarity of actions by calculating Euclidean distance.

[0191] Angles: A joint angle-based method that evaluates motion by calculating the difference in joint angles, but does not introduce semantic understanding into a large model.

[0192] Video LLM: A large model-based approach that takes the original video as input and uses a large model to evaluate motion quality.

[0193] MQA: The method proposed in this invention combines normalized skeleton keypoints, joint angle calculation, and large model semantic evaluation.

[0194] Experimental results show that this invention significantly outperforms other methods in terms of accuracy, consistency, efficiency, and interpretability. On the broadcast gymnastics dataset, MQA achieves an accuracy of 86%, higher than human evaluation (75.0%), RawCoordinates (58.5%), Normalized Coordinates (47.3%), Angles (81.2%), and Video LLM (68.2%). In terms of consistency, MQA reaches 95.2%, significantly higher than Raw Coordinates (52.3%), NormalizedCoordinates (78.6%), Angles (82.1%), Video LLM (88.9%), and human evaluation (45.8%). Regarding evaluation efficiency, RCFC achieves fully automated processing, averaging 1.5 minutes for a 2-minute video, approximately 5 times faster than human evaluation; Video LLM takes the longest, at 3.5 minutes. In terms of interpretability, RCFC can provide 100% natural language explanations and improvement suggestions, far exceeding human evaluation (60%) and other methods (0%).

[0195] This invention achieves significant results in terms of both assessment accuracy and efficiency. Through normalization and joint angle modeling, this invention effectively eliminates the interference of environmental and individual differences on the assessment results, improving the assessment accuracy by approximately 15%–25% compared to methods that directly use the original coordinates. Simultaneously, thanks to the fully automated processing flow, no manual intervention is required from video input to assessment result output, improving assessment efficiency by approximately 10–50 times compared to manual assessment.

[0196] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for evaluating the quality of human movements based on a large model, characterized in that: Includes the following steps: S1: Obtain publicly available videos of standard human body movements and standard action definition data set , Each standard human movement in the standard movement definition data set has a corresponding semantic description; each semantic description of the movement in the movement definition data set includes the movement name, start time, end time, movement description, and mirror mark; Specify the key joints that need to be collected; S2: Using the attitude estimation model to... Perform frame-by-frame analysis to extract the 3D coordinates of key points contained in all key joints on each video frame. Each key joint includes multiple key points. The original key point sequence set of each human body's standard movements ; S3: Yes Normalization is performed to obtain a set of normalized standard action key point sequences. ; S4: Based on the start and end times of each person's standard human movement, from Extract the key point sequence of each human standard movement within its corresponding start and end time intervals, and use these key point sequences as the standard movement summary file of the corresponding standard movement; All standard action summary files constitute set Y; S5: Select video A to be evaluated and use a large model based on... Analyze all actions to be evaluated in A and output the start and end times of each action; S6: Using the methods in steps S2-S4, obtain the set of summary files X of the actions to be evaluated corresponding to A; S7: Calculate the angle sequence set Y' of the key joints corresponding to the standard movements included in Y and the angle sequence set X' of the key joints corresponding to the movements to be evaluated included in X based on Y and X respectively; S8: Construct an action evaluation template, input Y' and X' into the visual large model, and determine whether the total number of characters in Y' and X' exceeds the character count threshold of the visual large model. If it does not exceed the character count threshold, the visual large model outputs the evaluation results of all actions to be evaluated in A according to the action evaluation template; otherwise, adaptive optimization processing is performed on Y' and X' to make the total number of characters in Y' and X' less than the character count threshold. Then, the adaptively optimized X' and Y' are input into the visual large model again, and the visual large model outputs the evaluation results of all actions to be evaluated in A according to the action evaluation template.

2. The method for assessing human motion quality based on a large model as described in claim 1, characterized in that: In S3, for The normalization process is as follows: S3-1: Select any video frame in x and define the hip center coordinates of that video frame. As the reference point for the human body, the calculation expression is as follows: in, Represents the three-dimensional coordinates of the key points on the left hip. Represents the three-dimensional coordinates of key points on the right hip; S3-2: The normalized scaling parameter is defined by calculating the Euclidean distance between the left and right shoulder joints of the video frame. The calculation formula is as follows: in, and The x and y coordinates of the left shoulder joint are represented sequentially. and The x and y coordinates of the right shoulder joint are represented sequentially. S3-3: Preset distance threshold ,Will and When comparing, hour, The value is 1.0; otherwise, the current value is used. The calculated value; S3-4: Perform a normalization operation on the original keypoint sequence on the video frame to obtain the normalized 3D coordinates of the keypoints corresponding to the video frame. The calculation formula is as follows: in, Represents the original three-dimensional coordinates of the key points; S3-5: Traverse all video frames on x, repeating S3-1 to S3-4 to obtain the normalized set of standard motion keypoint sequences. .

3. The method for assessing human motion quality based on a large model as described in claim 2, characterized in that: In step S7, the calculation process for calculating the angle sequence set Y' of the key joints corresponding to the standard movements included in Y and the angle sequence set X' of the key joints corresponding to the movements to be evaluated included in X is as follows: S7-1: Define each critical joint as containing three specified key points A, B, and C, wherein key points A, B, and C constitute the structure of the critical joint and are contained within... ; Using point B as the vertex of the key joint, construct vectors respectively. and Calculated by vector dot product and magnitude and Angle between cosine value ,in," " represents the vector dot product operation. Indicates the magnitude of the vector; This represents a small constant that prevents division by zero. S7-2: Using the inverse cosine function Converted to joint angle The formula is as follows: in, This means restricting the cosine value to 1 / 2. Within the range, Represents the inverse cosine function; S7-3: The joint angles of all key joints in each video frame are calculated using the methods of S7-1 and S7-2; the joint angles of the same key joint in all video frames are arranged in the time sequence of the video frames to form the angle sequence of that key joint. S7-4: Perform downsampling processing on the angle sequences corresponding to all key joints to obtain the set of angle sequences Y' of all key joints; The downsampling sampling rule can be expressed as follows: ,in, Indicates the downsampling step size. Indicates the first A key joint and , Indicates the total number of specified critical joints; The angle sequence set X' corresponding to the key joints of the action to be evaluated was obtained using the same method as Y'.

4. The method for evaluating human motion quality based on a large model as described in claim 3, characterized in that: The adaptive optimization process in S8 is as follows: First, count the total number of characters in Y' and X'. ,in, The number of characters representing Y' Indicates the number of characters in X'; Secondly, when Exceeding the preset character count threshold At that time, the data compression process is triggered; The data compression process is as follows: Set the downsampling step size ,according to By uniformly sampling Y' and X' respectively, the angle sequences corresponding to Y' and X' are obtained after compression. Next, for all angle values ​​in the angle sequences corresponding to the compressed Y' and X' respectively. All were obtained by performing precision compression processing. The expression is as follows: in, This indicates that the angle value is rounded. Indicates the number of decimal places to retain; Finally, all The resulting set of angle sequences serves as the angle sequence after adaptive optimization.

5. The method for assessing human motion quality based on a large model as described in claim 4, characterized in that: The process by which the visual large model in S8 outputs the evaluation results of all actions to be evaluated in A based on the action evaluation template is as follows: First, according to The action semantic descriptions included in the text divide x and A into K action segments, where each action segment includes at least one standard human action; For the The first action segment Calculate the angular deviation of each key joint during the motion segment. The calculation formula is as follows: in, This represents the peak value of the angle sequence within the action segment, and can be the maximum, minimum, or average value depending on the importance of the key points of the action. This represents the angle value of the j-th critical joint within the k-th motion segment to be evaluated. This represents the angle value of the j-th key joint within the k-th motion segment in the standard movement; Secondly, utilize Calculate the overall deviation of the action segment and deviation of movement range The calculation formula is as follows: in, This indicates the total number of key joints. and These represent taking the maximum and minimum values ​​of the angle sequence within the action segment, respectively. This represents the weight coefficient of the nth critical joint; according to and Calculate the score of the action to be evaluated in the k-th action segment. The calculation formula is as follows: in, Indicates the maximum allowable critical angle deviation. This represents the amplitude change of the j-th key joint within the k-th movement segment of the standard movement; Finally, the overall score for the action to be evaluated is calculated using the following formula: in, Indicates the weighting parameter; The scoring range is 0–100, and the corresponding levels of action standardization are: 90–100 highly consistent, 70–89 basically correct, 50–69 need improvement, and 0–49 seriously non-standard or missing.

Images