RLHF-based motion similarity score optimization method

Abstract

The invention discloses a motion similarity score optimization method based on RLHF, and the method comprises the following steps: S1, collecting a motion video of a user through a camera, and inputting the motion video as original video data; s2, adopting an OpenPose human body posture recognition algorithm for the original video data, and extracting coordinate information of human body key points in each frame of image; s3, constructing the human body key point data into a time sequence, inputting the time sequence into a Transform model, and extracting a time sequence dynamic feature and a global structure feature of the action; s4, jointly inputting the action feature representation and the feature representation of the preset standard action into a similarity calculation module, and calculating the matching degree between the action feature representation and the feature representation of the preset standard action; and S5, introducing a PPO algorithm and a human feedback mechanism, and constructing the feedback signal as an award. And S6, performing associated display on the optimized similarity score and the original action video. According to the method, reinforcement learning and human feedback technologies are combined, and motion similarity score optimization based on RLHF is realized.

Description

Technical Field

[0001] This invention relates to the field of intelligent action scoring technology, and in particular to an action similarity scoring optimization method based on RLHF. Background Technology

[0002] Driven by the rapid development of computer vision and artificial intelligence, video-based sports motion analysis has gradually become an important application area in various fields such as sports training, physical education, rehabilitation guidance, and smart fitness. By analyzing users' motion performance in videos, it is possible not only to compare standard movements with executed movements, but also to quantitatively evaluate the accuracy, consistency, and standardization of movements, providing users with scientific guidance and feedback support. Currently, mainstream methods mostly employ human pose recognition technology, such as OpenPose and MediaPipe models, to extract key human body information from video images, including skeletal structural features such as the head, limbs, and joints, and use this as the basis for subsequent feature modeling and motion comparison.

[0003] To accurately model and score motion processes, some research and applications have introduced traditional algorithms such as Euclidean distance and dynamic time warping to compare keypoint trajectories in time and space. These methods can, to some extent, measure differences in execution paths, rhythms, and amplitudes of movements and have been widely applied in scenarios such as instructional analysis, fitness exercise tracking, and rehabilitation training monitoring. Meanwhile, deep learning models, especially temporal modeling networks such as Transformer and LSTM, are also being gradually introduced into sports motion recognition tasks. By using keypoint sequences as input, the models can automatically learn the temporal dynamic features and global structural information of motion movements, achieving more robust similarity calculations and classification predictions.

[0004] Publication patent CN202311522881.9 provides a method for sports motion analysis based on video images. It primarily extracts multiple keyframes from the video, inputs them into a pre-trained model, and combines them with evaluation data of multiple key sub-motions to ultimately derive the overall evaluation score of the target motion. In this scheme, keypoint extraction relies on tools such as MediaPipe, while the motion comparison part uses the KNN algorithm for similarity calculation, possessing strong automated scoring capabilities. Patent CN202411158191.4 proposes a technical path for Baduanjin (Eight Pieces of Brocade) motion analysis. After the user uploads a video via a mobile device, the system completes 3D pose reconstruction on the server side, and then uses a Transformer network and DTW algorithm to compare the standard and actual movements. This scheme effectively supports spatial modeling and temporal alignment of complex movements, but it is essentially still offline analysis and cannot provide real-time guidance and scoring optimization during user operation. Another patent, CN202311549845.1, proposes a posture recognition-based scoring method for dance movements. It improves the accuracy of dance posture similarity assessment by aligning key point features in a spatiotemporal manner through a motion receptive field mechanism.

[0005] While the aforementioned technologies have achieved digital assessment and structured modeling of actions to some extent, they still have significant limitations in practical applications. First, existing technologies generally rely on model training and algorithmic rules for scoring, lacking the involvement of human subjective judgment. For example, whether calculating trajectory error through DTW or extracting feature vectors and calculating cosine similarity using Transformers, these methods are essentially static and rule-based calculations, failing to reflect the comprehensive perception and experiential judgment relied upon by humans in the scoring process. More importantly, when users perceive the scoring results as inconsistent with their own understanding, existing systems often fail to respond to this feedback and adjust accordingly, resulting in a lack of interpretability in model scoring and an inability to adapt to the personalized needs of different individuals or scenarios.

[0006] Secondly, current mainstream technologies mostly operate on an offline analysis model, meaning that motion data acquisition and scoring model training are separated, lacking interactivity and real-time capabilities. Users often only receive a system score after completing the entire movement, and this score is directly generated by a static model, lacking the ability to dynamically optimize based on user behavior. This approach may reduce feedback efficiency in training, rehabilitation, or teaching scenarios, failing to meet users' expectations for real-time correction and personalized adjustments. For example, in fitness training, users expect to receive system alerts the instant an incorrect movement occurs, thus avoiding the formation of incorrect movement patterns; while in rehabilitation scenarios, patients' range of motion may differ from standard movements due to physical limitations, and existing systems cannot distinguish between "reasonable differences" and "incorrect movements," resulting in distorted feedback.

[0007] Therefore, how to provide an action similarity scoring optimization method based on RLHF is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0008] One objective of this invention is to propose an action similarity scoring optimization method based on RLHF (Reinforcement Learning High-Frequency Handling). This invention combines advanced technologies such as reinforcement learning and human feedback. It extracts the temporal dynamic features of actions through a Transformer model and introduces subjective human feedback to construct a reward signal, achieving adaptive optimization of action similarity scoring. The system uses the PPO (Progressive Point of Interest) algorithm to update the initial scoring results, dynamically adjusting the scoring mechanism to continuously align the model with the user's actual perception. Simultaneously, it integrates visual comparison and graphical interface interaction, enabling real-time user participation in the scoring feedback process. This supports personalized adjustments and continuous optimization of scoring criteria, significantly improving the intelligence level of action analysis and the user interaction experience, and possessing subjective perception capabilities, dynamic learning capabilities, and high adaptability.

[0009] An action similarity scoring optimization method based on RLHF according to an embodiment of the present invention includes the following steps: S1. Capture the user's action video through the camera and use it as the raw video data input; S2. The OpenPose human pose recognition algorithm is used on the original video data to extract the coordinate information of human key points in each frame of the image and obtain the human key point data. S3. Construct the human body key point data into a time series and input it into the Transformer model to extract the temporal dynamic features and global structural features of the action to obtain the action feature representation; S4. Input the action feature representation and the feature representation of the preset standard action into the similarity calculation module, and use the cosine similarity measurement method to calculate the degree of matching between the two to obtain the preliminary similarity score. S5. Based on the preliminary similarity score, the PPO algorithm and human feedback mechanism are introduced to collect feedback information from multiple users on the score, construct the feedback signal as a reward, update the parameters of the human feedback optimization model, and output the optimized action similarity score result. S6. Associate the optimized similarity score with the original action video and display the result to provide the user with an action rating for reference.

[0010] Optionally, S1 specifically includes: S11. Use a camera to capture motion video or input an existing motion video; S12. Decompose the video into consecutive frames, usually at 30 frames per second. S13. Resize and normalize each frame, and store the processed video data in the form of a frame sequence.

[0011] Optionally, S2 specifically includes: S21. Perform human key point detection on video data; S22. Use OpenPose to detect human key points in each frame of the image; S23. Extract 18 key points of the human body, including the head, torso, and limbs; S24. Normalize the coordinates of key points in each frame to ensure that videos of different resolutions can be adapted to a unified model. Use the coordinates of key points as reference points to relativize the coordinates of all key points and eliminate differences in scale and position. S25. If the video lengths are different, use dynamic time warping or linear interpolation for alignment. S26. Concatenate the normalized data into a time series: ; in, For time series, T is the number of video frames.

[0012] Optionally, S3 specifically includes: S31. Input the time series data into the transformer model for training; S32. Preprocess and embed the data, add position encoding to each frame, and map the input sequence to a higher-dimensional feature space through an embedding layer. S33. After position encoding and embedding, a feature sequence with temporal information is obtained:

[0013] in, It is a feature sequence with temporal information. Let T be the number of video frames, N be the number of human keypoints, and E be the dimension of each keypoint. S34. Input the feature sequence with temporal information into the Transformer model, and after calculation by multiple encoder layers, obtain the output: ; in, The output features computed by the Transformer, and the attention mechanism in the Transformer are as follows: ; in, This is the attention weight matrix. The calculated attention scores are normalized. For querying the matrix, The key matrix, This is the transpose of the key matrix. For feature dimensions; S35. Perform global average pooling on the output of the Transformer to obtain the aggregated action features: ; in, For time steps, For the first Output features at each time step.

[0014] Optionally, S4 specifically includes: S41. In the output of the Transformer model, the similarity between the standard action and the action to be evaluated is calculated using cosine similarity: ; in, The similarity between the standard movement and the movement to be evaluated. The characteristic representation of standard actions, The feature representation of the action to be evaluated. The dot product of the standard movement and the movement to be evaluated. For the standard motion, the Euclidean norm is... Let be the Euclidean norm of the action to be evaluated; S42. Map the cosine similarity from [-1,1] to the scoring range of [0,1] or [0,100] through linear transformation and output the preliminary similarity score.

[0015] Optionally, S5 specifically includes: S51. After obtaining the action similarity score, the score is optimized by RLHF, specifically by using the PPO algorithm and combining human feedback to optimize the model output. S52. The reward function is designed to optimize the model's scoring based on human feedback. For each pair of standard actions and actions to be evaluated, the model calculates the reward based on human feedback. If the model's score is close to the true score, the reward is larger; if the model's score deviates from the true score, the reward is smaller, or even negative.

[0016] in, This is the final calculated average reward value. The number of users who participated in the feedback. For the first The reward value for each feedback, For the first User feedback tags For the model to the first The scoring results for each action; S53 and PPO improve upon traditional policy optimization algorithms by introducing a truncation advantage. The objective function is divided into two parts, and the goal of PPO is to maximize the following policy objective function: ; in, Update the target for the strategy. For the parameters of the policy network, For the expected value, The ratio of the policy probabilities. For the dominant function, Cut to the strategy ratio; S54. In order for the strategy to learn reward signals, the value function also needs to be optimized:

[0017] in, Current value function with respect to state The predicted value, The target value; S55. The final loss function is a weighted sum of the following parts: - + ; in, For the final loss function, For the entropy of the strategy, To balance the weights of each part using hyperparameters; S56. Stochastic gradient descent is used to update the parameters for policy updates. : ; in loss function Regarding parameters gradient, This is the learning rate.

[0018] Optionally, S6 specifically includes: S61. Use a graphical interface to display similarity scores and action evaluation results; S62. Provides visual feedback, enabling key point comparison and synchronized video playback; S63. Update similarity results in real time based on user ratings to guide users in adjusting their actions.

[0019] The beneficial effects of this invention are: This invention provides an optimized method for human motion similarity scoring based on RLHF (Dynamic Time Warping), aiming to achieve a more intelligent, accurate, and subjectively perceptive motion evaluation system. Traditional motion similarity calculation methods often rely on pre-defined algorithmic metrics, such as dynamic time warping, Euclidean distance, or feature similarity extracted by fixed deep learning models. However, these methods often fail to accurately capture the subjective feelings of humans when evaluating motion quality, especially in scenarios involving sports training, rehabilitation guidance, and dance instruction, where different coaches or experts may give different scores for the same motion. Therefore, current objective scoring methods are significantly lacking in flexibility and personalization, making it difficult to meet the higher requirements for feedback accuracy and interactivity in practical applications.

[0020] This invention incorporates reinforcement learning and human feedback mechanisms, adding subjective human judgment to the model scoring process. This makes the scoring results more closely resemble "human evaluation" in real-world usage scenarios. After seeing the initial model score, users can provide feedback based on their own standards, experience, or preferences, such as believing the score is too high or too low, or that the action is inaccurate or the rhythm is unstable. The system collects this feedback information, constructs reward signals, and drives the policy network to update parameters, thereby gradually optimizing the similarity scoring model. Through continuous iterative learning, subjective human judgment is internalized into the model's behavioral guidelines, making subsequent scoring decisions more consistent with public perception, truly achieving a "human-taught-machine" learning process. This learning method, which combines subjective feedback, overcomes the problem of traditional algorithms relying solely on data pattern matching and neglecting human perception, significantly improving the interpretability and user experience of the scoring system.

[0021] On the other hand, this invention also supports real-time optimization and dynamic score updates, enabling the scoring system to possess high interactivity and responsiveness. Traditional systems mostly operate offline, completing action recognition, feature extraction, and similarity calculation before outputting a fixed score, lacking interactive feedback with the user. In contrast, this invention constructs a real-time updated scoring model, allowing users to see the score results after action execution during training. If the system score does not match their expectations, they can directly provide feedback through the interface. Upon receiving this feedback, the system immediately updates the model parameters and dynamically adjusts the scoring logic. This real-time interactive mechanism not only improves the efficiency and practicality of action training but also allows the model to continuously evolve during use, gradually building personalized scoring strategies suitable for specific user groups or application areas. For example, in rehabilitation training, different patients may have physiological differences in the range of motion compared to standard movements. A model trained using RLHF can gradually adapt to these individual differences, outputting more instructive results.

[0022] Furthermore, this invention designs a complete motion data processing flow, from video acquisition, key point extraction, time series construction, Transformer modeling, similarity calculation to RLHF optimized scoring, and finally to the visualization of the results, forming an end-to-end intelligent motion scoring system. The system not only possesses high-precision perception capabilities but also adaptive learning capabilities, enabling it to optimize its judgment model through multiple interactions. The introduction of a graphical interface further enhances the user experience, allowing users to intuitively understand the differences between their own movements and standard movements through synchronized video playback and key point comparison, thus assisting them in improving training effectiveness.

[0023] In summary, this invention establishes an intelligent action scoring framework that combines accuracy, personalization, and interactivity by integrating the action recognition and modeling capabilities of deep learning, the optimization strategies of reinforcement learning, and the interactive design of human subjective feedback. This method not only solves the problems of insufficient subjectivity and weak real-time performance in existing action evaluation systems, but also enables continuous optimization and self-learning. It is applicable to various practical scenarios such as physical education, sports training, rehabilitation guidance, and human-computer interaction, demonstrating significant technical advantages and application value. Attached Figure Description

[0024] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0025] Figure 1 This is a flowchart of an action similarity scoring optimization method based on RLHF proposed in this invention; Figure 2 This is a schematic diagram of an action similarity scoring optimization method based on RLHF proposed in this invention; Figure 3 This is a data flow diagram of an action similarity scoring optimization method based on RLHF proposed in this invention. Detailed Implementation

[0026] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0027] refer to Figure 1-3 An action similarity scoring optimization method based on RLHF includes the following steps: S1. Capture the user's action video through the camera and use it as the raw video data input; S2. The OpenPose human pose recognition algorithm is used on the original video data to extract the coordinate information of human key points in each frame of the image and obtain the human key point data. S3. Construct the human body key point data into a time series and input it into the Transformer model to extract the temporal dynamic features and global structural features of the action to obtain the action feature representation; S4. Input the action feature representation and the feature representation of the preset standard action into the similarity calculation module, and use the cosine similarity measurement method to calculate the degree of matching between the two to obtain the preliminary similarity score. S5. Based on the preliminary similarity score, the PPO algorithm and human feedback mechanism are introduced to collect feedback information from multiple users on the score, construct the feedback signal as a reward, update the parameters of the human feedback optimization model, and output the optimized action similarity score result. S6. Associate the optimized similarity score with the original action video and display the result to provide the user with an action rating for reference.

[0028] This invention introduces an RLHF-based action similarity scoring optimization method. It utilizes OpenPose to extract human keypoint data and combines it with a Transformer model to model the temporal features of the keypoint sequence, generating stable action feature representations. Furthermore, it calculates the matching degree between the user's action and the standard action using cosine similarity to obtain preliminary scoring results. The system incorporates human feedback signals as a reward mechanism and continuously adjusts model parameters based on a reinforcement learning strategy to achieve dynamic optimization of the scoring results. This method effectively integrates objective data analysis with subjective evaluation perception, improving the personalization and accuracy of the scoring results, and is applicable to various sports training and action guidance scenarios.

[0029] In this embodiment, S1 specifically includes: S11. Use a camera to capture motion video or input an existing motion video; S12. Decompose the video into consecutive frames, usually at 30 frames per second. S13. Resize and normalize each frame, and store the processed video data in the form of a frame sequence.

[0030] This invention achieves structured preprocessing of raw data through frame-by-frame extraction and standardization of action videos, ensuring the consistency and trainability of subsequent model inputs. A video decomposition strategy of 30 frames per second is employed to retain sufficient action detail information, and frame size adjustment and normalization operations are used to adapt to video data from various resolutions and device sources. The processed video data is stored as a frame sequence, providing a high-quality, uniformly formatted data foundation for subsequent keypoint detection and temporal modeling. This method improves the universality and stability of video input across different devices and scenarios, ensuring the continuity and accuracy of the action recognition process.

[0031] In this embodiment, S2 specifically includes: S21. Perform human key point detection on video data; S22. Use OpenPose to detect human key points in each frame of the image; S23. Extract 18 key points of the human body, including the head, torso, and limbs; S24. Normalize the coordinates of key points in each frame to ensure that videos of different resolutions can be adapted to a unified model. Use the coordinates of key points as reference points to relativize the coordinates of all key points and eliminate differences in scale and position. S25. If the video lengths are different, use dynamic time warping or linear interpolation for alignment. S26. Concatenate the normalized data into a time series: ; in, For time series, T is the number of video frames.

[0032] This invention constructs action description data with a unified structure and consistent scale by accurately extracting and normalizing key points on the human body, effectively improving the robustness and generalization ability of action comparison. Eighteen key points are extracted using OpenPose, and their coordinates are normalized and relativized to eliminate interference from different resolutions, viewpoints, and positions. For different video lengths, dynamic time warping or linear interpolation is used for sequence alignment to ensure consistency in the temporal dimension. Finally, the data is stitched together into a time series data in a unified format, providing high-quality input for subsequent feature modeling. This method enhances the adaptability and comparison accuracy of action data under multi-source input conditions.

[0033] In this embodiment, S3 specifically includes: S31. Input the time series data into the transformer model for training; S32. Preprocess and embed the data, add position encoding to each frame, and map the input sequence to a higher-dimensional feature space through an embedding layer. S33. After position encoding and embedding, a feature sequence with temporal information is obtained:

[0034] in, It is a feature sequence with temporal information. Let T be the number of video frames, N be the number of human keypoints, and E be the dimension of each keypoint. S34. Input the feature sequence with temporal information into the Transformer model, and after calculation by multiple encoder layers, obtain the output: ; in, The output features computed by the Transformer, and the attention mechanism in the Transformer are as follows: ; in, This is the attention weight matrix. The calculated attention scores are normalized. For querying the matrix, The key matrix, This is the transpose of the key matrix. For feature dimensions; S35. Perform global average pooling on the output of the Transformer to obtain the aggregated action features: ; in, For time steps, For the first Output features at each time step.

[0035] This invention performs deep modeling of the temporal sequence of human keypoints based on the Transformer model. It effectively incorporates temporal information through positional encoding and embedding mechanisms, and extracts global dynamic features using a multi-layer encoder structure. The system accurately models the dependencies between keypoints through an attention mechanism, capturing the temporal evolution and structural features during action execution, thus enhancing the discriminative power of feature representations. Furthermore, global average pooling aggregates the output into a stable action feature vector, ensuring the stability and consistency of subsequent similarity calculations. This method significantly improves the contextual understanding capability of action modeling, providing high-quality semantic feature support for accurate comparison of complex actions.

[0036] In this embodiment, S4 specifically includes: S41. In the output of the Transformer model, the similarity between the standard action and the action to be evaluated is calculated using cosine similarity: ; in, The similarity between the standard movement and the movement to be evaluated. The characteristic representation of standard actions, The feature representation of the action to be evaluated. The dot product of the standard movement and the movement to be evaluated. For the standard motion, the Euclidean norm is... Let be the Euclidean norm of the action to be evaluated; S42. Map the cosine similarity from [-1,1] to the scoring range of [0,1] or [0,100] through linear transformation and output the preliminary similarity score.

[0037] This invention constructs an action representation based on feature vectors and uses cosine similarity to accurately measure the matching degree between user actions and standard actions, effectively solving the problem of traditional distance measurement methods being sensitive to changes in direction and amplitude. A normalized similarity score is obtained by using the ratio of the dot product and norm between the standard action and the action to be evaluated, ensuring that the similarity calculation results are comparable within a unified scoring scale. Furthermore, a linear mapping is used to convert the similarity from [-1,1] to an intuitive scoring range, such as [0,1] or [0,100], improving user understanding and experience. This method combines computational efficiency and interpretability, providing a stable and quantifiable foundation for action scoring.

[0038] In this embodiment, S5 specifically includes: S51. After obtaining the action similarity score, the score is optimized by RLHF, specifically by using the PPO algorithm and combining human feedback to optimize the model output. S52. The reward function is designed to optimize the model's scoring based on human feedback. For each pair of standard actions and actions to be evaluated, the model calculates the reward based on human feedback. If the model's score is close to the true score, the reward is larger; if the model's score deviates from the true score, the reward is smaller, or even negative.

[0039] in, This is the final calculated average reward value. The number of users who participated in the feedback. For the first The reward value for each feedback, For the first User feedback tags For the model to the first The scoring results for each action; S53 and PPO improve upon traditional policy optimization algorithms by introducing a truncation advantage. The objective function is divided into two parts, and the goal of PPO is to maximize the following policy objective function: ; in, Update the target for the strategy. For the parameters of the policy network, For the expected value, The ratio of the policy probabilities. For the dominant function, Cut to the strategy ratio; S54. In order for the strategy to learn reward signals, the value function also needs to be optimized:

[0040] in, Current value function with respect to state The predicted value, The target value; S55. The final loss function is a weighted sum of the following parts: - + ; in, For the final loss function, For the entropy of the strategy, To balance the weights of each part using hyperparameters; S56. Stochastic gradient descent is used to update the parameters for policy updates. : ; in loss function Regarding parameters gradient, This is the learning rate.

[0041] This invention introduces a reinforcement learning mechanism based on the PPO algorithm into the action scoring process. It designs a reward function incorporating human feedback signals to optimize the initial similarity score, achieving continuous improvement and dynamic adjustment of the scoring results. The system constructs a stable policy update objective by truncating the advantage function and uses a loss weighting method that combines the value function error and policy entropy to ensure a balance between exploration and convergence in the optimization process. This method transforms user subjective feedback into learnable signals, driving the model to continuously iterate and update the scoring strategy, significantly enhancing the adaptive capability, personalization level, and scoring accuracy of the action scoring system, thereby improving practicality and interactive experience.

[0042] In this embodiment, S6 specifically includes: S61. Use a graphical interface to display similarity scores and action evaluation results; S62. Provides visual feedback, enabling key point comparison and synchronized video playback; S63. Update similarity results in real time based on user ratings to guide users in adjusting their actions.

[0043] This invention constructs a graphical interactive interface to visually link similarity scoring results with user action videos, enhancing users' understanding of the scoring criteria and results. The system supports key point comparison and synchronized video playback, allowing users to intuitively identify differences between their own actions and standard actions, effectively improving action correction efficiency. Simultaneously, it updates in real-time based on user scoring results, achieving dynamic closed-loop adjustment of scoring feedback. This method possesses high interactivity and interpretability, increasing user engagement while enhancing the transparency and practicality of model scoring, making it widely applicable to real-time interactive scenarios such as sports teaching, fitness guidance, and rehabilitation training.

[0044] Example 1: To verify the feasibility of this invention in practice, it was applied to a physical training coach assistance system at a national-level sports training base. This system is mainly aimed at athletes in track and field, martial arts, and artistic gymnastics, and aims to improve the accuracy and feedback efficiency of training movements through intelligent means.

[0045] In real-world scenarios, training bases process over 200 training videos of athletes daily. Taking track and field sprint training as an example, coaching teams aim to conduct detailed analyses of movements during the starting, acceleration, and sprint phases, particularly regarding body posture control at the start. Coaches often rely on their own experience for judgment, lacking quantitative tools. Traditional movement assessment tools typically only provide error analysis from a single perspective, failing to offer real-time feedback and adjustments based on the coach's professional judgment, and lacking the flexibility to provide evaluation results tailored to individual differences.

[0046] To address the aforementioned issues, this invention is deployed in a training system. Athletes record training videos using a dedicated sports camera, and the system automatically extracts keyframes and uses OpenPose to detect human key points. Subsequently, the system normalizes and temporally aligns the key point sequences and constructs a standard action feature library as a comparison benchmark. In the Transformer module, the system extracts the temporal dynamic features and global structural features of the actions, forming high-dimensional representations of standard actions and user actions, and uses cosine similarity for preliminary scoring.

[0047] Compared to traditional methods, this system incorporates a human feedback mechanism after scoring. Coaches can manually adjust the scores, marking them as "reasonable" or "unreasonable." This feedback is then used as a reward signal input to the reinforcement learning module, which optimizes the scoring strategy through the PPO algorithm. After two weeks of system use, scoring accuracy significantly improved. Initially, the average deviation between the system's scores and the coach's scores was 18.7 points, but after introducing RLHF optimization, the deviation decreased to 5.4 points, and it can adapt to individual differences more quickly. For example, the scoring criteria for Category A and Category B athletes running the same posture can be automatically differentiated, ensuring that movements deemed "reasonable" by the coach are not misjudged as "deviations."

[0048] The system also supports a graphical interface display function, overlaying motion scores, key point annotations, and standard movements onto the video window. Athletes can intuitively see the offset and matching degree of their movements in each frame after each training session. In addition, during a 30-day training cycle, the system recorded that the actual motion scoring error decreased from an average of 15.3 points in the first week to 3.8 points in the fourth week; the average number of times users modified their scores also decreased from 5.2 times per person per day to 1.1 times, indicating that the model gradually learned an evaluation strategy consistent with the coach's standards and has good adaptive capabilities.

[0049] Table 1 Comparison of the optimization effects of the RLHF-based action similarity scoring method

[0050] As can be seen from the table above, this invention demonstrates significantly superior scoring consistency and human-machine integration efficiency compared to traditional methods in training multiple projects. Especially in the initial individual scoring learning phase, this system can quickly converge to a reasonable scoring strategy, reducing reliance on secondary manual correction. The system's fast response time, low scoring error, and strong interpretability fully demonstrate its practicality and advanced nature in complex action scenarios.

[0051] In summary, this embodiment verifies that the present invention can not only solve the problems of traditional scoring systems lacking subjective judgment and being unable to optimize in real time, but also achieve the gradual self-evolution of the scoring model through human-computer collaboration. It is applicable to a variety of sports training and teaching scenarios and has extremely high promotional value.

[0052] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A RLHF-based action similarity score optimization method, characterized in that, The method comprises the following steps: S1, acquiring the action video of a user through a camera as the original video data input; S2, using the OpenPose human posture recognition algorithm on the original video data to extract the coordinate information of the human key points in each frame of image to obtain the human key point data; S3, constructing the human key point data into a time sequence and inputting the same into a Transformer model to extract the time sequence dynamic feature and the global structure feature of the action to obtain the action feature representation; S4, inputting the action feature representation and the feature representation of a preset standard action into a similarity calculation module together to calculate the matching degree between the two by using the cosine similarity measurement method to obtain the preliminary similarity score; S5, based on the preliminary similarity score, introducing a PPO algorithm and a human feedback mechanism, collecting the feedback information of multiple users on the score, constructing the feedback signal as a reward, updating the parameters of the human feedback optimization model, and outputting the optimized action similarity score result; S6, associating and displaying the optimized similarity score with the original action video to output the action score result for the user to refer.

2. The RLHF-based action similarity score optimization method of claim 1, wherein, The S1 specifically comprises: S11, using a camera to shoot a motion video or input an existing action video; S12, decomposing the video into continuous frames, usually extracting at 30 frames per second; S13, adjusting the size and normalizing each frame, and storing the processed video data in the form of frame sequence.

3. The RLHF-based action similarity score optimization method of claim 1, wherein, The S2 specifically comprises: S21, performing human key point detection on the video data; S22, using OpenPose to perform human key point detection on each frame of image; S23, extracting 18 key points of the human body, including the head, the trunk and the limbs; S24, performing normalization processing on the key point coordinates of each frame to ensure that videos of different resolutions can adapt to a unified model, using the coordinates of the key points as reference points to relativize the coordinates of all key points and eliminate the differences in scale and position; S25, if the video lengths are different, performing dynamic time warping or linear interpolation for alignment; S26, splicing the normalized data into a time sequence: ； wherein is a time series, T is the number of video frames.

4. The RLHF-based action similarity score optimization method of claim 1, wherein, The S3 specifically comprises: S31, inputting the time sequence data into a transformer model for training; S32, pre-processing and embedding the data, adding position encoding to each frame, and inputting the sequence into an embedding layer to map it to a higher-dimensional feature space; S33, after the position encoding and embedding processing, a feature sequence with time sequence information is obtained:

5. wherein, is a feature sequence with timing information, is a real number field, T is the number of video frames, N is the number of human key points, and E is the dimension of each key point; S34, inputting the feature sequence with time sequence information into the Transformer model, and calculating through multiple encoder layers to obtain the output: ； wherein, is the output feature calculated by the Transformer, and the attention mechanism in the Transformer is: ； wherein, is an attention weight matrix, normalizing the computed attention scores, is a query matrix, is a key matrix, is a transpose of the key matrix, is a feature dimension; S35, performing global average pooling on the output of the Transformer to obtain the aggregated action feature: ； wherein, is the time step, is the output feature of the th time step.

6. The RLHF-based action similarity score optimization method of claim 1, wherein, The S4 specifically comprises: S41, calculating the similarity between the standard action and the action to be evaluated in the output of the Transformer model by using the cosine similarity: ； wherein, is a similarity between the standard motion and the motion to be evaluated, is a feature representation of the standard motion, is a feature representation of the motion to be evaluated, is a dot product of the standard motion and the motion to be evaluated, is a Euclidean norm of the standard motion, is a Euclidean norm of the motion to be evaluated; S42, mapping the cosine similarity from [-1, 1] to [0, 1] or a score range of [0, 100] through linear transformation and outputting a preliminary similarity score.

7. The RLHF-based action similarity score optimization method of claim 1, wherein, The S5 specifically comprises: S51, after obtaining the action similarity score, the score is optimized through RLHF, specifically using PPO algorithm for optimization, and combining human feedback to optimize the model output; S52, the reward function is designed to optimize the preference of the model score according to human feedback, and for each pair of standard action and action to be evaluated, the model calculates the reward according to human feedback, if the score given by the model is close to the true score, the reward is larger, if the score given by the model deviates from the true score, the reward is smaller, even negative reward:

8. wherein, is the average reward value obtained by final calculation, is the number of users participating in feedback, is the reward value of the first feedback, is the feedback label of the first user, is the score result of the model on the first action; S53, PPO improves the traditional policy optimization algorithm by introducing truncated advantage, the objective function is divided into two parts, the goal of PPO is to maximize the following policy objective function: E ; wherein, is a policy update target, is a parameter of a policy network, is an expected value, is a ratio of policy probabilities, is a advantage function, is a policy ratio clipping; S54, in order to let the strategy learn the reward signal, the value function also needs to be optimized:

9. wherein, the current value function has a predicted value for a state of the target value, as a target value; S55, the final loss function is the weighted sum of the following parts: - + ； where, is the final loss function, is the entropy of the policy, is a weight hyperparameter balancing the parts. S56, the policy update uses a stochastic gradient descent method to update the parameters : + ； where is the loss function with respect to the parameters gradient, is the learning rate.

10. The RLHF-based action similarity score optimization method of claim 1, wherein, The S6 specifically comprises: S61, using a graphical interface to display the similarity score and action evaluation result; S62, providing visual feedback, comparing key points and playing video synchronously; S63, updating the similarity result in real time according to the user score, guiding the user to adjust the action.

Images