Motion-aware and dual-stream spatio-temporal graph convolution based action quality assessment method

Abstract

The motion quality evaluation method based on motion perception and double-flow space-time graph convolution relates to the motion quality evaluation field, solves the problems that the existing rehabilitation motion quality evaluation method is difficult to accurately model the cooperation between joints, the fixed joint grouping strategy is not flexible enough, the model has insufficient recognition of the detail differences of complex rehabilitation motions, and the reliability of the evaluation result is affected, a dynamic joint grouping strategy based on motion amplitude driving is provided, a double-flow STGCN architecture is designed, the position and orientation information of the joints are processed respectively and then fused, a two-stage self-attention module SADG is developed, the inter-group time sequence mode and the intra-group spatial relationship are modeled in stages, and multi-granularity feature interaction fusion is realized. The attention mechanism introduces a constraint guided by the motion amplitude, so that the attention weight is more interpretable and has physical meaning.

Description

Technical Field

[0001] This invention relates to the field of motion quality assessment technology, specifically to a motion quality assessment method based on motion perception and two-stream spatiotemporal graph convolution. Background Technology

[0002] Action Quality Assessment (AQA) is a sophisticated action understanding task in the field of computer vision. Its goal is not only to identify action categories but also to quantify and score the execution of those actions. This technology has wide applications in medical rehabilitation, sports training, and skills assessment. Particularly in rehabilitation medicine, AQA can provide patients with quantitative indicators reflecting their rehabilitation progress, ensuring the correctness of action execution and reducing the risk of secondary injury.

[0003] In recent years, advancements in pose estimation and sensor technology have enabled the acquisition of skeletal data reflecting human movement, making automated assessment possible. Existing technologies have seen some research attempting to use deep learning models for AQA, such as convolutional neural networks (CNNs) or recurrent neural networks (RNNs) to process video or sequence data. However, these methods often fail to fully utilize the spatial connectivity information between the joints of the human body. Ignoring skeletal topology makes it difficult for models to accurately capture motion details and joint coordination changes, thus limiting assessment accuracy.

[0004] To better utilize skeletal structural information, Graph Convolutional Networks (GCNs) have been introduced into the field of motion analysis. Among them, the STGCN model proposed in existing technologies extracts skeletal features in spatial and temporal dimensions through graph convolution, achieving significant results in motion recognition. However, traditional STGCNs use fixed joint connection topologies, essentially predefining skeletal joint relationships, which are not easily adapted to changes in the collaborative relationships of non-directly adjacent joints under different movements. In rehabilitation movements, different limbs may exhibit dynamic collaboration at different stages, and a fixed topology alone cannot capture these high-order relationships. Furthermore, some studies have attempted to adaptively learn the connection weights between joints, such as the two-stream adaptive GCN method, which allows the network to adjust the skeletal graph structure itself. However, fully adaptive grouping lacks prior knowledge guidance, potentially leading to unreasonable connections that are difficult to physically interpret and generalize.

[0005] Other areas for improvement include multimodal information fusion and spatiotemporal attention mechanisms. For example, existing technologies improve scoring accuracy by fusing audio and video information; the proposed Tubular Self-Attention Network (TSA-Net) performs attention computation within local spatiotemporal regions to aggregate context. These approaches improve evaluation performance to some extent, but they still do not fundamentally solve the problem of fine-grained modeling of skeletal joint synergy. Existing methods struggle to accurately model inter-joint synergy; fixed joint grouping strategies are not flexible enough, while fully adaptive grouping lacks prior knowledge constraints. This results in insufficient model recognition of subtle differences in complex rehabilitation movements, affecting the reliability of evaluation results. Summary of the Invention

[0006] To overcome the limitations of existing rehabilitation movement quality assessment methods in accurately modeling inter-joint synergies, the inflexibility of fixed joint grouping strategies, and the lack of prior knowledge constraints in fully adaptive grouping, which leads to insufficient model recognition of subtle differences in complex rehabilitation movements and affects the reliability of assessment results, this invention proposes a movement quality assessment method based on motion perception and dual-stream spatiotemporal graph convolution.

[0007] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0008] Option 1: This invention proposes a motion quality assessment method based on motion perception and two-stream spatiotemporal graph convolution, the method comprising the following steps:

[0009] Step 1: Obtain the skeletal sequence data of the rehabilitation movements to be evaluated. For each joint in each time step, extract the position data stream and orientation data stream.

[0010] Step 2: Construct a two-stream spatiotemporal convolutional network. Input the position data stream and orientation data stream extracted in Step 1 into two parallel STGCN branches respectively. Each branch extracts spatiotemporal features through hierarchical spatial graph convolution and temporal convolution modules. Concatenate the features output by the two parallel STGCN branches in the channel dimension to obtain a spatiotemporal feature tensor that fuses position and orientation information.

[0011] Step 3: Design a motion-aware dynamic grouping module to automatically generate grouping masks based on the joint motion amplitude; calculate the standard deviation of the input feature tensor in the time dimension to obtain a matrix reflecting the motion amplitude of each joint; input the matrix into a learnable network containing two layers of linear transformation, and finally... The function generates a probability distribution mask α that maps J joints to G functional groups;

[0012] Step 4: Utilize a multi-head self-attention mechanism to model the spatial feature associations within groups and the spatiotemporal dependencies between groups to obtain the fused features;

[0013] Step 5: Remap the fused features from Step 4 back to the original joint space so that the network layer processes the standard joint feature structure and outputs the feature input regression head.

[0014] Step 6: Construct a regression head network. Input the features output from Step 5 into the regression head, map the high-dimensional spatiotemporal features into a one-dimensional action quality score, and output the final predicted score to evaluate the quality of rehabilitation actions.

[0015] Furthermore, a preferred embodiment is provided in which the position data stream in step 1 includes the three-dimensional coordinates of the joint in the Cartesian coordinate system. The direction data stream contains the direction quaternions of the joints. It is used to characterize the rotation and orientation information of a joint.

[0016] Furthermore, a preferred embodiment is provided, wherein the method for obtaining the spatiotemporal feature tensor with fused position and orientation information in step 2 is as follows:

[0017] Step 2.1: In the spatial convolution stage, perform a 1x1 convolution operation on the joint feature map of each frame to extract local features joint by joint.

[0018] Step 2.2: In the temporal convolution stage, two-dimensional convolution is used to extract dynamic features along the time dimension;

[0019] Step 2.3: The local features extracted in Step 2.1 and the dynamic features proposed in Step 2.2 are merged along the channel dimension, i.e., feature splicing operation.

[0020] Furthermore, a preferred embodiment is provided, wherein the method for calculating the standard deviation of the input feature tensor in the time dimension in step 3 to obtain the matrix reflecting the motion amplitude of each joint is as follows:

[0021] ,

[0022] in, This represents the feature value of the b-th sample at time t, the j-th joint, and channel c. This is the average eigenvalue of the joint over the entire sequence.

[0023] Furthermore, a preferred embodiment is provided in which, in step 3, the matrix is ​​input into a learnable network containing two layers of linear transformation, and finally... The function that generates the probability distribution mask α that maps J joints to G functional groups is as follows:

[0024] ,

[0025] in and For trainable weight matrix, and For bias vectors, express Activation function.

[0026] Furthermore, a preferred embodiment is provided, in which step 4 utilizes a multi-head self-attention mechanism to model the spatial feature associations within groups and the spatiotemporal dependencies between groups, and the method for obtaining the fused features is as follows:

[0027] Calculate the attention weights between groups:

[0028] ,

[0029] The similarity between two sets of features is measured using the dot product method, with respect to k. Normalization yields an inverse attention matrix, and the value matrix V is weighted and summed using weights A to obtain the fused group feature representation:

[0030] ,

[0031] The outputs of each attention head are concatenated along the h-dimensional axis to obtain This refers to the functional group features that integrate cross-group information.

[0032] Furthermore, in a preferred embodiment, step 5 further includes using a dynamic grouping mask. The step of assigning group-level features to each joint is...

[0033] ,

[0034] in, This represents the characteristics of group g after fusion.

[0035] Furthermore, a preferred embodiment is provided, wherein in step 6, the regression head network is constructed by inputting the features output in step 5 into the regression head, mapping the high-dimensional spatiotemporal features into a one-dimensional action quality score, and outputting the final predicted score using the following method:

[0036]

[0037] Where N is the training batch size. For predicting scores, For genuine ratings, For predicting scores Compared to real ratings The error between them Optimize network parameters for backpropagation

[0038] Option 2: A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in Option 1.

[0039] Option 3: A computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the method described in Option 1.

[0040] The advantages of this invention are:

[0041] The motion quality assessment method based on motion perception and two-stream spatiotemporal graph convolution described in this invention significantly improves the accuracy and robustness of the model assessment, and can meticulously distinguish subtle differences in the quality of motion completion. Experimental results show that compared with existing methods, this invention reduces the mean absolute error by approximately 26.5%. On publicly available rehabilitation motion datasets, this invention achieves state-of-the-art scoring results on most assessment items. For example, for the KIMORE rehabilitation training dataset, this method has lower scoring errors and higher correlations in most motion categories; it also achieves performance comparable to the current state-of-the-art methods on the UI-PRMD dataset. Meanwhile, ablation experiments verify the necessity of dynamic grouping and two-stream feature design: removing the dynamic grouping module significantly degrades model performance; using only single-stream (without joint orientation information) nearly doubles the scoring error, making it difficult to capture the subtle differences in complex movements. In other words, the modules of this invention complement each other, jointly ensuring the accuracy and robustness of the assessment.

[0042] Due to its modular design and attention-weighted approach, the method described in this invention is adaptable to sequences of different lengths, insensitive to occasional abnormal tremors of individual joints, and exhibits good robustness. In other words, this invention can help automatically quantify the quality of patient movements during rehabilitation treatment, providing objective data support and reducing reliance on human experience.

[0043] This invention is also applied to motion quality assessment methods based on motion perception and dual-stream spatiotemporal convolution in medical rehabilitation monitoring and sports training scenarios. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the architecture of the motion quality assessment method based on motion perception and two-stream spatiotemporal graph convolution described in Implementation Method 1.

[0045] Figure 2 This is a visual diagram illustrating the joint motion amplitude and grouping probability as described in Implementation Method 1.

[0046] Figure 3This is a visualization diagram of the attention weights for different rehabilitation movements on the human skeleton as described in Implementation Method 1.

[0047] Figure 4 This is a scatter plot comparing the predicted scores of the model of the present invention described in Implementation Method 1 on the test set with the manually labeled real scores. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0049] Implementation Method 1: This implementation method proposes a motion quality assessment method based on motion perception and two-stream spatiotemporal graph convolution. It introduces motion amplitude as prior knowledge and fuses joint position and orientation features through a two-stream architecture, achieving more accurate motion quality scoring that better aligns with clinical assessment logic. The method specifically includes the following steps:

[0050] Step 1: Data Acquisition and Preprocessing: Acquire the skeletal sequence data of the rehabilitation movements to be evaluated. For each joint at each time step, extract the position data stream and orientation data stream. The position data stream contains the three-dimensional coordinates of the joint in the Cartesian coordinate system. The direction data stream contains the direction quaternions of the joints. It is used to characterize the rotation and orientation information of a joint.

[0051] Step 2: Dual-Stream Spatiotemporal Feature Extraction: Construct a Dual-Stream Spatiotemporal Convolutional Network (Dual-Stream STGCN). The position and orientation data streams are input into two parallel STGCN branches. Each branch extracts spatiotemporal features through hierarchical spatial graph convolution and temporal convolution modules. The features output from the two branches are concatenated along the channel dimension to obtain a spatiotemporal feature tensor that integrates position and orientation information.

[0052] First, in the spatial convolution stage, a 1x1 convolution operation is performed on the joint feature map of each frame to extract local features joint by joint. This 1x1 convolution is applied along the joint dimension, which is equivalent to performing a linear transformation on the feature vector of each joint without mixing features from different joints. This design can be seen as learning a nonlinear mapping for each joint independently, maintaining the independence of joint features and laying the foundation for subsequent grouping. Next, in the temporal convolution stage, a two-dimensional convolution is used to extract dynamic features along the temporal dimension. The optimal convolution kernel size is 3x1 (covering a 3-frame time window), with a stride of 2, performing temporal downsampling, and padding with one frame at temporal boundaries to ensure temporal alignment. Temporal convolution kernel Sliding along the time axis, the local features of every 3 frames are mapped to the output, represented as:

[0053] ,

[0054] in, This is the output of the spatial convolution stage, where T / 2 represents the time length after a stride of 2 downsampling. J represents the number of joints, and C represents the number of channels. After convolution, the input and output features are fused using residual connections, then added element-wise with the temporal convolution result, and then processed... Activation, denoted as:

[0055] .

[0056] In the actual network, each branch is connected in series with three STGCN modules, gradually increasing the number of channels to learn richer features: for example, starting with 64 channels, it increases to 256 after three layers. The last two STGCN modules perform downsampling with a stride of 2 in the time dimension, so the output sequence length of the original sequence of length T becomes T' = T / 4. After the above processing, the position flow and direction flow each obtain feature tensors of size T' x J x 256. and The outputs of the two branches are then merged along the channel dimension, which is the feature concatenation operation.

[0057] Step 3: Dynamic Grouping Mask Generation Based on Motion Amplitude: Design a Motion-Aware Dynamic Grouping (SADG) module to automatically generate grouping masks based on the motion amplitude of the joints. Calculate the standard deviation of the input feature tensor in the time dimension to obtain a matrix reflecting the motion amplitude of each joint. Input the matrix into a learnable network containing two layers of linear transformation (including...). Activation function), and finally through The function generates a probability distribution mask α that maps J joints to G functional groups. This process adaptively clusters functionally related joints using learnable parameters. First, the joint's motion amplitude metric M represents the range of change of that joint over time, obtained by calculating the temporal standard deviation of the features.

[0058] ,

[0059] in, This represents the feature value of the b-th sample at time t, the j-th joint, and channel c. This is the average eigenvalue of the joint over the entire sequence, therefore This reflects the range of motion or changes of joint j in that channel. When a joint moves frequently and with large amplitude during an action, its corresponding... The value will be larger, and vice versa.

[0060] Next, the motion intensity M of each joint is mapped to a preset functional group, and the motion amplitude of the joint is transformed into the membership probability distribution of each group through two fully connected layers:

[0061] ,

[0062] in, and For trainable weight matrix, and For bias vectors, express Activation function It operates on the group dimension g. Based on the mask... The fused feature X is weighted and aggregated to calculate the group features of each functional group. Specifically, for each time frame t and each group g, the joint features contained in that group are weighted and summed:

[0063] ,

[0064] in, This represents the probability weight that the b-th sample joint j belongs to group g. The above formula projects the original J-joint dimension features to G-functional group dimensions, achieving intra-group aggregation of joint features.

[0065] Step 4: Intra-group and inter-group attention interaction: Apply multi-head self-attention mechanism to group features to model the spatial feature association within groups and the spatiotemporal dependency between groups.

[0066] During the inter-group attention interaction phase, information is shared among different functional groups to model global collaborative relationships. Specifically, the group features X are transformed linearly to obtain the query matrix Q, key matrix K, and value matrix V, with the following shape: Where H is the number of attention heads, and D = C / H is the subspace dimension of each head. For each time t, group g, and attention head h, the attention weights between groups are calculated:

[0067] ,

[0068] The above formula calculates the relevance weight of group g to group k. The similarity between two sets of features is measured using the dot product method, and then... Scaling is used to prevent the values ​​from becoming too large, and finally, k is adjusted. Normalization yields an attention matrix that reflects the intensity of information interaction between different functional groups at each time step. Subsequently, weighted summation of the value matrix V using weights A yields the fused group feature representation:

[0069]

[0070] The outputs of each attention head are concatenated along the h-dimensional axis to obtain This refers to the functional group features that integrate cross-group information.

[0071] Step 5: Feature backprojection and residual fusion. The fused features... The joint space is remapped back to the original joint space, allowing subsequent network layers to still handle standard joint feature structures. This is achieved through the reverse operation of step three, using dynamically grouped masks. Assign group-level features to each joint:

[0072] ,

[0073] in, This represents the features of the fused group g. The backprojection operation weights and distributes the group features back to the joint nodes based on the probability that each joint belongs to each functional group, thus restoring the output tensor. The skeletal structure is shown. This ensures that subsequent model processing can still be performed at the standard joint dimension. Simultaneously, residual connections are established, adding the initial fused feature X (the dual-stream spliced ​​feature output from step two) element-wise to the current reprojected feature. This residual fusion combines the original joint features with the attention-enhanced features, preserving basic motion information while overlaying higher-order relationships captured by grouped attention, thus improving training stability and feature representation ability.

[0074] Step Six: Action Quality Score Regression: Construct a Regression Head network. Input the features output from Step Five into the regression head. The regression head consists of fully connected layers (Linear), The activation function maps high-dimensional spatiotemporal features to one-dimensional movement quality scores. The final predicted score is output to evaluate the quality of rehabilitation movements. During training, a mean squared error loss function is used to adjust the predicted scores. Compared to real ratings The error between them is denoted as Optimize network parameters through backpropagation The formula is expressed as follows:

[0075] ,

[0076] Where N is the training batch size. After sufficient training, the model can output a stable and reliable quality score based on the input skeletal motion sequence.

[0077] In summary, the method of this invention executes the above steps sequentially to complete the quality assessment of rehabilitation movement sequences. Its working principle lies in extracting positional and directional features through a two-stream network to obtain a more comprehensive movement description; introducing clinical experience based on dynamic grouping of joint movement intensity to categorize relevant joints, improving flexibility while incorporating prior knowledge; and finally, using a two-stage self-attention mechanism to fuse features within and between groups, fully capturing the local coordination and global coordination patterns during movement execution, and outputting a quantitative scoring result. The entire algorithm flow is as follows: Figure 1 As shown.

[0078] This invention proposes a dynamic joint grouping strategy driven by motion amplitude, which differs from fixed or unconstrained grouping. This allows the model to adaptively adjust joint relationships and ensures the grouping is biologically sound. Secondly, a dual-stream STGCN architecture is designed, processing joint position and orientation information separately before fusion, capturing richer posture change features than single-stream models. Thirdly, a two-stage self-attention module (SADG) is developed, modeling intra-group temporal patterns and inter-group spatial relationships in stages, achieving multi-granular feature interaction fusion. This attention mechanism introduces constraints guided by motion amplitude, making attention weights more interpretable and physically meaningful. The combination of these key innovations enables this invention to quantitatively assess the quality of rehabilitation movements with high precision, overcoming the shortcomings of existing technologies in dynamic collaborative modeling.

[0079] See Figure 1 As shown, this invention performs quality scoring on input human skeletal motion sequences through four stages: feature extraction, dynamic grouping, self-attention fusion, and regression output. Stage 1 uses a dual-stream STGCN to extract joint position and orientation features, and downsamples the temporal dimension layer by layer to compress information. Stage 2 adaptively divides joints into six functional groups based on joint motion intensity and generates a probability group mask. Stage 3 utilizes two layers of self-attention modules to first model temporal relationships within groups and then fuse spatial information between groups to obtain enhanced group features, which are then mapped back to the joint feature space through the group mask. Finally, the fused joint features are processed through average pooling and a fully connected regression network to output the motion quality score. Different colors and labels in the figure represent the main computational units of each stage, such as the STGCN module, linear layers, and self-attention units. "cat" indicates feature concatenation, and "+" indicates residual operations for feature addition.

[0080] Figure 2Intermediate results of the dynamic grouping mechanism of this invention are presented. The left half (Range of Motion) is a heatmap of the normalized range of motion for each body joint, with colors ranging from purple to yellow indicating increasing motion intensity. It can be seen that different joints exhibit significant differences in their range of motion during specific movements: for example, the base of the spine and shoulder joints have larger ranges of motion, while some finger joints have smaller ranges. The right half (Group Assignment Probabilities) is a heatmap of the grouping probabilities for the corresponding joints, showing the probability that each joint is assigned to one of the six functional groups (Group1~Group6). Colors range from purple to yellow indicating increasing probabilities. It is evident that joints with similar ranges of motion tend to have similar grouping probability distributions, indicating that this invention performs reasonable grouping based on joint motion characteristics. For example, the relevant joints of the arms show higher assignment probabilities in Groups 2 and 3, while the lower limb joints are mainly concentrated in Groups 4 and 5. This visualization demonstrates that the dynamic grouping mask α can effectively distinguish the functionality of different parts of the body.

[0081] Figure 3 The visualization results of attention weights for different rehabilitation movements on the human skeleton are presented. The left image corresponds to rehabilitation movement sequence 2 from the KIMORE dataset, and the right image corresponds to rehabilitation movement sequence 5 from the KIMORE dataset. Each skeleton consists of 25 joint nodes and their connecting bone segments. The color intensity of the nodes indicates the weight of that joint in the self-attention mechanism. The higher the weight, the higher the attention the evaluation model of this invention pays to that joint in the current movement, and thus the greater its contribution to the final movement quality score. By comparing the two skeleton images, it can be seen that there are significant differences in the distribution of attention in different parts of the body under different movements. The model adaptively focuses attention on the dominant force-generating joints and stability-related joints of the movement.

[0082] Figure 4A scatter plot comparing the predicted scores of the model in this invention with the manually labeled actual scores on the test set, along with the regression line, is visualized. The horizontal axis represents the actual labeled scores, and the vertical axis represents the model's predicted scores. Blue dots represent model predictions, orange dots represent actual scores, the red solid line is the fitted regression line (the equation is approximately y = 0.99x - 0.04), and the gray dashed line represents the ideal y = x line. As can be seen from the figure, most scatter points are closely distributed near the ideal line, indicating a high degree of consistency between the model predictions and the actual scores. The slope of the regression line is approximately 0.99, and the intercept is close to 0, further quantifying that the linear relationship between the model predictions and the actual values ​​is close to 1:1. This means that the evaluation method of this invention can accurately reproduce the expert scoring standards and has excellent accuracy and reliability. A few outliers in the figure (such as a few blue dots in the lower left and upper right corners) may correspond to a very small number of special samples, but overall, the correlation is good, verifying the effectiveness of this invention in the rehabilitation movement quality assessment task.

[0083] Example 1: To verify the effectiveness of the method of the present invention, experiments were conducted on two publicly available rehabilitation movement quality assessment datasets. The first dataset is KIMORE, which contains five physical therapy exercises (such as squatting and balance), each with a quality score given by a clinical expert. This dataset provides the three-dimensional coordinates of skeletal joints obtained from motion capture and the corresponding joint orientation quaternion information. Each sequence is 100 frames long after alignment and has 25 joints. For evaluation metrics, we used Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Mean Absolute Percentage Error (MAPE) to quantify the deviation between the model score and the expert score; smaller values ​​indicate higher accuracy. Table 1 shows the performance comparison of the present invention and existing methods on the KIMORE dataset.

[0084] Table 1. Performance comparison on the KIMORE dataset

[0085]

[0086] Example 2: To verify the generalization ability of our model, we validated it on another dataset: the UI-PRMD dataset, which contains 10 physical therapy exercises. This dataset provides positional data for 25 joints captured by a Kinect sensor and orientation data for 39 joints captured by a Vicon sensor. The number of joints in the positional and orientation data is inconsistent; we still achieved competitive results using only the orientation data. Tables 2 and 3 show the performance comparison of our model with existing methods on the UI-PRMD dataset, with evaluation metrics consistent with the KIMORE dataset.

[0087] Table 2. Comparison of MADs on UI-PRMD

[0088]

[0089] Table 3. Comparison of RMSE and MAPE on UI-PRMD

[0090]

[0091] Example 3: Ablation Experiment Verification of Key Modules of the Invention. Ablation experiments were conducted on the KIMORE dataset to verify the actual contributions of the dual-stream structure and the motion-aware dynamic grouping attention module to the evaluation performance. The experimental data used was the KIMORE dataset containing five typical rehabilitation movements. The training / validation / testing split was consistent with the baseline model, the random seed was fixed at 3407, the total number of training rounds was set to 100, and the learning rate was [missing information]. The batch size was 32, and all other training parameters were the same as the baseline model to ensure fairness in the comparison. Under the same dataset and parameter configuration, the following four variant models were examined:

[0092] (1) Single-flow model that retains only the position branch (removes the direction information branch);

[0093] (2) Single-flow model that retains only the directional branch (removes the position branch);

[0094] (3) Replace the motion-sensing dynamic grouping module of the present invention with conventional multi-head self-attention (MHSA);

[0095] (4) Completely remove the SADG dynamic grouping attention module and retain only the STGCN feature extraction module and the regression head.

[0096] Experimental results show that the average performance of the complete model of this invention on five types of rehabilitation movements is: MAD=0.1187, RMSE=0.2214, MAPE=0.3064. When only the position branch is used, the error increases significantly to MAD=0.2308, RMSE=0.4064, MAPE=0.8446. When only the direction branch is used, the error further increases to MAD=0.2851, RMSE=0.5342, MAPE=0.9501, indicating that a single modality is insufficient to fully express the fine-grained differences in rehabilitation movements. When the dual-stream structure is retained but the dynamic grouping module of this invention is replaced with ordinary multi-head self-attention, the model performance degrades to MAD=0.1776, RMSE=0.2742, MAPE=1.0746, indicating that unconstrained attention is difficult to effectively model clinically relevant functional joint synergies. When the SADG module was completely removed, the error further increased to MAD=0.2464, RMSE=0.3752, and MAPE=2.9218, demonstrating that dynamic grouping and group attention play a key role in reducing scoring error.

[0097] In summary, the dual-flow structure is necessary for fully utilizing joint position and orientation information, while the dynamic grouping and group self-attention mechanism based on motion amplitude significantly improves the evaluation accuracy and is one of the core technical means for this invention to achieve performance superior to existing technologies.

[0098] Those skilled in the art will understand that the above description is merely a preferred embodiment of the present invention, and the features described in the various embodiments and / or technical solutions of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. This is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0099] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended technical solutions are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the present invention. Clearly, those skilled in the art can make various modifications and variations to the present invention without departing from its spirit and scope. Thus, if these modifications and variations of the present invention fall within the scope of the present invention and its equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A motion quality assessment method based on motion perception and two-stream spatiotemporal graph convolution, characterized in that, The method includes the following steps: Step 1: Obtain the skeletal sequence data of the rehabilitation movements to be evaluated. For each joint in each time step, extract the position data stream and orientation data stream. Step 2: Construct a two-stream spatiotemporal convolutional network. Input the position data stream and orientation data stream extracted in Step 1 into two parallel STGCN branches respectively. Each branch extracts spatiotemporal features through hierarchical spatial graph convolution and temporal convolution modules. Concatenate the features output by the two parallel STGCN branches in the channel dimension to obtain a spatiotemporal feature tensor that fuses position and orientation information. Step 3: Design a motion-aware dynamic grouping module to automatically generate grouping masks based on the joint motion amplitude; calculate the standard deviation of the input feature tensor in the time dimension to obtain a matrix reflecting the motion amplitude of each joint; input the matrix into a learnable network containing two layers of linear transformation, and then... The function generates a probability distribution mask α that maps J joints to G functional groups; Step 4: Utilize a multi-head self-attention mechanism to model the spatial feature associations within groups and the spatiotemporal dependencies between groups to obtain the fused features; Step 5: Remap the fused features from Step 4 back to the original joint space so that the network layer processes the standard joint feature structure and outputs the feature input regression head. Step 6: Construct a regression head network. Input the features output from Step 5 into the regression head, map the high-dimensional spatiotemporal features into a one-dimensional action quality score, and output the final predicted score to evaluate the quality of rehabilitation actions. The position data stream extracted in step 1 contains the three-dimensional coordinates of the joints in the Cartesian coordinate system. The direction data stream contains the direction quaternions of the joints. It is used to characterize the rotation and orientation information of a joint; The method for obtaining the spatiotemporal feature tensor with fused position and orientation information in step 2 is as follows: Step 2.1: In the spatial convolution stage, perform a 1x1 convolution operation on the joint feature map of each frame to extract local features joint by joint. Step 2.2: In the temporal convolution stage, two-dimensional convolution is used to extract dynamic features along the time dimension; Step 2.3: The local features extracted in Step 2.1 and the dynamic features proposed in Step 2.2 are merged along the channel dimension, i.e., feature splicing operation.

2. The motion quality assessment method based on motion perception and two-stream spatiotemporal graph convolution according to claim 1, characterized in that, The method for calculating the standard deviation of the input feature tensor in the time dimension in step 3 to obtain the matrix reflecting the motion amplitude of each joint is as follows: in, This represents the feature value of the b-th sample at time t, the j-th joint, and channel c. This is the average eigenvalue of the joint over the entire sequence.

3. The motion quality assessment method based on motion perception and two-stream spatiotemporal graph convolution according to claim 2, characterized in that, In step 3, the matrix is ​​input into a learnable network containing two layers of linear transformations, through... The function that generates the probability distribution mask α that maps J joints to G functional groups is as follows: , in and For trainable weight matrix, and For bias vectors, express Activation function.

4. The motion quality assessment method based on motion perception and two-stream spatiotemporal graph convolution according to claim 3, characterized in that, Step 4 utilizes a multi-head self-attention mechanism to model spatial feature associations within groups and spatiotemporal dependencies between groups, obtaining the fused features as follows: Calculate the attention weights between groups: , The similarity between two sets of features is measured using the dot product method, with respect to k. Normalization yields an inverse attention matrix, and the value matrix V is weighted and summed using weights A to obtain the fused group feature representation: , The outputs of each attention head are concatenated along the h-dimensional axis to obtain This refers to the functional group features that integrate cross-group information.

5. The motion quality assessment method based on motion perception and two-stream spatiotemporal graph convolution according to claim 4, characterized in that, Step 5 also includes using dynamic grouping masks. The step of assigning group-level features to each joint is... , in, This represents the characteristics of group g after fusion.

6. The motion quality assessment method based on motion perception and two-stream spatiotemporal graph convolution according to claim 1, characterized in that, In step 6, the regression head network is constructed by inputting the features output from step 5 into the regression head, mapping the high-dimensional spatiotemporal features into a one-dimensional action quality score, and outputting the final predicted score. Where N is the training batch size. For predicting scores, For genuine ratings, For predicting scores Compared to real ratings The error between them Optimize network parameters for backpropagation.

7. A computer storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-6.

8. A computer device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the method of any one of claims 1-6.

Images