A joint angle prediction method based on spatiotemporal dual-stream Transformer
By constructing a spatiotemporal dual-stream Transformer model, the problem of large joint angle prediction errors in existing technologies has been solved, achieving stable and accurate prediction of lower limb joint angles, and providing technical support for lower limb rehabilitation exoskeleton control.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHANGCHUN UNIV OF TECH
- Filing Date
- 2026-05-14
- Publication Date
- 2026-07-17
AI Technical Summary
Existing technologies for predicting lower limb joint angles using surface electromyography signals struggle to effectively capture the spatial coordination and temporal dependence characteristics between multiple muscle channels, resulting in large errors between predicted and actual values. Furthermore, the lack of constraints on the continuity of movement makes it difficult to achieve stable joint angle predictions.
A spatiotemporal dual-stream Transformer model is adopted. The electromyography signal is smoothed through multi-level filtering, and temporal features are extracted. Spatial and temporal attention branches are constructed. Combined with multi-head attention mechanism and cross-gating fusion strategy, feature weights are adaptively adjusted. A smoothing constraint loss function is introduced to suppress jitter and achieve stable prediction of joint angles.
It improves the accuracy and stability of joint angle prediction, reduces model complexity, and provides effective technical support for lower limb rehabilitation exoskeleton control.
Smart Images
Figure CN122182067B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and rehabilitation robots, and specifically to a joint angle prediction method based on spatiotemporal dual-stream Transformer. Background Technology
[0002] Lower limb rehabilitation exoskeletons require control modes to be set based on the patient's movement intentions. Surface electromyography (EMG) signals are a reliable data source for identifying movement intentions. Generated 30-150ms before limb movement, they reflect muscle activity levels and have a strong nonlinear correlation with hip, knee, and ankle joint movements in the lower limbs. Joint angles can be predicted in advance based on these signals. Traditional angle prediction methods mostly employ machine learning, relying on artificial time-domain features, and can only fit simple linear patterns, with limited generalization ability when muscles are moving in synergistic ways. Recurrent neural networks can initially capture the temporal dependencies of surface EMG signals, but feature extraction is limited, and nonlinear expression capabilities remain insufficient.
[0003] Lower limb movements exhibit significant spatiotemporal coupling characteristics. Multiple muscles show spatial correlation features of coordinated activation and mutual constraints, and electromyographic signals exhibit regular temporal evolution with the progress of movement, displaying time-dependent characteristics. However, current network architectures often focus on mining temporal information, neglecting the spatial coordination relationships between multiple muscle channels. Some studies have integrated spatiotemporal features, but they cannot adaptively adjust the weight ratio of the two types of features, resulting in significant errors between predicted and actual values. Furthermore, conventional loss functions only constrain prediction errors, lacking constraints on the continuity of movement, and fail to accurately reflect the smooth characteristics of natural human movement. Summary of the Invention
[0004] This invention proposes a joint angle prediction method based on a spatiotemporal dual-stream Transformer, aiming to improve model accuracy, ensure prediction stability, and reduce model complexity. First, surface electromyography (EMG) signals of the subject's lower limbs are collected. The raw EMG signals are preprocessed, then feature engineering is introduced to construct a spatiotemporal dual-stream Transformer model, finally outputting the joint angle. The method is implemented through the following technical solutions:
[0005] Step 1: Collect electromyographic signals from the lower limbs of the subject.
[0006] Step 2: Preprocess the raw surface electromyography signals.
[0007] Step 2.1: Use a power frequency notch filter to remove interference from power line coupling.
[0008] Step 2.2: To address baseline drift caused by motion artifacts and changes in skin impedance, a high-pass filter is used to smooth the signal, and the absolute value is taken to convert the AC electromyography signal into a linear envelope.
[0009] Step 2.3: Perform low-pass filtering to remove high-frequency ripples in the envelope signal and extract the muscle activation gradient.
[0010] Step 3: Introduce feature engineering to extract four temporal features: mean absolute value Root mean square ,variance Waveform length The specific expression is as follows:
[0011] ,
[0012] ,
[0013] ,
[0014] ,
[0015] in, For channel indexing, This indicates the total number of sampling points within the window. It is the amplitude of the electromyographic signal after preprocessing.
[0016] Step 4: Construct a spatiotemporal dual-stream Transformer model to predict joint angles.
[0017] Step 4.1: Compress the feature dimension, reduce redundant information, and introduce nonlinear transformations to enhance muscle feature expression:
[0018] ,
[0019] in, For input features, , This represents the weight matrix for each layer. It is a non-linear activation function. , For learnable bias, This indicates the compressed features.
[0020] Step 4.2: Map the compressed features to model dimensions using a feature embedding layer:
[0021] ,
[0022] in, To embed the weight matrix, Indicates embedding bias. It is the input to the Transformer model.
[0023] Step 4.3: Construct a multi-head attention mechanism to capture feature dependencies comprehensively:
[0024] ,
[0025] ,
[0026] ,
[0027] in, It is attention weight. , , The matrix is arranged in the following order: Query matrix, Key matrix, and Value matrix. Represents the normalization function. It is the attention score. Scaling factor Indicates the first Head output, , , The order is the first Query projection, Key projection, and Value projection in the header. For splicing operations, Indicates output projection. It is the output of the multi-head attention mechanism.
[0028] Step 4.4: Use Spatial Flow Transformer to capture the synergistic relationships between muscle channels:
[0029] ,
[0030] ,
[0031] in, For the output of the cross-channel attention mechanism, Indicates random inactivation. It is the result of residual connection. For layer normalization, This indicates spatial stream output.
[0032] Step 4.5: Learn the dynamic evolution of the time series using the time-stream Transformer:
[0033] ,
[0034] ,
[0035] in, For the output of the cross-time attention mechanism, It is the result of residual connection. This indicates the output of the time stream.
[0036] Step 4.6: Perform cross-gating fusion and adaptively and dynamically allocate dual-stream weights:
[0037] ,
[0038] ,
[0039] in, It is the Sigmoid activation function. Represents the gate weight matrix. It is a gated bias. For spatiotemporal fusion weights, This indicates the result of gating weighted fusion.
[0040] Step 4.7: Map the final moment features to 3D joint angles:
[0041] ,
[0042] in, Features of the last time step of the fusion result, This represents the output weight matrix. It is the output bias. The angles of the hip, knee, and ankle joints predicted by the model.
[0043] Step 5: To suppress prediction jitter, in the loss function Introducing smoothing constraints The specific expression is as follows:
[0044] ,
[0045] ,
[0046] ,
[0047] in, Indicates the mean square error loss. For the sample size, It is the true value of the angle. , These represent the changes in the predicted angle and the actual angle between adjacent time points, respectively. For time-series smoothing loss, It is the smoothing coefficient. This represents the total loss function.
[0048] Step 6: Use root mean square error Coefficient of determination Unbiased error standard deviation The specific expression for evaluating the model's prediction accuracy is as follows:
[0049] ,
[0050] ,
[0051] ,
[0052] in, The average of all true angles. Indicates the prediction error. It is the average of all errors.
[0053] This invention smooths lower limb surface electromyography (EMG) signals through multi-level filtering and extracts temporal features. A spatiotemporal dual-stream Transformer network structure is constructed, with spatial and temporal attention branches set to learn the spatial coordination relationships of muscles and the temporal evolution of gait. A multi-head attention mechanism is introduced to enhance multi-scale feature capture capabilities, and a cross-gating fusion strategy is employed to achieve adaptive weighted fusion of dual-stream features. By combining mean squared error loss and temporal smoothing constraints, output jitter is suppressed while ensuring prediction accuracy, ultimately achieving continuous and stable angle prediction for multiple joints (hip, knee, and ankle), providing effective technical support for lower limb rehabilitation exoskeleton control. Attached Figure Description
[0054] Figure 1 This is an overall flowchart of an embodiment of the present invention. Detailed Implementation
[0055] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.
[0056] Figure 1 This is a flowchart of an embodiment. This embodiment provides a joint angle prediction method based on spatiotemporal dual-stream Transformer. The specific process includes: collecting surface electromyography (EMG) signals of the subject's lower limbs; preprocessing the raw surface EMG signals; introducing feature engineering to extract four temporal features; constructing a spatiotemporal dual-stream Transformer model to predict joint angles; and evaluating the model accuracy using root mean square error, coefficient of determination, and unbiased error standard deviation.
[0057] The specific implementation steps of a joint angle prediction method based on spatiotemporal dual-stream Transformer are as follows:
[0058] Step 1: Collect 7-channel lower limb surface electromyography signals and true values of the angles of the hip, knee and ankle joints while the subject is walking on a treadmill.
[0059] Step 2: Preprocess the raw surface electromyography signals.
[0060] Step 2.1: Use a power frequency notch filter to remove interference from power line coupling.
[0061] Step 2.2: To address baseline drift caused by motion artifacts and changes in skin impedance, a second-order high-pass Butterworth filter is used to smooth the signal, and the absolute value is taken to convert the AC electromyography signal into a linear envelope.
[0062] Step 2.3: Perform low-pass filtering to remove high-frequency ripples in the envelope signal and extract the muscle activation gradient.
[0063] Step 3: Introduce feature engineering to extract four temporal features: mean absolute value Root mean square ,variance Waveform length The specific expression is as follows:
[0064] ,
[0065] ,
[0066] ,
[0067] ,
[0068] in, For channel indexing, This indicates the total number of sampling points within the window. It is the amplitude of the electromyographic signal after preprocessing.
[0069] Step 4: Construct a spatiotemporal dual-stream Transformer model to predict joint angles.
[0070] Step 4.1: Two fully connected layers are used to compress the feature dimension, reduce redundant information, and a non-linear transformation is introduced to enhance the expression of muscle features.
[0071] ,
[0072] in, For input features, , This represents the weight matrix for each layer. It is a non-linear activation function. , For learnable bias, This indicates the compressed features.
[0073] Step 4.2: Map the compressed 16-dimensional features to 64 dimensions using a feature embedding layer to match the Transformer model dimensions.
[0074] ,
[0075] in, To embed the weight matrix, Indicates embedding bias. It is the input to the Transformer model.
[0076] Step 4.3: Construct a multi-head attention mechanism to capture feature dependencies comprehensively:
[0077] ,
[0078] ,
[0079] ,
[0080] in, It is attention weight. , , The matrix is arranged in the following order: Query matrix, Key matrix, and Value matrix. Represents the normalization function. It is the attention score. Scaling factor Indicates the first Head output, , , The order is the first Query projection, Key projection, and Value projection in the header. For splicing operations, Indicates output projection. It is the output of the multi-head attention mechanism.
[0081] Step 4.4: Use Spatial Flow Transformer to capture the synergistic relationships between muscle channels:
[0082] ,
[0083] ,
[0084] in, For the output of the cross-channel attention mechanism, Indicates random inactivation. It is the result of residual connection. For layer normalization, This indicates spatial stream output.
[0085] Step 4.5: Learn the dynamic evolution of the time series using the time-stream Transformer:
[0086] ,
[0087] ,
[0088] in, For the output of the cross-time attention mechanism, It is the result of residual connection. This indicates the output of the time stream.
[0089] Step 4.6: Perform cross-gating fusion and adaptively and dynamically allocate dual-stream weights:
[0090] ,
[0091] ,
[0092] in, It is the Sigmoid activation function. Represents the gate weight matrix. It is a gated bias. For spatiotemporal fusion weights, This indicates the result of gating weighted fusion.
[0093] Step 4.7: Map the final moment features to 3D joint angles:
[0094] ,
[0095] in, Features of the last time step of the fusion result, This represents the output weight matrix. It is the output bias. The angles of the hip, knee, and ankle joints predicted by the model.
[0096] Step 5: To suppress prediction jitter, in the loss function Introducing smoothing constraints The specific expression is as follows:
[0097] ,
[0098] ,
[0099] ,
[0100] in, Indicates the mean square error loss. For the sample size, It is the true value of the angle. , These represent the changes in the predicted angle and the actual angle between adjacent time points, respectively. For time-series smoothing loss, It is the smoothing coefficient. This represents the total loss function.
[0101] Step 6: Use root mean square error Coefficient of determination Unbiased error standard deviation The specific expression for evaluating the model's prediction accuracy is as follows:
[0102] ,
[0103] ,
[0104] ,
[0105] in, The average of all true angles. Indicates the prediction error. It is the average of all errors.
Claims
1. A joint angle prediction method based on spatiotemporal dual-stream Transformer, characterized in that, Includes the following steps: Step 1: Collect electromyographic signals from the lower limbs of the subject; Step 2: Preprocess the raw surface electromyography signals; Step 3: Introduce feature engineering to extract four temporal features: mean absolute value Root mean square ,variance Waveform length The specific expression is as follows: ， ， ， ， in, For channel indexing, This indicates the total number of sampling points within the window. It is the amplitude of the electromyographic signal after preprocessing; Step 4: Construct a spatiotemporal dual-stream Transformer model to predict joint angles; Step 5: Introduce a smoothing constraint into the loss function to suppress prediction jitter; Step 6: Use root mean square error, coefficient of determination, and unbiased error standard deviation to evaluate the accuracy of the model.
2. The joint angle prediction method based on spatiotemporal dual-stream Transformer according to claim 1, characterized in that, The construction of the spatiotemporal dual-stream Transformer model described in step 4 is specifically implemented according to the following steps: Step 4.1: Compress the feature dimension using two fully connected layers: ， in, For input features, , This represents the weight matrix for each layer. It is a non-linear activation function. , For learnable bias, Indicates the compressed features; Step 4.2: Match the Transformer model dimension through the feature embedding layer: ， in, To embed the weight matrix, Indicates embedding bias. It is the input to the Transformer model; Step 4.3: Construct a multi-head attention mechanism: ， in, Indicates the first Head output, For splicing operations, It is the output projection. This represents the output of the multi-head attention mechanism; Step 4.4: Use Spatial Flow Transformer to capture the synergistic relationships between muscle channels: ， ， in, For the output of the cross-channel attention mechanism, Indicates random inactivation. It is the result of residual connection. For layer normalization, Indicates spatial stream output; Step 4.5: Learn the dynamic evolution of the time series using the time-stream Transformer: ， ， in, For the output of the cross-time attention mechanism, It is the result of residual connection. Indicates time-stream output; Step 4.6: Perform cross-gating fusion: ， ， in, It is the Sigmoid activation function. Represents the gate weight matrix. It is a gated bias. For spatiotemporal fusion weights, This indicates the result of gated weighted fusion; Step 4.7: Map the final moment features to 3D joint angles: ， in, Features of the last time step of the fusion result, This represents the output weight matrix. It is the output bias. The angles of the hip, knee, and ankle joints predicted by the model.