Gait prediction method based on spectrum modulation cross attention segmented loop network
By employing the Spectrum Modulated Cross-Attention Segmented Recurrent Network (SMCA-SegRNN) prediction method, the accuracy and latency issues of lower limb gait prediction under various movement modes were addressed, achieving high-precision, low-latency knee joint angle prediction and improving the real-time performance and coordination of the exoskeleton system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-12
- Publication Date
- 2026-03-10
AI Technical Summary
Existing lower limb gait prediction methods struggle to simultaneously achieve high prediction accuracy, low latency, and strong generalization ability across multiple movement modes, impacting the real-time performance and coordination of exoskeleton human-computer interaction.
A prediction method based on Spectrum Modulation Cross-Attention Segmented Recurrent Network (SMCA-SegRNN) is adopted. By combining time-domain and frequency-domain features through a spectrum adaptive query generator and a cascaded modulation decoder, high-precision and low-latency prediction of knee joint angle is achieved.
It achieves high-precision prediction in multiple motion modes, reduces inference latency, and enhances the real-time control capability and wearing experience of the exoskeleton system.
Smart Images
Figure CN121621963A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of exoskeleton robots and the field of time series prediction, in particular to a gait trajectory prediction method for lower limb assistive exoskeletons, aiming to improve the real-time performance and coordination of human-robot interaction control. BACKGROUND
[0002] Lower limb assistive exoskeleton robots are wearable human-robot coupling systems, which have important application value in the fields of rehabilitation medicine and industrial logistics. The core of realizing natural and coordinated interaction between the device and the wearer is that the system can realize real-time and accurate gait prediction. At present, lower limb gait prediction methods can be mainly divided into two categories: physical modeling-based methods and data-driven methods. The physical modeling-based method analyzes the motion intention by constructing a mathematical model between joint torque and joint dynamics. The data-driven method uses machine learning or deep learning models to directly learn the mapping relationship between sensor data and gait parameters.
[0003] Although the physical modeling-based method has clear physical interpretability, it has obvious limitations in practical application: first, it needs to be calibrated with complex personalized parameters for each user, which significantly increases the cost and complexity of system deployment; second, the model has poor generalization ability and is difficult to adapt to the gait differences between different users or the same user in different motion modes (such as walking on flat ground and climbing stairs), thus restricting its promotion in clinical and industrial scenarios.
[0004] The data-driven method effectively reduces the dependence on prior physical models and exhibits strong feature adaptive extraction capability, but existing methods still have significant limitations: on the one hand, it is difficult to achieve an effective balance between prediction accuracy and real-time performance. Existing models either have heavy computational burden and significant reasoning delay due to complex structure, which cannot meet the real-time control requirements of exoskeleton systems, or simplify the model structure in pursuit of running speed, which further leads to a decline in prediction accuracy and makes it difficult to provide accurate prediction signals to the control system; on the other hand, the generalization ability for different motion modes and speed changes is insufficient. When the wearer switches between different motion modes such as walking on flat ground and climbing stairs or the walking speed changes dynamically, the model prediction performance will significantly degrade, and it lacks cross-scene robustness, ultimately limiting its application in real application environments.
[0005] In summary, although the existing data-driven lower limb gait prediction methods reduce the dependence on prior physical models and have certain feature self-adaptive extraction capabilities, there are still key defects: it is difficult to balance the prediction accuracy and real-time performance, and the generalization ability to motion mode switching and speed change is insufficient. These problems seriously restrict the application effect of exoskeleton in actual scenarios, and significantly affect the wearing experience and safety of human-computer interaction. Therefore, it is urgent to develop a lower limb gait prediction method that has high prediction accuracy, strong generalization ability and low reasoning delay in multiple motion modes, so as to enhance the real-time performance and coordination of human-computer interaction control of lower limb exoskeleton. SUMMARY
[0006] The present application aims at the technical problem that the existing lower limb exoskeleton gait prediction method is difficult to balance the prediction accuracy, real-time performance and multi-mode adaptability, and proposes a prediction method based on Spectrum-Modulated Cross-Attention Segment Recurrent Neural Network (SMCA-SegRNN). The purpose of the present application is to realize high-precision and low-delay prediction of the future trajectory of a key one-dimensional target signal (such as the knee joint angle of the lower limb) by using multi-dimensional sensor time series input (such as IMU and angle data).
[0007] The method aims to realize the prediction through a unique architecture, which mainly includes a spectrum adaptive query generator (for fusing time domain state and frequency domain context to generate a dynamic query vector) and a cascaded modulation decoder (for driving cross-attention and feature linear modulation (FiLM) fusion using the dynamic query vector). The method includes the following steps: S1. Data collection and preprocessing: The original gait data is obtained through a designed single knee joint data acquisition device (including an inertial measurement unit and a joint angle encoder). The collected gait data is time-aligned based on timestamps, normalized based on sensor physical units and sampling accuracy (for example, IMU based on its ±8g physical range, knee joint angle based on the physical range), low-pass filtered, and finally the continuous data stream is divided into fixed-length time series samples using a sliding window to construct a gait dataset.
[0008] S2. Double-flow parallel coding: This step extracts deep features of the input signal from the time domain and the frequency domain through two parallel coding paths.
[0009] S2.1 Time domain dependent extraction (SegRNN flow): Based on the construction of Segmented Recurrent Neural Network (SegRNN), the time sequence dependence of the signal is captured to obtain the final hidden state . The specific steps are as follows: (a) Baseline normalization: extract the sampling value of the last time step in the gait data set input sample, defined as the posture baseline of the input sample.
[0010] (b) Incremental signal calculation: subtract the posture baseline from all sampling values in the input sample to eliminate the absolute posture offset, and obtain an incremental signal representing only the relative motion change relative to the window end posture.
[0011] (c) Segmented time sequence encoding: the incremental signal is input into the encoder SegRNN; the encoder divides the incremental signal into multiple fixed-length segments, and then converts it into a vector through a linear layer and an activation function, and then extracts the time sequence dependence between each segment through a gated recurrent unit GRU, to generate a time domain encoder hidden state sequence containing all time steps , the last element of the hidden state sequence as the final hidden state .
[0012] S2.2 Frequency domain context extraction (DSE flow): This path extracts the global frequency domain features of the input sample in the gait data set through a decimated differentiable spectrum encoder (DSE), to obtain a frequency domain context vector , to make up for the defect that the pure time domain model is not sensitive to periodic changes. The specific sub-steps are as follows: (a) Key signal extraction: according to domain knowledge, a characteristic element is selected to represent the periodicity of gait as the main signal through correlation screening in the input sample, i.e. the knee joint angle signal, which can clearly represent the gait change.
[0013] (b) Differentiable downsampling: anti-aliasing low-pass filtering and downsampling (from 100Hz to 20Hz) are performed on the main signal to significantly reduce the subsequent computational load.
[0014] (c) Spectrum transformation: perform real fast Fourier transform rFFT on the short sequence after downsampling and discard the direct current component to obtain its spectrum amplitude .
[0015] (d) Feature encoding: input the spectrum amplitude into a lightweight one-dimensional convolutional neural network CNN to extract deep features within the frequency band.
[0016] (e) Context vector generation: Map the CNN extracted features through fully connected layers to generate a compact frequency domain context vector representing the global periodicity prior .
[0017] S3. Parallel decoder prediction: This step employs a parallel decoder to generate prediction sequences for multiple future time steps at once. The core of this step is to utilize the features generated in S2 to make predictions through a cascaded modulation mechanism.
[0018] S3.1 Spectrum adaptive query generation: This step aims to generate a dynamic query vector that fuses the frequency domain context vector and the final hidden state .
[0019] (a) Adaptive positional encoding: Input the frequency domain context vector generated by the DSE in S2.2 into an adaptive positional encoder, which is composed of a multi-layer perceptron (MLP). The function of this encoder is to dynamically generate an adaptive positional embedding sequence that is directly related to the input gait frequency , where is the prediction step number, and is the model dimension.
[0020] (b) Initial template generation: Map the final hidden state generated by the SegRNN in S2.1 through a linear mapping to generate an initial prediction template representing the current state.
[0021] (c) Query vector synthesis: Repeat N times along the time dimension, and then element-wise add it to to generate the final dynamic query vector: S3.2 Cascaded modulation decoding - phase one: Retrieval Introduce a cross-attention mechanism. Take the dynamic query vector sequence generated in step S3.1 as the query (Query), and take the complete hidden state sequence generated by the time domain encoder in step S2.1 as the key (K) and value (V), calculate the attention weight and weighted sum to generate a historical context feature vector for each prediction step: where is the dimension of the key vector.
[0022] S3.3 Cascade modulation decoding-phase two: calibration This phase adopts the feature linear modulation FiLM mechanism to fine-tune the retrieved context features C.
[0023] (a) Modulation parameter generation: the dynamic query vector Q generated in step S3.1 is input into the input parameter generation network (for example, an MLP) to learn a set of modulation parameters, namely scaling factor and translation factor . .
[0024] (b) Dynamic feature modulation: the modulation parameters generated in step (a) and are used to perform affine transformation on the feature vector retrieved in step S3.2: wherein: represents element-wise multiplication, ensuring that the modulation parameters independently scale and offset each channel of the feature vector; is used to maintain the basic response amplitude of the feature vector, avoiding weakening of the feature information due to being too small; is the output feature vector after modulation, which not only retains the core information of the historical context features, but also injects personalized modulation information of the global cycle through dynamic parameters.
[0025] S3.4 Output generation: Based on the calibrated feature vector in step S3.3, mapping and prediction are performed to obtain the gait prediction result.
[0026] (a) The feature vector after affine transformation is mapped in parallel through a fully connected layer to obtain the prediction value of the "incremental signal" for the next N steps .
[0027] (b) Then, trajectory recovery is performed: from the posture baseline stored in S2.1, only the baseline value B corresponding to the prediction target is extracted, and the baseline value is extended to the prediction length N to obtain .
[0028] (c) The predicted incremental signal is added element-wise to the extended baseline value to recover the predicted trajectory as an absolute angle: Here is the predicted knee joint angle at the future time. Finally, the is denormalized to output the final physical value of the knee joint angle.
[0029] Compared with the prior art, the present application has the following beneficial effects: (1) Strong real-time performance: the model is based on the SegRNN framework and a parallel decoding strategy, and has a fast inference speed. In particular, the DSE module generates a compact frequency domain context vector through the strategy of "down-sampling first and then FFT" at a very small computational cost , which enables the entire decoder as a top-level control signal, achieving a synergistic gain in computational efficiency and model performance. Experimental results on an embedded device (such as Raspberry Pi 4B) show that the average time consumption of a single inference of the model is only 21.78 milliseconds, meeting the real-time control requirements of exoskeletons.
[0030] (2) High prediction accuracy: the present application uses a cascaded adaptive mechanism to finely fuse global periodic priors and local time-domain features, which consists of two levels: The first level (intention generation): uses and to generate a dynamic query vector that contains comprehensive prediction intentions .
[0031] The second level (feature calibration): uses this vector to retrieve historical context on the one hand, and to generate FiLM parameters and to modulate on the other hand.
[0032] This "query-modulation" cascaded design makes the model's prediction of key turning points in gait (such as wave peaks and troughs) more accurate.
[0033] (3) Good robustness and adaptive speed change: the present application solves the problem of model generalization to speed changes. When the wearer's walking speed changes dynamically, the extracted by DSE changes in real time and quickly adjusts the prediction mode of the decoder through the above-mentioned cascaded path. This mechanism enables the model to maintain stable and accurate prediction performance in various complex scenarios such as walking on flat ground, going up and down stairs, and changing speed. BRIEF DESCRIPTION OF DRAWINGS
[0034] Figure 1 is a structural schematic diagram of the SMCA-SegRNN model in the embodiment of the present application; Figure 2 is a schematic diagram of the lower limb exoskeleton gait trajectory prediction method of the embodiment of the present application; Figure 3 is a schematic diagram of the data acquisition device in the embodiment of the present application; Figure 4 is data collected from a certain experimenter in a flat motion mode; Figure 5 Data collected for a certain experimenter in the up stair motion mode; Figure 6 Data collected for a certain experimenter in the down stair motion mode; Figure 7 Prediction of the knee joint angle of the experimenter in the flat ground variable speed case; Figure 8 Comparison chart of the prediction results of the knee joint angle of the invention and the comparative model in the flat ground walking mode and the true value; Figure 9 Comparison chart of the prediction results of the knee joint angle of the invention and the comparative model in the up stair motion mode and the true value; Figure 10 Comparison chart of the prediction results of the knee joint angle of the invention and the comparative model in the down stair motion mode and the true value; Figure 11 Comparison chart of the prediction results of the knee joint angle of the invention and the ablation model in the flat ground walking motion mode and the true value; Figure 12 Comparison chart of the prediction results of the knee joint angle of the invention and the ablation model in the up stair motion mode and the true value; Figure 13 Comparison chart of the prediction results of the knee joint angle of the invention and the ablation model in the down stair motion mode and the true value. DETAILED DESCRIPTION
[0035] To make the purpose, technical scheme and points of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.
[0036] As shown in Figures 1 to 13 , the present application proposes a lower limb exoskeleton gait prediction method based on SMCA-SegRNN, which includes the following steps: S1. Data collection and preprocessing: Human motion data is collected through a sensor assembly, and is preprocessed to construct a gait dataset. The data collection device is as shown in Figure 3 , and its collection and preprocessing can be divided into the following steps: (1) Hardware and sampling: as shown in Figure 3 , single-sided data collection uses at least two inertial measurement units (IMU, including three-axis accelerometers and three-axis gyroscopes) and a set of knee joint angle encoders (resolution ≥0.1 ). The sampling frequency of all sensors is 100 Hz, and is transmitted to the embedded controller through the CAN bus.
[0037] (2) Time synchronization: At the embedded controller, based on the timestamps of each data, Kalman filter combined with sliding window strategy is used to estimate the state and time calibration of asynchronous data, and finally align the multi-source data to the unified time baseline.
[0038] (3) Filtering and denoising: Second-order Butterworth low-pass filter is used for acceleration, angular velocity and angle encoder signals, with a cutoff frequency of 10 Hz to reduce high-frequency noise.
[0039] (4) Normalization: According to the physical range of each parameter, the acceleration, angular velocity and angle data are normalized to the unified scale of [-1, 1] to ensure consistent data reference. The normalization formula is as follows: where .
[0040] (5) Sliding window construction: A fixed-length sliding window is used, with an input window length of frames (i.e. 3 seconds, 100 Hz), a prediction window length of , and a sliding step size of s = 1 (frame-by-frame sliding) to construct more training samples. Each window forms a sample, containing multi-channel time series input features and the target output corresponding to the window.
[0041] S2. Model training - encoding: As shown in Figure 1 , the model mainly consists of encoding and decoding parts, where the encoding part consists of the following parts: S2.1 Time domain feature extraction - SegRNN (1) Baseline normalization: Extract the sampling value of the last time step in the input sample in the gait data set, defined as the posture baseline of the input sample.
[0042] (2) Incremental signal calculation: Subtract the posture baseline from all sampling values in the input sample to eliminate absolute posture offset and make the model learn dynamic changes relative to the current posture.
[0043] (3) Segmentation strategy: Divide the input length of incremental signal into segments with a segment length of , a total of segments, where . And because of the reason of SegRNN framework, the input length and prediction length in the experiment should be divisible by the segment length .
[0044] (4) Segment-level projection and activation: Linear mapping and nonlinear activation are performed on the data of each segment, such as Figure 1As shown, each segment is mapped to a fixed-dimension segment vector using single-layer linear projection and ReLU activation function. The signal changes in 3) segmenting strategy and 4) segment-level projection and activation process as follows: where is a real sequence of length L is the sequence channel identifier; is a matrix containing n segments (each of length w); denotes the vector after linear projection and ReLU activation; , denotes the dimension of GRU hidden state.
[0045] (5) GRU encoding: the sequence of segment vectors is input to the gated recurrent unit GRU. The hidden dimension of GRU is set to d. The GRU outputs a complete hidden state sequence as the Key and Value of the encoder, the hidden state sequence the last element as the final hidden state is passed to the encoder for use as the initial prediction template generation.
[0046] S2.2 Frequency-domain context extraction - DSE: (1) Main signal selection: according to domain knowledge, a channel representing the periodicity of gait is selected as the main signal (knee angle channel is selected in this embodiment) by correlation screening from the input sample. Similarly, foot pressure or ankle angle channels representing gait periodicity can also be used as input.
[0047] (2) Differentiable down-sampling: for the main signal (knee angle channel), first pass through an anti-aliasing low-pass filter (third-order Butterworth low-pass filter with a cutoff frequency of 10 Hz in this embodiment), then use a differentiable down-sampling in the differential structure to reduce the sampling rate from 100 Hz to 20 Hz.
[0048] (3) Spectral transformation: perform real fast Fourier transform rFFT on the down-sampled short sequence, only retaining its spectral amplitude characteristics.
[0049] (4) Feature encoding: the amplitude feature is input to a one-dimensional convolutional neural network CNN, and a compact frequency-domain context vector representing the global periodicity prior is generated through full connection. where denotes the extracted univariate main signal of length 300; This is a spectral feature vector containing 30 effective frequency bins (obtained by transforming a downsampled sequence of length 60). This represents the frequency domain context vector after extraction and linear projection by a convolutional neural network.
[0050] S3. Model Training - Parallel Decoder Prediction: The parallel decoder outputs the predicted increment sequence for the next N steps (N=20 in this embodiment) in one go. The decoder consists of a spectrum adaptive query generator, a cross-attention mechanism, FiLM calibration, and a parallel output layer. The specific steps are as follows: S3.1 Adaptive Spectrum Query Generation: (1) The frequency domain context vector generated in S2.2 Input to the context-adaptive position encoder: The context-adaptive position encoder consists of a multilayer perceptron (MLP), whose function is to dynamically generate an adaptive position embedding sequence that is directly related to the input gait frequency. ,in To predict the number of steps, For model dimensions; (2) Hide the encoder in S2.1 in the final state. An initial prediction template representing the current state is generated through linear mapping. .
[0051] (3) Repeat N times along the time dimension, then... Add each element together to generate the final dynamic query vector: S3.2 Cross-attention Use the Q output from S3.2 as the Query to extract the encoder segment-level hidden sequence. As key K and value V, use cross-attention to extract the contextual feature vector: Where K and V are pairs Linear projection: The attention output generates a historical context feature vector for each prediction step. .
[0052] S3.3 FiLM Calibration (1) The dynamic query vector generated in S3.1 Input parameter generation network (a small MLP in this example) to produce FiLM parameters: i.e. scaling factors. Translation factor .
[0053] (2) The retrieved context features are subjected to affine transformation. FiLM helps to modulate the distribution of the retrieved features in time domain with the frequency domain prior, thus adapting to the speed and gait variations. The affine transformation is calculated as follows: In the formula: represents element-wise multiplication, ensuring that the modulation parameters independently scale and shift each channel of the feature vector; is used to maintain the basic response amplitude of the feature vector, avoiding weakening of the feature information due to being too small; is the output feature vector after modulation, which not only retains the core information of the historical context features, but also injects personalized modulation information of the global cycle through dynamic parameters.
[0054] S3.4 Parallel output generation and trajectory recovery: (1) The feature vector after affine transformation is mapped in parallel through a fully connected layer to obtain the incremental signal prediction value for the next N steps
[0055] (2) From the saved pose baseline, extract the baseline value B corresponding to the predicted target (i.e. knee angle) and expand it, then add the increment to recover the absolute trajectory: Here is the result of expanding the baseline value B to the prediction length N, is the predicted knee angle at future time. Finally, the is denormalized to output the final physical value of the knee angle.
[0056] Experimental verification: To verify the accuracy and reliability of the SMCA-SegRNN model in the invention for gait prediction under different speeds and different motion patterns, experiments were conducted under the exoskeleton shown in Figure 3
[0057] The exoskeleton is mainly used for motion posture collection and prediction in the sagittal plane of the knee joint. Figure 3 The exoskeleton shown in uses CAN for communication between modules, and the power module converts 24V of the lithium battery to 5V for sensor (two IMUs) power supply, and the driver / angle sensor uses 24V power supply.
[0058] To ensure the safety of the user, the knee exoskeleton is provided with physical limits, which limits the motion range in the sagittal plane to (extension)- (flexion).
[0059] In the experiment, 6 healthy subjects were recruited, and three daily exercise modes including flat ground variable speed walking and up and down stairs were designed. During the experiment, the subjects completed the flat ground variable speed walking and up and down stairs exercise according to the preset process in the designated test site, and the duration of each exercise was not less than 2 minutes. The length of all model inputs was set to 300 frames, the length of the output was set to 20 frames, and the segmentation length of the model in the SMCA-SegRNN and its ablation experiment was set to 10.
[0060] The knee joint angle of one of the subjects in three exercise modes is shown in Figure 4 , Figure 5 and Figure 6 , respectively, which are flat ground, up stairs and down stairs.
[0061] In order to ensure the safety of the subjects and prevent the model from interfering, only the SMCA-SegRNN was used for prediction and did not form a control closed loop to drive the servo motor for assistance during the experiment, and the actuator was set to zero torque output mode. As shown in Figure 7 , in the case of flat ground variable speed, the future 20 frame knee joint angle prediction was made through the input of 300 frame past data, and the prediction result was highly consistent with the actual knee joint motion angle trajectory of the subject during walking.
[0062] As shown in Figure 8 , Figure 9 and Figure 10 , the comparison curve diagrams of the knee joint angle prediction of SMCA-SegRNN and other baseline methods (LSTM, CNN-LSTM-Attention and SegRNN) in different exercise modes are shown. In each exercise mode prediction diagram, the real data of the right knee joint is represented by a black solid line; the red line represents the SMCA-SegRNN model proposed in this study, the gray line represents the basic comparison model LSTM, the blue line represents the comparison model CNN-LSTM-Attention, and the last yellow line represents the original SegRNN model.
[0063] As can be observed from Figure 8 , Figure 9 and Figure 10 , the gait trajectory generated by the SMCA-SegRNN network is very close to the actual gait of the subject, showing good tracking performance. In contrast, although the other baseline methods can also achieve prediction of the gait trajectory, the overall prediction error is larger compared with the SMCA-SegRNN.
[0064] In order to verify the rationality and effectiveness of the technical scheme of the present application, the MAE, RMSE, and prediction length as evaluation metrics. The implementation of this embodiment is based on the deep learning framework Pytorch, and except for the model training on a high-performance server (32GB memory, Nvidia GeForce RTX3080 graphics card), other experiments are implemented on Raspberry Pi 4B (2G version).
[0065] Table 1 is a comparison of experimental results of different models under three motion modes
[0066] The experimental results of Table 1 prove that the SMCA-SegRNN model exhibits better comprehensive performance compared with baseline methods such as SegRNN, LSTM and CNN-LSTM with attention mechanism: in the three motion modes of walking on flat ground, going upstairs and going downstairs, the MAE (1.43-1.65) and RMSE (0.84-1.02) are the lowest, the R 2 (0.9965-0.9979) is the highest, and the prediction accuracy is significantly better than other models; the prediction length (21.27-22.27 ms) is slightly higher than LSTM, but much lower than CNN-LSTM-Attention and SegRNN, achieving a balance between high prediction accuracy and fast prediction speed.
[0067] Table 2 is a comparison of experimental results of ablation models under three motion modes
[0068] As Figure 11 , Figure 12 and Figure 13 and Table 2 show the performance of the SMCA-SegRNN model during the ablation experiment. The ablation experiment evaluates the contribution of these modules to the overall performance by removing key parts of the model (such as replacing the FiLM module with a normal stack, replacing the DSE module with a general position embedding, and replacing the SegRNN base framework with an LSTM framework). The results show that under different motion modes, the complete SMCA-SegRNN model has more advantages in comprehensive performance (such as prediction accuracy and prediction speed) compared with each ablation variant.
[0069] Referring to Figure 11 , Figure 12 and Figure 13 , the prediction effects of the method of the present application (SMCA-SegRNN, red thick line), actual value (Actual, black thick line) and other ablation models under three motion modes are shown.
[0070] As can be seen from the figure, the prediction curve (red thick line) of the SMCA-SegRNN model of the application is almost coincided with the actual value curve (black thick line), especially at the key turning points such as the peak and valley of gait, and the prediction accuracy is obviously higher than that of other comparative models.
[0071] The quantitative analysis results (as shown in Table 2) further confirm this point: Prediction accuracy: The average absolute error (MAE) of the method of the application in three modes is 1.55, 1.43 and 1.65 respectively; the root mean square error (RMSE) is 1.01, 0.84 and 1.02 respectively. These error indicators are significantly lower than all the comparative models.
[0072] Real-time performance: When deployed on Raspberry Pi 4B, the average time consumption of a single inference of the model of the application in three modes is 22.27ms, 21.27ms and 21.80ms respectively. The average time consumption is about 21.78ms, which is lower than the delay requirement of human-computer interaction and meets the real-time control requirement.
[0073] In summary, the specific embodiment of the application achieves Figure 1 the SMCA-SegRNN model architecture shown in Figure 2 the training and measurement process shown in Figure 3 the system hardware shown in a high-precision, high-real-time lower limb exoskeleton gait prediction method.
Claims
1. A gait prediction method based on a spectrum modulation cross-attention segmented recurrent network, characterized in that, The method comprises the following steps: S1. Collect unilateral knee joint data and preprocess to build a gait dataset; S2. Data in the gait dataset is extracted through two parallel encoding paths to extract deep features of the input signal from the time domain and the frequency domain respectively; S3. The deep features extracted from the time domain and the frequency domain are decoded through a cascaded modulation mechanism to obtain a gait prediction result.
2. The gait prediction method based on the frequency spectrum modulation cross-attention segmented recurrent network according to claim 1, characterized in that, The step S1 is specifically implemented as follows: acquiring original gait data through a unilateral knee joint data acquisition device; performing time alignment based on a timestamp, normalization based on a sensor physical unit and sampling accuracy, low-pass filtering on the collected multi-dimensional data, and finally using a sliding window to divide the continuous data stream into fixed-length time sequence samples to build a dataset.
3. The gait prediction method based on the spectrum modulation cross-attention segmented recurrent network according to claim 1, characterized in that, The step S2 is specifically implemented as follows: S2.1 Capture the time-dependent relationship of signals based on Segmented Recurrent Neural Network (SegRNN) for data in gait dataset, and obtain final hidden state ; S2.2 Extract global frequency domain features of the input samples in the gait data set by down-sampling the differentiable spectral encoder DSE, to obtain a frequency domain context vector .
4. The gait prediction method based on the spectrum modulation cross-attention segmented recurrent network according to claim 3, characterized in that, The step S2.1 is specifically implemented as follows: The sampling value of the last time step in the input sample in the gait dataset is extracted, which is defined as the posture baseline of the input sample; All sampling values in the input sample are subtracted from the posture baseline to eliminate the absolute posture offset, thereby obtaining an incremental signal representing the relative motion change relative to the posture of the window end; The incremental signal is taken as the input of the encoder SegRNN; the encoder divides the incremental signal into multiple fixed-length segments, converts them into vectors through a linear layer and an activation function, and then extracts the time sequence dependency between the segments through a gated recurrent unit (GRU) to generate a time-domain encoder hidden state sequence containing all time steps , the hidden state sequence The last element is taken as the final hidden state .
5. The gait prediction method based on the spectrum modulation cross-attention segmented recurrent network according to claim 4, characterized in that, The step S2.2 is specifically implemented as follows: A characteristic element is selected as a main signal, i.e., a knee joint angle signal, through correlation screening from the input sample to represent the periodicity of the gait cycle; The main signal is subjected to anti-aliasing low-pass filtering and down-sampling; performing a real fast Fourier transform rFFT on the down-sampled sequence and discarding the direct current component to obtain a spectral magnitude thereof ; The frequency spectrum amplitude is input into a one-dimensional convolutional neural network (CNN) to extract deep features in the frequency band; The CNN extracted features are mapped through a fully connected layer to generate a compact, frequency domain context vector that represents the global periodicity prior .
6. The gait prediction method based on the spectrum modulation cross-attention segmented recurrent network according to claim 5, characterized in that, The step S3 is specifically implemented as follows: S3.1 Based on the adaptive position encoding, generate a fusion frequency domain context vector and the final hidden state of the dynamic query vector ; S3.2 Introduce cross-attention mechanism to the dynamic query vector sequence As queries Query, the hidden state sequence As keys K and values V, compute attention weights and weighted sum, generate one historical context feature vector for each prediction step ; S3.3 Adopting a feature-wise linear modulation FiLM mechanism to the retrieved contextual features calibration; S3.4 Feature vector calibrated based on step S3.3 The mapping and prediction are performed to obtain the gait prediction result.
7. The gait prediction method based on the spectrum modulation cross-attention segmented recurrent network according to claim 6, characterized in that, The adaptive position encoding is specifically implemented as follows: the DSE-generated frequency-domain context vector to a context-adaptive position encoder, which is composed of a multi-layer perceptron (MLP) to dynamically generate an adaptive position embedding sequence directly related to the input gait frequency wherein for predicting the step count, is the dimensionality; The final hidden state generated by SegRNN By linear mapping, generate an initial prediction template representing the current state ; will be repeated N times along the time dimension, and then will be repeated N times along the time dimension, and then will be repeated N times along the time dimension, and then .
8. The gait prediction method based on the spectrum modulation cross-attention segmented recurrent network according to claim 7, characterized in that, The step S3.3 is specifically implemented as follows: The generated dynamic query vector In the input parameter generation network, a set of modulation parameters, i.e., scaling factors and translation factors are learned to generate The generated modulation parameters and performing an affine transformation on the feature vector : wherein: represents an element-wise multiplication, is the output feature vector after modulation.
9. The gait prediction method based on the spectrum modulation cross-attention segmented recurrent network according to claim 8, characterized in that, The step S3.4 is specifically implemented as follows: affine-transformed feature vector through the fully connected layer, and parallel mapping to obtain the incremental signal prediction value of the future N steps ; From the pose baseline stored in S2.1, only the single feature channel baseline value B corresponding to the prediction target is extracted, and the baseline value is extended to the prediction length N to obtain ; The predicted incremental signal and the extended baseline value are added element by element to restore the predicted trajectory to an absolute angle: Here That is, the predicted knee joint angle at a future moment; finally, for Perform inverse normalization to output the final physical value of the knee joint angle, thus completing gait prediction.