Gait prediction method for lower limb exoskeleton wearer

Abstract

The invention provides a gait prediction method for a lower limb exoskeleton wearer, and the method comprises the steps: obtaining an sEMG signal and joint angle data of the wearer, and carrying out the data preprocessing; inputting the preprocessed sEMG signal into a renetwork feature extraction layer to obtain an sEMGResNet feature map; performing image processing on the sEMGResNet feature map through a visual converter to obtain a sEMG-ViT feature map; inputting the preprocessed joint angle data into a visual converter to obtain an angle ViT feature map; respectively inputting the sEMGResNet feature map, the sEMG-ViT feature map and the angle ViT feature map into a first-layer element learning device, and outputting feature maps of different speeds; and carrying out aggregation processing on the feature maps of different speeds through a second-layer element learning device to obtain a gait prediction result of the wearer. According to the method, the accuracy of a gait prediction result is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of robots, and in particular, to a gait prediction method for lower limb exoskeleton wearers. Background Art

[0002] Patients suffering from diseases related to lower limb movement caused by diseases such as brain injury or spinal cord injury have a huge demand for assistive walking devices. Among them, the lower limb exoskeleton (LLE) for assisting walking has many advantages. The lower limb exoskeleton robot can help patients with walking disabilities such as hemiplegia, quadriplegia, and arthritis to attempt to walk independently, obtain a certain degree of self-care ability, and gradually recover the walking ability during the rehabilitation treatment process, which can reduce the manpower demand and improve the efficiency. Therefore, the lower limb exoskeleton robot has great development potential in restoring or enhancing the independent movement ability of lower limb injury patients.

[0003] Gait prediction is an important technology for realizing the assisted walking of lower limb exoskeletons. This technology can predict the movement intention of walking-disabled people when wearing lower limb exoskeletons in real time, adjust the control strategy of the lower limb exoskeletons, and help them carry out safe and effective rehabilitation training. In the gait prediction task, a key challenge is to accurately and effectively estimate the human movement intention to ensure that the robot responds in a natural and timely manner. However, most existing gait prediction methods can only predict the gait in the fast walking scenario and do not consider the variable-speed walking gait. Among the few existing variable-speed studies, the results obtained all require a large amount of data as support. For the user group of exoskeletons, especially those with a higher level of motor dysfunction, obtaining a large amount of walking data is a challenge, which makes it difficult to implement these variable-speed gait prediction methods. How to use as few samples as possible to reasonably estimate the cross-speed movement and also consider the differences in walking habits between individuals is an important issue in exploring the comfortable walking of lower limb exoskeletons.

[0004] In the existing technical solutions, the human movement intention can be estimated through various signals, and these signals can be divided into biological signals and non-biological signals. The non-biological signals can generate the walking gait in the human exoskeleton system through a mathematical model. These methods based on the mathematical model have strong interpretability and can reduce the time required to construct the prediction model. Another method is data-driven pattern recognition, which judges the human movement intention by constructing the mapping relationship between the surface electromyogram signal, the sensor measurement signal and the biological quantity to be measured. This method is more flexible, but at the same time has a greater requirement for the amount of data required to construct the model. These methods rely on an accurate mathematical model of the human exoskeleton system, and obtaining such an accurate model is still a huge challenge; secondly, the gait generated by these methods only follows the mathematical model and does not contain human biological signals, resulting in limited bionics. Summary of the Invention

[0005] In view of this, the present invention provides a gait prediction method for lower limb exoskeleton wearers to solve the above problems.

[0006] The present invention provides a gait prediction method for lower limb exoskeleton wearers, including: acquiring the sEMG signals and joint angle data of the wearer, and performing data preprocessing; inputting the preprocessed sEMG signals into the ResNet feature extraction layer to obtain the sEMGResNet feature map; performing image processing on the sEMGResNet feature map through a vision transformer to obtain the sEMG-ViT feature map; inputting the preprocessed joint angle data into the vision transformer to obtain the angle ViT feature map; respectively inputting the sEMGResNet feature map, the sEMG-ViT feature map, and the angle ViT feature map into the first-layer meta-learner, and outputting feature maps at different speeds; performing aggregation processing on the feature maps at different speeds through the second-layer meta-learner to obtain the gait prediction result of the wearer.

[0007] In another implementation manner of the present invention, the acquiring the sEMG signals and joint angle data of the wearer includes: recording the sEMG signals and joint angle data of the lower limbs of the wearer through a Trigno avant sensor; wherein, the Trigno avant sensor integrates an electromyogram sensor and an inertial measurement unit.

[0008] In another implementation manner of the present invention, the electromyogram sensor samples the sEMG signals at a frequency of 1926 Hz; the inertial measurement unit samples the joint angle data at a frequency of 148 Hz.

[0009] In another implementation manner of the present invention, the data preprocessing includes: performing DC component removal, band-pass filtering, full-wave rectification, and low-pass filtering on the sEMG signals.

[0010] In another implementation manner of the present invention, the performing image processing on the sEMGResNet feature map through a vision transformer to obtain the sEMG-ViT feature map includes: extracting a plurality of patches from the sEMG sequence in the sEMGResNet feature map through a sliding window method; performing label embedding and position encoding on the plurality of patches to obtain an input sequence; inputting the input sequence into the vision transformer for data processing to obtain the sEMG-ViT feature map.

[0011] In another implementation manner of the present invention, the position encoding is represented by sine and cosine functions:

[0012]

[0013] wherein, pos represents the position; dmodel represents the model dimension; i represents the frequency of each dimension.

[0014] In another implementation manner of the present invention, it further includes: establishing relationships between different dimension information in the sEMG sequence through a multi-head self-attention mechanism in the visual transformer.

[0015] On the other hand, the present invention provides a gait prediction system for a lower limb exoskeleton wearer, including: a data acquisition module: acquiring the sEMG signal and joint angle data of the wearer, and performing data preprocessing; a feature map generation module: inputting the preprocessed sEMG signal into a heavy network feature extraction layer to obtain an sEMGResNet feature map; performing image processing on the sEMGResNet feature map through a visual transformer to obtain an sEMG-ViT feature map; inputting the preprocessed joint angle data into the visual transformer to obtain an angle ViT feature map; a feature map processing module: respectively inputting the sEMGResNet feature map, the sEMG-ViT feature map, and the angle ViT feature map into a first-layer meta-learner, and outputting feature maps at different speeds; a result prediction module: performing aggregation processing on the feature maps at different speeds through a second-layer meta-learner to obtain the gait prediction result of the wearer.

[0016] On the other hand, the present invention provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the computer program, it implements the steps of a gait prediction method for a lower limb exoskeleton wearer as described in any one of the above.

[0017] On the other hand, the present invention provides a computer storage medium, characterized in that a computer program is stored on the computer storage medium, and when the computer program is executed by a processor, it implements the steps in a gait prediction method for a lower limb exoskeleton wearer as described in any one of the above.

[0018] The gait prediction method for a lower limb exoskeleton wearer of the present invention designs a two-layer meta-learning structure including a meta-learner I and a meta-learner II. This structure can effectively extract the common features of sEMG signals and joint angles at different speeds, and improve the generalization performance of the network at different speeds; establish a mapping from surface electromyogram signals to joint angles, which can accurately predict the gait of the wearer in complex motion situations, effectively solve the challenge of multi-scenario gait prediction on a small sample data set, and improve the accuracy of the gait prediction result. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for describing the embodiments or the prior art. By reading the detailed description of the following embodiments, the advantages and benefits in the solutions become clear to those skilled in the art. The accompanying drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention. In the accompanying drawings:

[0020] Figure 1 Schematic diagram of the gait prediction method for lower limb exoskeleton wearers according to an embodiment of the present invention.

[0021] Figure 2 Schematic diagram of the information processing flow of Mlformer according to an embodiment of the present invention.

[0022] Figure 3 Feature map of the base learner in the MLformer framework according to an embodiment of the present invention.

[0023] Figure 4 Schematic diagram of the LLER system used in a physical experiment according to an embodiment of the present invention.

[0024] Figure 5 Schematic diagram of the joint position method of the MLformer network and baseline prediction at three different walking speeds according to an embodiment of the present invention.

[0025] Figure 6 Schematic diagram of the experimental results of cross - speed comparison according to an embodiment of the present invention.

[0026] Figure 7 Schematic diagram of the experimental results of cross - speed ablation according to an embodiment of the present invention.

[0027] Figure 8 Schematic diagram of the experimental results of cross - subject according to an embodiment of the present invention.

[0028] Figure 9 Schematic diagram of the experimental results of knowledge transfer according to an embodiment of the present invention. Detailed implementation manners

[0029] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the following will clearly and detailedly describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art should fall within the scope protected by the embodiments of the present invention.

[0030] Figure 1A schematic flowchart of a gait prediction method for lower limb exoskeleton wearers provided by an embodiment of the present invention is as follows Figure 1 As shown, this embodiment mainly includes:

[0031] S101. Obtain the sEMG signals and joint angle data of the wearer, and perform data preprocessing.

[0032] S102. Input the preprocessed sEMG signals into the re-network feature extraction layer to obtain the sEMGResNet feature map.

[0033] Exemplarily, as Figure 2 shown, introduce the transfer learning method into the neural network MLformer, which consists of three parts: meta learner, ViT network, and ResNet feature extraction layer. Use ResNet to capture the features of new and existing subject data to obtain the sEMG-ResNet feature map. The adopted residual network architecture has two residual blocks, each block includes a convolutional layer, a regularization layer, and a rectified linear unit (ReLU) activation function. In order to maintain a consistent size during the residual connection process, ResNet adds a shortcut connection to the input.

[0034] S103. Perform image processing on the sEMGResNet feature map through a vision transformer to obtain the sEMG-ViT feature map.

[0035] S104. Input the preprocessed joint angle data into the vision transformer to obtain the angle ViT feature map.

[0036] S105. Input the sEMGResNet feature map, sEMG-ViT feature map, and angle ViT feature map into the first-layer meta-learner respectively, and output feature maps at different speeds.

[0037] Exemplarily, these three feature maps are respectively input into three base learners, and their outputs are finally encoded as the personalized gait of the wearer. Finally, the gait prediction task can be divided into solving three mapping problems.

[0038] Integrate the two-layer metal learning structure composed of meta-learner I and meta-learner II into MLformer. Meta-learner I consists of three base learners, and each meta-learner in the first layer receives three feature maps corresponding to a single speed. As Figure 3 shown, the three feature maps included in meta-learner I can be seen that as the training progresses, the feature maps begin to form a hierarchical structure, learning the attributes unique to each data. The self-attention mechanisms in ResNet and the base learners show very obvious feature extraction capabilities in different experimental scenarios.

[0039] It should be understood that by integrating the meta - learning structure, it is possible to effectively handle few - shot learning tasks. In practical applications, especially for exoskeleton users with impaired motor functions, it is difficult to obtain a large amount of walking data. This feature of MLformer enables the training of a model with good performance even under limited data conditions, reducing the cost and difficulty of data collection.

[0040] S106. Aggregate the feature maps of different speeds through the second - layer meta - learner to obtain the gait prediction result of the wearer.

[0041] Exemplarily, the output of the first - layer meta - learner consists of feature maps of different speeds and is sent to the second - layer meta - learner for aggregation to obtain the final result.

[0042] The gait prediction method for lower - limb exoskeleton wearers of the present invention designs a two - layer meta - learning structure including Meta - learner I and Meta - learner II. This structure can effectively extract the common features of sEMG signals and joint angles at different speeds, improving the generalization performance of the network at different speeds; it establishes a mapping from surface electromyogram signals to joint angles, can accurately predict the gait of the wearer in complex motion situations, effectively solves the challenge of multi - scenario gait prediction on small - sample data sets, and improves the accuracy of the gait prediction result.

[0043] In another implementation manner of the present invention, the obtaining of the sEMG signal and joint angle data of the wearer includes: recording the sEMG signal and joint angle data of the lower limbs of the wearer through a Trigno Vanguard sensor; wherein, the Trigno Vanguard sensor integrates an electromyogram sensor and an inertial measurement unit.

[0044] Exemplarily, joint angle signals and surface electromyogram (sEMG) signals belong to biological signals. The data set collection and pre - processing steps include a preparation stage and a data collection and pre - processing stage. In the preparation stage of the data set collection and pre - processing experiment, electrodes are attached to the main muscles of the lower limbs of the wearer, including the rectus femoris, lateral thigh, tibialis anterior, biceps femoris, semitendinosus, and gastrocnemius. The sEMG signal and joint angle of the lower limbs are recorded through a Trigno Vanguard sensor, which integrates an electromyogram sensor and an inertial measurement unit to obtain accurate muscle activity and motion data.

[0045] In another implementation manner of the present invention, the electromyogram sensor samples the sEMG signal at a frequency of 1926 Hz; the inertial measurement unit samples the joint angle data at a frequency of 148 Hz.

[0046] In another implementation manner of the present invention, the data pre - processing includes: performing DC component removal, band - pass filtering, full - wave rectification, and low - pass filtering on the sEMG signal.

[0047] Exemplarily, after collecting sEMG signals and joint angles, preprocessing is required to reduce interference generated by industrial frequency noise and electrocardiogram (ECG) signals. First, the DC component in each sEMG signal is removed; the sEMG signal is band-pass filtered through a fourth-order Butterworth filter (20 - 450 Hz), and then full-wave rectification is performed; the signal is low-pass filtered at 20 Hz to obtain linear envelope characteristics.

[0048] The waveforms of the measured sEMG signals and joint angles exhibit periodic similarity, and a consistent time interval is observed between the two waveforms. The average time interval between the peak joint angle and the sEMG signal is 75 milliseconds. The observed progress of the sEMG signal during joint movement supports the feasibility of using deep k learning methods for intention recognition.

[0049] In another implementation of the present invention, the image processing of the sEMGResNet feature map by the vision transformer to obtain the sEMG-ViT feature map includes: extracting a plurality of patches from the sEMG sequence in the sEMGResNet feature map by a sliding window method; performing label embedding and position encoding on the plurality of patches to obtain an input sequence; inputting the input sequence into the vision transformer for data processing to obtain the sEMG-ViT feature map.

[0050] In another implementation of the present invention, the position encoding is represented by sine and cosine functions:

[0051]

[0052] where pos represents the position; d model represents the model dimension; i represents the frequency of each dimension.

[0053] Exemplarily, in previous VIT tests, the image was divided into small regions of uniform size, and then compressed and scaled. The sequential task of gait prediction can be understood as dividing the time series into multiple windows of a certain size and length, and then inputting them into the subsequent structure. Dividing the time series into multiple patches can not only improve the ability to capture local features of the sequence, but also reduce the number of required parameters and computational complexity.

[0054] Preferably, the sliding window method is used to extract multiple patches from the sEMG sequence. To make full use of the periodic structure of the given sequence, the encoder input is subjected to label embedding and positional encoding. This process encodes the input data while maintaining its relative position, thereby enhancing the consistency of model prediction. Positional encoding uses sine and cosine functions to represent the input encoding. Each element is a sin / cos function with a predefined frequency index.

[0055] In another implementation of the present invention, it further includes: establishing relationships between different-dimensional information in the sEMG sequence through a multi-head self-attention mechanism in the visual transformer.

[0056] Exemplarily, the input sequence of the ViT is processed by combining three different groups of matrices, which can be expressed as:

[0057] and

[0058] These vectors are used to calculate the weighted sum according to their correlation, and the weighted sum can be expressed as:

[0059]

[0060] where L Q , L K and L V represent the lengths of the query, key, and value respectively; d represents the dimension of all vectors.

[0061] The softmax function S can be expressed as:

[0062]

[0063] where S is also called the attention matrix.

[0064] The multi-head self-attention mechanism establishes relationships between different-dimensional information in the sEMG sequence, enhances the ability to extract key relationships, and improves the efficiency and accuracy of feature map generation. The multi-head attention mechanism can be written as:

[0065] MultiheadA(Q,K,V)=Concat(head1,…,head H )W O

[0066] head i =A(QW i Q ,KW i K ,VW i V )

[0067] Among them, the projection is a parameter matrix, described as follows:

[0068]

[0069]

[0070] An MLP is used as the regression part for human intention recognition. In ViT, two MLP layers are used to transform features and improve the nonlinear problem of the model:

[0071] MLP(x) = FN(σ(FN(x)))

[0072] FN(x) = W X + B

[0073] Among them, W and B represent the MLP; σ is the activation function.

[0074] By pre-training the ViT model on general domain data and then fine-tuning the unfrozen parameters of MLformer using a small amount of personalized domain data, cross-subject prediction is achieved. This strategy makes full use of the existing data, reduces the amount of data that needs to be provided by new subjects, and improves the practicability and operability of the model. Compared with the traditional method of training the model from scratch, it greatly saves time and resources.

[0075] In another implementation manner of the present invention, in order to evaluate the feasibility of the MLformer method proposed in this application in cross-speed and cross-disciplinary prediction tasks, cross-speed prediction experiments, cross-disciplinary prediction experiments, and physical experiments are carried out. In the cross-speed prediction experiment, comparison and ablation studies are carried out to evaluate the cross-speed performance of MLformer and to evaluate the necessity and effectiveness of the meta-learning structure and the ViT network. In the cross-disciplinary experiment, the transfer learning performance of MLformer in various new disciplines is evaluated and verified. Finally, in order to verify the effectiveness of MLformer and the reliability of predicting gait, a physical walking experiment is successfully carried out using the LLER carrying various subjects.

[0076] In order to verify the accuracy and reliability of predicting gait between different speeds and different subjects based on the Mlformer network in the present invention, this application conducts experiments on the lower limb exoskeleton robot developed by the Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences.

[0077] As Figure 4 shown, the lower limb exoskeleton robot (LLER) is a machine used to assist the walking of lower limb disabled persons and is suitable for rehabilitation training and autonomous walking. The hip and knee sagittal planes are respectively driven by motors, and the ankle joint is a universal joint, which enables the device to have self-balancing ability and flexible gait.

[0078] To achieve rehabilitation exercises, a four-degree-of-freedom exoskeleton robot used in the experiment developed hip and knee modules based on rotary actuators. This module consists of a brushless DC motor with friction braking, a harmonic drive gear unit, a servo drive, incremental and absolute position encoders, and an EtherCAT communication bus. The joint motors of this device are all integrated with an encoder, which can be used to feedback joint angles. In addition, when the device is walking, the motor torque signal can be read. The joint module can provide a continuous joint torque of 52 Nm and a peak joint torque of 107 Nm, and the maximum rotational speed is still 30 rpm. To balance the complexity of the rehabilitation task and the safety of human-exoskeleton interaction, the corresponding motion ranges of the hip and knee modules of the four-degree-of-freedom exoskeleton robot are 90° (flexion) - 25° (extension) and 0° (extension) - 90° (flexion).

[0079] A series of physical experiments were carried out. The lower limb exoskeleton involved in the experiment has a unique design, with each leg having five degrees of freedom of movement, enabling the hip, knee, and ankle to rotate flexibly in the sagittal and coronal planes. In addition, this exoskeleton also has a self-balancing function, which can support various actions such as standing, squatting, and walking, effectively helping people with movement disorders walk according to the predetermined gait.

[0080] In the experiment, three subjects (S1: 62 kg / 168 cm, S2: 77 kg / 181 cm, S3: 85 kg / 176 cm) were recruited. S1 and S2 helped to form the dataset, while S1 and S3 participated in the physical walking experiment to verify the effectiveness of the proposed method. Electrodes were attached to the main muscles of the subjects' lower limbs, including the rectus femoris, lateral thigh, tibialis anterior, biceps femoris, semitendinosus, and gastrocnemius. In the dataset collection and preprocessing experiment, to prevent muscle fatigue, all wearers walked at speeds of 1 km / h, 1.5 km / h, 2 km / h, 2.5 km / h, 3 km / h, 3.5 km / h, 4 km / h, and 4.5 km / h, with a 10-minute break between each speed change. During these processes, sEMG signals, hip joint movement / extension angles, and knee joint movement / extension angles were measured simultaneously.

[0081] Let the subjects walk along the gait trajectory predicted by the MLformer network. It is worth mentioning that the subjects do not need to actively exert force to move the exoskeleton, but rely on the joint motors of the exoskeleton to drive the walking. When the subjects walk while wearing the exoskeleton, the movement angles of their hip and knee joints show a highly consistent pattern with the predicted gait trajectory. At the same time, the torque of the joint motors also shows obvious periodic changes with the alternating swing of the lower limbs. In particular, the hip and knee motors of the exoskeleton provide about 52 - 107 Nm of assistance to the wearers during walking, and this assistance level will be dynamically adjusted according to the gait phase of the wearers.

[0082] Through the results of the above physical experiments, the feasibility and effectiveness of the MLformer network in gait trajectory prediction have been successfully verified, providing more accurate gait support for individuals wearing lower limb exoskeletons.

[0083] As Figure 5 shown, it presents a comparative curve graph of the hip and knee joint angles predicted by the MLformer network and other baseline methods (Conv-LSTM, MBNN, TCCT) at different walking speeds. The figure respectively shows the real data obtained from the left hip joint and the left knee joint; the red line represents the gait trajectory predicted by the MLformer network; the yellow line represents the prediction result of Conv-LSTM; the gray line represents the prediction result of MBNN; the blue line represents the prediction result of TCCT. The green line represents the actual joint angle (ground truth), which is the data result measured by the sensors in the experiment.

[0084] It can be clearly observed from the figure that the gait trajectory generated by the MLformer network is very close to the actual gait of healthy subjects, showing good tracking performance. In contrast, although Conv-LSTM, MBNN, and TCCT can track the true values well at some moments, overall, compared with the MLformer, the deviation of their prediction results is relatively large, especially in the regions where the joint angles change rapidly.

[0085] To verify the performance of the MLformer network, a comparative experiment was conducted. sEMG was used as the input, gait prediction was carried out through the MLformer framework, and it was compared with common sequence prediction models Conv-LSTM, MBNN, and TCCT.

[0086] When evaluating the performance of the proposed framework, the present invention adopted two widely recognized and commonly used evaluation criteria to ensure the comprehensiveness and accuracy of the evaluation results.

[0087] First of all, the Mean Absolute Error (MAE) was adopted as one of the evaluation indicators. MAE measures the size of the prediction error by calculating the average of the absolute values of the differences between the predicted values and the true values. This indicator can directly reflect the error level of the prediction model on average and will not be offset by the positive and negative of the errors.

[0088] MAE is defined as:

[0089] where y t and are the actual measured values and the corresponding predicted values respectively; T represents the sampling time.

[0090] Secondly, the Pearson's Correlation Coefficient (CC) was used to evaluate the linear correlation between the predicted values and the true values. The Pearson correlation coefficient can quantify the degree of correlation between the predicted values and the true values, and its value range is between -1 and 1. The closer the value is to 1, the stronger the positive correlation between the two. By calculating this metric, the ability of the prediction model to capture the underlying trends in the data can be understood.

[0091] CC is defined as:

[0092] where, is the average value of the corresponding predicted value y t .

[0093] The experimental results show that the MLformer network can accurately predict joint positions at different walking speeds and is highly consistent with the true values. In contrast, other baseline methods (Conv-LSTM, MBNN, TCCT) are inferior to MLformer in terms of prediction accuracy and tracking performance. This verifies the reliability of MLformer in cross-speed gait prediction tasks.

[0094] As Figure 6 shown, the gait prediction performance comparison between the MLformer network and several other baseline methods (such as Conv-LSTM, MBNN, TCCT) at different walking speeds is presented. The experiment evaluated the prediction results of the model under various speed conditions and used MAE and CC as evaluation metrics. These metrics are used to measure the closeness and correlation between the predicted values and the true values. The results verify the prediction accuracy and stability of MLformer at different speeds, indicating that it is superior to other baseline methods in cross-speed prediction tasks.

[0095] As Figure 7 shown, the performance of MLformer during the ablation experiment is recorded. The ablation experiment evaluated the contribution of certain key components in the model (such as the meta-learning structure, the angular feature base learner, the pre-trained ViT structure) by removing them. The results show the performance differences between the complete MLformer model and each ablation variant under different speed conditions. Through these comparisons, the importance of each component in improving the prediction accuracy and generalization ability of the model can be clarified.

[0096] As Figure 8As shown, the experimental results of MLformer in cross-subject prediction tasks are presented. The experiment aims to verify the model's transfer learning ability between different subjects, that is, the model's adaptability and prediction performance on new subjects. By fine-tuning on a small amount of new subject data, the generalization ability of MLformer is evaluated. The results show that MLformer can quickly adapt to new subjects and accurately predict their gait with only a small amount of data, demonstrating the superior performance of the model in cross-subject tasks.

[0097] As Figure 9 shown, the effects of two different training strategies on the performance of MLformer are compared. One is to train only using a small amount of new subject data, and the other is to pre-train the model on general domain data and then fine-tune it with a small amount of new subject data. The results show that the pre-training plus fine-tuning strategy can significantly improve the model's prediction performance on new subjects, reduce the dependence on a large amount of data, and improve the practicality and operability of the model. This verifies the effectiveness of knowledge transfer in enhancing the model's generalization ability.

[0098] The present invention proposes an innovative MLformer network framework for gait prediction of lower limb exoskeleton wearers. By combining a residual network (ResNet), a vision transformer (ViT), and a meta-learning structure, this framework achieves high-precision mapping from surface electromyogram (sEMG) signals to joint angles, and performs excellently in cross-speed and cross-subject scenarios, significantly improving the prediction accuracy and generalization ability, providing effective technical support for the autonomous walking and rehabilitation training of lower limb exoskeletons. This method has great potential to enable paralyzed lower limb patients to achieve independent activities without relying on external assistance.

[0099] On the other hand, the present invention provides a gait prediction system for lower limb exoskeleton wearers, including:

[0100] Data acquisition module: Acquire the sEMG signals and joint angle data of the wearer and perform data preprocessing.

[0101] Feature map generation module: Input the preprocessed sEMG signals into the ResNet feature extraction layer to obtain the sEMGResNet feature map; perform image processing on the sEMGResNet feature map through a vision transformer to obtain the sEMG-ViT feature map; input the preprocessed joint angle data into the vision transformer to obtain the angle ViT feature map.

[0102] Feature map processing module: Input the sEMGResNet feature map, sEMG-ViT feature map, and angle ViT feature map into the first-layer meta-learner respectively, and output feature maps at different speeds.

[0103] Result prediction module: Aggregate the feature maps of different speeds through the second-layer meta-learner to obtain the gait prediction result of the wearer.

[0104] The gait prediction system for lower-limb exoskeleton wearers of the present invention designs a two-layer meta-learning structure including Meta-learner I and Meta-learner II. This structure can effectively extract the common features of sEMG signals and joint angles at different speeds, improving the generalization performance of the network at different speeds; a mapping from surface electromyogram signals to joint angles is established, which can accurately predict the gait of the wearer under complex motion conditions, effectively solving the challenge of multi-scenario gait prediction on small-sample data sets and improving the accuracy of gait prediction results.

[0105] On the other hand, the electronic device of the present invention includes: a processor, a memory, and a communication bus and a communication interface.

[0106] Wherein:

[0107] The processor, the memory, and the communication interface complete communication with each other through the communication bus.

[0108] The communication interface is used to communicate with other electronic devices or servers.

[0109] The processor is used to execute a program, and specifically can execute the steps of any one of the above-mentioned gait prediction methods for lower-limb exoskeleton wearers in the embodiments.

[0110] Specifically, the program may include program code, and the program code includes computer operation instructions.

[0111] The processor may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application. One or more processors included in the intelligent device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0112] The memory is used to store the program. The memory may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0113] The program can be specifically used to enable the processor to execute steps for implementing any one of the gait prediction methods for lower limb exoskeleton wearers described in the embodiments. For the specific implementation of each step in the program, reference can be made to the corresponding descriptions in the steps and units of any one of the gait prediction methods for lower limb exoskeleton wearers described above, which will not be elaborated here. Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the devices and modules described above can refer to the corresponding process descriptions in the foregoing method embodiments.

[0114] The method according to the embodiment of the present invention can be implemented in a server equipped with a central processing unit (CPU) and an image processing unit (COU).

[0115] So far, specific embodiments of the present invention have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown to achieve the desired result.

[0116] It should be noted that all directional indications (such as up, down, left, right, back...) in the embodiments of the present invention are only used to explain the relative positional relationship between components in a specific order (as shown in the drawings). If this specific order changes, the directional indications will change accordingly.

[0117] In the description of the present invention, the terms "first" and "second" are only used for the convenience of describing different components or names, and cannot be understood as indicating or implying an order relationship, relative importance, or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one such feature.

[0118] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0119] It should be noted that although the specific embodiments of the present invention have been described in detail in conjunction with the accompanying drawings, it should not be construed as a limitation on the protection scope of the present invention. Within the scope described in the claims, various modifications and deformations that can be made by those skilled in the art without creative labor still belong to the protection scope of the present invention.

[0120] The examples of the embodiments of the present invention are intended to concisely illustrate the technical features of the embodiments of the present invention, so that those skilled in the art can intuitively understand the technical features of the embodiments of the present invention, and are not an improper limitation of the embodiments of the present invention.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A gait prediction method for lower limb exoskeleton wearers, characterized in that, including: Obtain the sEMG signal and joint angle data of the wearer, and perform data preprocessing; Input the preprocessed sEMG signal into the ResNet feature extraction layer to obtain the sEMGResNet feature map; Perform image processing on the sEMGResNet feature map through a vision transformer to obtain the sEMG-ViT feature map; Input the preprocessed joint angle data into the vision transformer to obtain the angle ViT feature map; Input the sEMGResNet feature map, sEMG-ViT feature map, and angle ViT feature map into the first-layer meta-learner respectively, and output feature maps at different speeds; Perform aggregation processing on the feature maps at different speeds through the second-layer meta-learner to obtain the gait prediction result of the wearer.

2. The method according to claim 1, wherein The obtaining of the sEMG signal and joint angle data of the wearer includes: Record the sEMG signal and joint angle data of the wearer's lower limbs through the Trigno avant-garde sensor; Wherein, the Trigno avant-garde sensor integrates an electromyogram sensor and an inertial measurement unit.

3. The method according to claim 2, characterized in that The electromyogram sensor samples the sEMG signal at a frequency of 1926 Hz; The inertial measurement unit samples the joint angle data at a frequency of 148 Hz.

4. The method according to claim 2, wherein The data preprocessing includes: Perform DC component removal, band-pass filtering, full-wave rectification, and low-pass filtering on the sEMG signal.

5. The method according to claim 1, wherein The performing of image processing on the sEMGResNet feature map through a vision transformer to obtain the sEMG-ViT feature map includes: Extract multiple patches from the sEMG sequence in the sEMGResNet feature map through a sliding window method; Perform label embedding and position encoding on the multiple patches to obtain an input sequence; Input the input sequence into the vision transformer for data processing to obtain the sEMG-ViT feature map.

6. The method according to claim 5, wherein The position encoding is represented by sine and cosine functions: where pos represents the position; d model represents the model dimension; i represents the frequency of each dimension.

7. The method according to claim 5, wherein It also includes: Establish the relationship between different-dimensional information in the sEMG sequence through the multi-head self-attention mechanism in the vision transformer.

8. A gait prediction system for lower limb exoskeleton wearers, characterized in that, including: Data acquisition module: Obtain the sEMG signal and joint angle data of the wearer, and perform data preprocessing; Feature map generation module: Input the preprocessed sEMG signal into the ResNet feature extraction layer to obtain the sEMGResNet feature map; perform image processing on the sEMGResNet feature map through a vision transformer to obtain the sEMG-ViT feature map; input the preprocessed joint angle data into the vision transformer to obtain the angle ViT feature map; Feature map processing module: Input the sEMGResNet feature map, sEMG-ViT feature map, and angle ViT feature map into the first-layer meta-learner respectively, and output feature maps at different speeds; Result prediction module: Perform aggregation processing on the feature maps at different speeds through the second-layer meta-learner to obtain the gait prediction result of the wearer.

9. An electronic device, characterized in that, including: A memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the steps of a gait prediction method for a lower limb exoskeleton wearer according to any one of claims 1 to 7 are implemented.

10. A computer storage medium, characterized in that, A computer program is stored on the computer storage medium, and when the computer program is executed by a processor, the steps in a gait prediction method for a lower limb exoskeleton wearer according to any one of claims 1 to 7 are implemented.

Images