Rehabilitation lower limb exoskeleton eeg decoding method based on ensemble learning

Abstract

The application discloses a rehabilitation lower limb exoskeleton EEG decoding method based on ensemble learning, relates to the technical field of EEG decoding, and comprises the following steps: acquiring an EEG signal of a user's motor cortex through an EEG device; performing shallow layer processing on the EEG signal to obtain a first type of feature; constructing a deep layer processing network model of the EEG signal, performing deep layer processing on the EEG signal, and obtaining a second type of feature; combining the first type of feature and the second type of feature, training a classifier, and decoding the motor intention of the EEG signal; performing visual feedback on the user according to the decoded motor intention, then executing the decoded movement on a lower limb exoskeleton device, and obtaining proprioceptive feedback. The application combines shallow layer and deep layer neural networks to improve the performance of lower limb motor intention, performs continuous motor imagination experiments on the lower limb exoskeleton device, collects continuous EEG signals, and designs an experimental paradigm of online visual and proprioceptive feedback.

Description

[0001] This application is a divisional application. The application number of the original application is 202311366447.6, the application date is October 20, 2023, and the name of the invention is "An integrated rehabilitation lower limb exoskeleton EEG decoding method and system." Technical Field

[0002] The present invention relates to the technical field of electroencephalogram (EEG) decoding, and more particularly to an ensemble learning-based EEG decoding method for a lower limb rehabilitation exoskeleton. Background Art

[0003] While traditional rehabilitation training, such as physical therapy and occupational therapy, has been effective in restoring function, it is time-consuming and relies on the expertise of physical therapists. With the advancement of robotics, lower-limb exoskeletons have become an emerging treatment strategy, replacing traditional labor-intensive therapies. These exoskeletons can be used to train, enhance, assist, or replace human movement, thereby improving walking ability.

[0004] During rehabilitation, powered robots can provide high-intensity training and objective, quantifiable metrics. However, whether or not a patient actively participates in rehabilitation training affects the smooth progress of the rehabilitation process and is closely related to the ultimate rehabilitation outcome. To enhance rehabilitation outcomes, a series of improvements have been made. For example, devices equipped with virtual reality technology, such as the Lokomat, provide patients with a more engaging and motivating training experience, enhancing their sense of immersion and thus improving training effectiveness. Other rehabilitation interfaces, such as forearm crutches, wrist-worn devices, joysticks, and buttons, are also available, but these devices are only suitable for patients with normal upper limb function.

[0005] Brain-computer interfaces (BMIs) offer patients a new way to communicate with and control external devices as a natural form of interaction. However, BMI performance is hampered by numerous factors, including noise in EEG signals, high sensitivity to intra-trial data variability, and a heavy reliance on manually selected features. The introduction of a deep learning algorithm (DeepconvNet) with batch normalization, exponential linear units, and a clipping training strategy has improved its performance, surpassing that of the filter bank common spatial pattern (FBCSP). The advantage of FBCSP is that it can extract features using frequency band information, whereas ConvNets lack a dedicated feature extraction process and instead use features acquired during training. Convolutional networks have also outperformed EEGNets in other BMI paradigms, such as sensorimotor rhythm (SMR), P300 visual evoked potential, error-related negativity (ERN), and movement-related cortical potential (MRCP).

[0006] While other advanced deep learning algorithms, such as Transformer and Temporal Convolutional Networks (TCN), have also been applied to motor imagery EEG classification and visualization, research using these advanced deep learning algorithms for online decoding has not yet been conducted.

[0007] Therefore, how to provide an integrated learning-based EEG decoding method for rehabilitation lower limb exoskeleton for online and offline decoding of lower limb exoskeletons, enhance the sense of control over lower limb movements by providing visual and proprioceptive feedback, and thus improve the ability to generate reliable and accurate brain signals is a problem that technicians in this field urgently need to solve. Summary of the Invention

[0008] In view of this, the present invention provides an EEG decoding method for rehabilitation lower limb exoskeleton based on ensemble learning, which combines shallow and deep neural networks to improve the performance of lower limb movement intention. Continuous motor imagery experiments are performed on lower limb exoskeleton devices, and continuous EEG signals are collected. An experimental paradigm for online visual and proprioceptive feedback is designed.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: an integrated EEG decoding method for rehabilitation lower limb exoskeleton, comprising:

[0010] Obtain EEG signals from the user's motor cortex through an EEG device;

[0011] Performing shallow processing on the EEG signal to obtain a first type of feature;

[0012] Constructing a deep processing network model of the EEG signal, performing deep processing on the EEG signal to obtain the second type of features;

[0013] Merging the first and second features, and training a classifier to decode the movement intention of the EEG signal;

[0014] The user is given visual feedback based on the decoded movement intention, and then the decoded movement is executed on the lower limb exoskeleton device to obtain proprioceptive feedback.

[0015] The first type of features are superficial features, while the second type are deep features. Through proprioceptive feedback, users can perceive the position and movement of their own body parts, providing a more interactive experience. Users can adjust their movements based on their performance, thereby increasing their participation and interest.

[0016] Preferably, shallow processing of the EEG signal includes:

[0017] Filtering and wavelet transforming the EEG signal to obtain different frequency components of the EEG signal;

[0018] The different frequency components were processed by cospatial pattern analysis to separate the EEG signal into additional subcomponents with the largest variance differences between different categories;

[0019] The features of different frequency components that have been processed by common space pattern analysis are combined to obtain the first type of features; the first type of features are shallow features.

[0020] Preferably, the EEG signal of the user's motor cortex includes: offline sequence data, first online sequence data and second online sequence data;

[0021] Training a deep processing network model of the EEG signal using the offline sequence data;

[0022] The deep processing network model of the EEG signal is tested using the first online sequence data and the second online sequence data.

[0023] Preferably, the deep processing network model of the EEG signal includes: a deep convolution layer, a first temporal convolution layer, a second temporal convolution layer, a self-attention layer and a TCN layer;

[0024] Reducing the number of trainable parameters by using deep convolution to learn spatial filters through the deep convolutional layer;

[0025] Extracting feature maps of EEG signals in different frequency bands through the first temporal convolutional layer;

[0026] Extracting feature maps representing data at different time scales through the second temporal convolutional layer;

[0027] Determine the most effective motor imagery information through the self-attention layer;

[0028] All available temporal information is captured through the TCN layer, and after flattening, the extracted deep features are output and classified.

[0029] Preferably, the self-attention layer adopts a multi-head attention mechanism, performs a single attention head in different spaces, maps Q, K and V to different spaces through linear transformation, splices the attention results of different spaces, and performs a linear transformation to map them to the output space to obtain the final result.

[0030] Preferably, the process of concatenating the attention results of different spaces and performing a linear transformation to map them to the output space is as follows:

[0031] Apply the projection matrix of each attention head to the input Q, K and V respectively to obtain h different queries Q, keys K and values ​​V; the parameter matrix of the projection matrix includes

[0032] The self-attention calculation is performed for each attention head, and the output of h attention heads is generated, and the formula is as follows:

[0033]

[0034] Wherein, i∈(1,h); And Indicates that the input Q, K and V apply the projection matrix of each attention head respectively;

[0035] The outputs of these attention heads are spliced together, and the spliced result is weighted and processed for linear transformation to obtain the final output, and the formula is as follows:

[0036] MultiHead(Q,K,V)=Concat(head1,…,head h )W O ;

[0037] Wherein, W O Indicates the weight matrix.

[0038] Preferably, all available time information is captured through the TCN layer, and the extracted deep features are output after flattening and classification, including:

[0039] The EEG signal is preprocessed; for example, filtering and denoising to reduce the influence of noise on the classification performance.

[0040] The EEG signal is cut into time windows, and the corresponding motor intention category is labeled for each time window;

[0041] The TCN model is constructed, which is composed of multiple time convolution layers, and each time convolution layer uses a convolution kernel of different size to capture features of different time scales;

[0042] Global average pooling or global maximum pooling is applied to capture the information of the overall sequence;

[0043] The pooling output is connected to the fully connected layer to perform the classification task, and the prediction of different motor intention categories is output;

[0044] The TCN model is trained using the labeled training data, and the TCN model is optimized through the loss function and the optimization algorithm.

[0045] Preferably, the acquisition of the offline sequence data includes:

[0046] The first prompt time, the rest time period, the visual prompt time period, the motor imagination time period and the second prompt time are preset;

[0047] The user is prompted via a display screen. Starting from a first prompt time, the user undergoes a rest period, and then is instructed in the direction of movement via a visual prompt. The user performs offline motor imagery training according to the visual prompt. After the motor imagery period ends, a second prompt is given. The acquisition of offline sequence data for one trial is completed.

[0048] The first online sequence data is based on visual feedback, and an online test of visual feedback is performed during the motor imagery time period to obtain discrete visual feedback;

[0049] The second online sequence data is based on proprioceptive feedback, and visual feedback is performed during the movement imagery period, and then the decoded movement is executed on the lower limb exoskeleton device to obtain proprioceptive feedback.

[0050] Preferably, the method further comprises: using LOSO cross validation evaluation technology to perform performance evaluation;

[0051] Accuracy and Cohen's Kappa coefficient are used as performance indicators, as shown in formula (1) and formula (2), and the performance of the algorithm is evaluated by statistically analyzing the Cohen's Kappa coefficient k;

[0052]

[0053] Among them, TP represents a true positive example of a certain class, FP represents a false positive example, FN represents a false negative example, and TN represents a true negative example;

[0054]

[0055] Among them, P o Represents the classification accuracy, P e Represents the results of random classification.

[0056] Preferably, an integrated lower limb rehabilitation exoskeleton EEG decoding system comprises:

[0057] EEG equipment, used to obtain EEG signals from the user's motor cortex;

[0058] A first processing module is used to perform shallow processing on the EEG signal to obtain a first type of feature;

[0059] A second processing module is used to build a deep processing network model of the EEG signal, perform deep processing on the EEG signal, and obtain the second type of features;

[0060] A decoding module is used to combine the first and second features and feed them into a trained classifier (the first sequence of EEG signals is used to train the classifier) ​​to decode the movement intention of the EEG signal.

[0061] The feedback module is used to provide visual feedback to the user based on the decoded movement intention, and then the decoded movement is executed on the lower limb exoskeleton device to obtain proprioceptive feedback.

[0062] It can be seen from the above technical solution that compared with the existing technology, the present invention discloses an EEG decoding method for rehabilitation lower limb exoskeleton based on ensemble learning, including: obtaining the EEG signal of the user's motor cortex through an EEG device; shallowly processing the EEG signal to obtain a first type of feature; constructing a deep processing network model of the EEG signal, deeply processing the EEG signal to obtain a second type of feature; merging the first type of feature and the second type of feature, and training a classifier to decode the movement intention of the EEG signal; providing visual feedback to the user based on the decoded movement intention, and then executing the decoded movement on the lower limb exoskeleton device to obtain proprioception feedback.

[0063] The present invention combines shallow and deep neural networks to improve the performance of lower limb motor intention. Continuous motor imagery experiments were conducted on the lower limb exoskeleton devices independently developed by the laboratory, and continuous EEG brain wave signals were collected, and an experimental paradigm of online visual and proprioceptive feedback was designed. Three thinking tasks (left limb, right limb and rest) and two thinking tasks (movement intention and rest) were classified and performance evaluated. Unlike traditional machine learning or convolutional neural network technology, the present invention connects the spatial-spectral characteristics of wavelets with the temporal convolutional network feature representation that integrates the multi-head attention mechanism, thereby obtaining performance significantly higher than the chance level, indicating the feasibility of applying the integrated method to brain-controlled lower limb rehabilitation. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0065] Figure 1 This is an experimental flow chart provided for an embodiment of the present invention.

[0066] FIG2( a ) is a flowchart of an online test based on visual feedback provided by an embodiment of the present invention.

[0067] FIG2( b ) is a flowchart of an online test based on proprioception feedback provided by an embodiment of the present invention.

[0068] Figure 3 A flowchart of shallow processing of EEG signals provided by an embodiment of the present invention.

[0069] Figure 4 A deep processing flowchart of an EEG signal is provided for an embodiment of the present application.

[0070] Figure 5 A schematic diagram of the overall structure of a Transformer is provided for an embodiment of the present application.

[0071] Figure 6 A schematic diagram of an attention mechanism is provided for an embodiment of the present application.

[0072] Figure 7 A schematic diagram of multi-head attention is provided for an embodiment of the present application.

[0073] Figure 8 A schematic diagram of a CN residual network structure is provided for an embodiment of the present application.

[0074] Figure 9 A schematic diagram of an expanded convolution structure is provided for an embodiment of the present application.

[0075] Figure 10 A schematic diagram of an EEGNet structure is provided for an embodiment of the present application.

[0076] Figure 11 A schematic diagram of the average confusion matrix of an ATCNet and a reproduced model is provided for an embodiment of the present application.

[0077] Figure 12(a) is a schematic diagram of the average electroencephalogram signal of the Cz channel over time under visual feedback, provided for an embodiment of the present application.

[0078] and Figure 12(b) is a schematic diagram of the average electroencephalogram signal of the Cz channel over time under body feedback, provided for an embodiment of the present application. DETAILED DESCRIPTION

[0079] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0080] The embodiment of the present invention discloses an integrated EEG decoding method and system for rehabilitation lower limb exoskeleton. The method comprises: acquiring EEG signals; shallowly processing the EEG signals to obtain first-class features; deeply processing the EEG signals to obtain second-class features; merging the first-class features and the second-class features to decode the movement intention of the EEG signal; providing visual feedback to the user based on the decoded movement intention, and then executing the decoded movement on the lower limb exoskeleton device to obtain proprioceptive feedback. The embodiment of the present invention combines shallow and deep neural networks to improve the performance of lower limb movement intention, conducts continuous movement imagery experiments on the lower limb exoskeleton device, collects continuous EEG brainwave signals, and designs an experimental paradigm for online visual and proprioceptive feedback.

[0081] In a specific embodiment of the present invention, the embodiment of the present invention discloses an integrated rehabilitation lower limb exoskeleton EEG decoding method, including: an integrated rehabilitation lower limb exoskeleton EEG decoding method, including: obtaining the EEG signal of the user's motor cortex through an EEG device; shallowly processing the EEG signal to obtain a first type of feature; constructing a deep processing network model of the EEG signal, deeply processing the EEG signal to obtain a second type of feature; the first type of feature is the shallow feature, and the second type of feature is the deep feature; merging the first type of feature and the second type of feature, and training a classifier to decode the movement intention of the EEG signal; providing visual feedback to the user based on the decoded movement intention, and then executing the decoded movement on the lower limb exoskeleton device to obtain proprioception feedback.

[0082] Specifically, eight subjects (four males and four females, with an average age of 27.13 years) were selected for this study. All subjects had previously conducted brain-computer interface (BCI) experiments and performed left- and right-hand motor imagery tasks, but had no experience using similar devices for lower limb proprioception motor imagery. All subjects had normal or corrected-to-normal vision and reported no history of any known neurological or psychiatric disorders.

[0083] In terms of mechanical structure, the embodiment of the present invention uses a laboratory-made bionic three-degree-of-freedom lower limb exoskeleton device and a novel parallel mechanism for gait rehabilitation. In terms of algorithm, it combines the feedback control framework with the repetitive learning algorithm to enhance the anti-interference ability of the neural controller. The lower limb exoskeleton device is integrated with the electroencephalogram recording system to perform the rehabilitation process in an active and natural way, such as Figure 1The system includes EEG recording and a robotic system. A display and lower-limb exoskeleton provide visual and proprioceptive feedback, respectively. During the experiment, the subject stood on the exoskeleton, with their legs strapped to bionic knee joints, a wire harness, and their back suspended from a weight-reducing device. Visual cues were presented via a 50-inch display approximately 2 meters away, and control commands were sent using a Python script.

[0084] For ease of understanding, Figure 1 The process information shown is explained as follows, including CNN (Convolutional Neural Networks); MSA (multi-head self-attention); TCN (Temporal Convolutional Network); WPD (Wavelet Packet Decomposition); CSP (Common Spatial Pattern); Concat; HO (post-hoc tests); LOSO (Leave-One-Out Cross-Validation).

[0085] For wireless EEG signal recording, a NeuSenW system from Boricon (China) was selected. The system includes 64 built-in scalp electrodes arranged according to the 10 / 20 international system with a sampling rate of 1000 Hz. To acquire EEG signals from the motor cortex, peripheral channels were removed, retaining the following 26 EEG electrodes (FT7, FT8, FCz, FC1, FC2, FC3, FC4, FC5, FC6, T7, T8, Cz, C1, C2, C3, C4, C5, C6, TP7, TP8, CP1, CP2, CP3, CP4, CP5, and CP6). Unipolar EEG signals were amplified and transmitted to a computer (operating system: Windows 10, CPU: Intel Core i7, RAM: 16 GB) via an intelligent synchronization center. Another multi-parameter synchronizer was used to generate various trigger signals to synchronize the EEG acquisition system. Synchronization of EEG signals with the lower-limb exoskeleton was achieved by sending them to the exoskeleton's host computer interface via TCP / IP. The host computer then transmitted gait information to the lower-limb exoskeleton via several USB serial ports. The experimental protocol was designed based on the manufacturer's trigger interface and the EEGLAB open-source signal processing environment.

[0086] Before each experiment, the lower limb exoskeleton device is calibrated, including the maximum speed and acceleration of the actuators, to achieve a comfortable training experience. During the recording, the subjects cannot move any part of their body, especially not holding the handrails or shaking the rope. For safety reasons, if the interaction force exceeds the allowable value, the emergency stop button next to it can be pressed to perform an emergency stop operation, and a dedicated person will monitor the experimental process at all times. In addition, the preparation of the EEG acquisition system includes the positioning of the active electrodes and the application of conductive gel. The impedance of the electrodes must be checked to ensure that it is below 10k. The time required to disassemble the experimental setup is approximately 20 minutes.

[0087] Specifically, the EEG signal of the user's motor cortex includes: offline sequence data, first online sequence data and second online sequence data;

[0088] Training a deep processing network model of the EEG signal using the offline sequence data;

[0089] The deep processing network model of the EEG signal is tested using the first online sequence data and the second online sequence data.

[0090] Specifically, the acquisition of the offline sequence data includes:

[0091] Preset the first prompt time, rest time period, visual prompt time period, motor imagery time period and second prompt time;

[0092] The user is prompted via a display screen. Starting from a first prompt time, the user undergoes a rest period, and then is instructed in the direction of movement via a visual prompt. The user performs offline motor imagery training according to the visual prompt. After the motor imagery period ends, a second prompt is given. The acquisition of offline sequence data for one trial is completed.

[0093] The first online sequence data is based on visual feedback, and an online test of visual feedback is performed during the motor imagery time period to obtain discrete visual feedback;

[0094] The second online sequence data is based on proprioceptive feedback, and visual feedback is performed during the movement imagery period, and then the decoded movement is executed on the lower limb exoskeleton device to obtain proprioceptive feedback.

[0095] Specifically, as shown in Figures 2(a) and 2(b), three training paradigms were employed: Sequence 1 was an offline recording, Sequence V was visual feedback, and Sequence P was proprioceptive feedback. Each participant recorded three sequences according to the timeline shown in Figures 2(a) and 2(b): offline sequence data, first online sequence data, and second online sequence data. Each sequence was separated by two days. The experiment was divided into three sequences, each consisting of five rounds, each containing 60 trials. These 60 trials were divided into 30 left-foot and 30 right-foot movements, and were performed in a random order to prevent adaptation. The offline sequence data was used for offline training, while the first and second online sequence data were used for online testing. The offline sequence began with a 4-second rest period, which consisted of a fixation cross displayed on the screen. Subsequently, a cursor-like visual cue, indicating the direction of movement, was displayed on the screen for 2 seconds as a visual cue period. Afterward, a blank screen was displayed on the screen for 4 seconds as a motor imagery period to facilitate the motor imagery task. Each trial lasted a total of 10 seconds. At the end of each trial, a blue line was displayed as a second cue time, indicating the end of the trial, with a random interval of 1.5-3.5 seconds to prevent fatigue. Offline sequence data recording took approximately 50 minutes to complete.

[0096] Due to the limited number of subjects, the embodiment of the present invention does not perform cross-validation because the results may not be statistically reliable. Therefore, the first online sequence data and the second online sequence data are designed as online tests, wherein the first online sequence data uses visual feedback and the second online sequence data uses proprioceptive feedback. In both feedback modes, the subjects will also have visual feedback similar to the offline sequence. The movement of the progress bar still starts from the middle position, but the direction and speed at this time are determined by the output of the classifier at each time point (such as posterior probability, classification results, etc.). Depending on the user's performance, it does not necessarily reach both ends. There are two situations for the end of each trial. The first is that the trial ends after the brain-computer interface decodes the subject's intention, and the second is that it ends after a timeout, that is, it ends after a 5-second timeout if the intention is not decoded.

[0097] However, in the second sequence, the subjects received additional discrete visual feedback as the progress bar flashed several times, and in the third sequence, the subjects returned to the middle starting position after performing the decoded movement (movement imagination of the left or right leg) on ​​the lower limb exoskeleton device, at which time they received proprioceptive feedback. The total time required for the second and third sequences was approximately 60 and 70 minutes, respectively. Through proprioceptive feedback, the subjects can perceive the position and movement of their own body parts, and get a more interactive experience. The subjects can adjust their movements in time according to their own performance, thereby increasing the subjects' participation and interest.

[0098] Specifically, shallow processing of the EEG signal includes:

[0099] Filtering and wavelet transforming the EEG signal to obtain different frequency components of the EEG signal;

[0100] The different frequency components were processed by cospatial pattern analysis to separate the EEG signal into additional subcomponents with the largest variance differences between different categories;

[0101] The features of different frequency components that have been processed by common space pattern analysis are combined to obtain the first type of features; the first type of features are shallow features.

[0102] Specifically, Python MNE (Electroencephalogram (EEG) and Magnetoencephalogram (MEG) Analysis and Visualization Tool) 1.3, Tensorflow 2.11, and Matlab R2015b (by MathWorks, USA) combined with EEGLAB 13.5.4b toolbox were used for the online and offline data processing of EEG signals. Figure 1 A flow chart of the signal processing method is shown, including a deep processing part and a shallow processing part.

[0103] Specifically, such as Figure 3 Figure 2 shows the detailed architecture of the shallow layer. It includes notch filtering, spatial filtering, wavelet transform, and cospatial patterns. The EEG data is filtered using an 8th-order Butterworth bandpass filter with cutoff frequencies of 0.5 Hz and 50 Hz, and a 4th-order 50 Hz notch filter. The wavelet transform is used to decompose the signal into a set of wavelets. Compared to the traditional Fourier transform, which captures global frequency information, wavelet decomposition is useful in EEG applications because the signal has characteristic short-interval oscillations. The signal is decomposed using the 'db4' wavelet at five levels, and eight frequency band coefficients are selected from the 4-32 Hz range. Subsequently, cospatial patterns (CSP) are used to separate the signal into additional subcomponents with the greatest variance differences between different classes. The number of CSP components is four.

[0104] Embodiments of the present invention combine wavelet transform and common spatial pattern analysis (CSP) to extract and enhance relevant features in signals, particularly in the fields of brain-computer interface (BCI) and biomedical signal processing. This combination can help obtain better feature representations, thereby improving the performance of classification, recognition, or parsing tasks. Preprocessing begins by performing a wavelet transform on the original signal to obtain its different frequency components. This helps capture signal features across different frequency ranges. For data with more spatial information, such as EEG signals, CSP preprocessing can be performed to enhance task-relevant signal components. CSP generates a set of spatial filters that maximize the variance differences between different classes. Feature extraction is then performed. CSP is applied to each frequency subband of the wavelet transformed signal to enhance task-relevant features. For each frequency subband, CSP generates a set of spatial filters that project the signal into a specific space, maximizing the variance of certain classes while minimizing the variance of others. Feature combination is then performed, combining the features processed by CSP in different frequency subbands to obtain a comprehensive feature representation. This can be achieved by simply concatenating feature vectors, averaging, etc. Finally, classification is performed and the combined features are used to classify and identify the motion intention category.

[0105] Specifically, the deep processing network model of the EEG signal includes: a deep convolution layer, a first temporal convolution layer, a second temporal convolution layer, a self-attention layer and a TCN layer; the deep convolution layer is used to learn spatial filters to reduce the number of trainable parameters; the first temporal convolution layer is used to extract feature maps of the EEG signal in different frequency bands; the second temporal convolution layer is used to extract feature maps representing data at different time scales; the self-attention layer is used to determine the most effective motor imagery information; the TCN layer is used to capture all available time information, and the extracted deep features are output after flattening and classified.

[0106] Specifically, such as Figure 4 The figure shows a detailed description of the deep layer architecture, including traditional convolutional neural networks (CNNs), multi-head self-attention, and temporal convolutional networks. The deep network is inspired by the EEGNet series and further optimized through filter bank features and Transformer-based spatial features. Consider a single period of raw EEG data represented as R(C*T) and its corresponding labels y{L, R, O}, where C is the number of EEG channels, T is the number of time points, and Nc is the number of different categories; L, R, and O represent left leg, right leg, and rest, respectively. Specifically, Nc = 3.

[0107] Among them, EEGNet is a deep learning network architecture for processing electroencephalogram (EEG) data. Figure 10 The figure shows a deep learning architecture specifically designed for processing EEG data. It aims to extract features from EEG data and perform tasks such as classification, recognition, or regression. This architecture takes into account the characteristics of EEG data, such as temporal, spatial, and spectral information. It consists of data preprocessing layers, convolutional layers, pooling layers, flattening layers, fully connected layers, and an output layer. Lines represent the connectivity between the convolution kernels (called feature maps) between the input and output. The network begins with temporal convolutions (second column) to learn frequency filters, then uses depthwise convolutions (middle column) to connect to each feature map separately to learn frequency-specific spatial filters. Separable convolutions (fourth column) are a combination of depthwise convolutions that learn a temporal summary of each feature map separately, followed by a pointwise convolution that learns how to optimally combine these feature maps. The EEGNet architecture can be adapted and optimized for specific tasks and data. It has achieved excellent performance in many EEG-related tasks, such as brain-computer interfaces, emotion recognition, and disease diagnosis. The success of this architecture is partly due to its ability to automatically learn features from raw EEG data, eliminating the need for hand-crafted feature extractors.

[0108] Specifically, the deep processing network model for EEG signals consists of five layers: a deep convolutional layer, a first temporal convolutional layer, a second temporal convolutional layer, a self-attention layer, and a TCN layer. The deep convolutional layer extracts feature maps of the EEG signal at different frequency bands. The first temporal convolutional layer uses deep convolution to learn spatial filters and reduce the number of trainable parameters. The deep convolutional layer is followed by a first average pooling layer to reduce the sampling rate. The second temporal convolutional layer extracts feature maps to represent data at different time scales. The second temporal convolutional layer is followed by a second average pooling layer with a size of P2 to reduce the sampling rate. The fourth layer is a self-attention module that helps the model focus on the most effective motor imagery information. Multi-head attention helps focus on multiple locations and produce multiple attention representations. The fifth layer is a TCN network that further utilizes temporal information. This embodiment of the present invention stacks L residual blocks and selects a receptive field size that is greater than 17, allowing the TCN to capture all available temporal information.

[0109] The outputs of all windows are concatenated and fed into the SoftMax classifier. This helps to effectively increase data and improve classification accuracy.

[0110] The deep processing network model of EEG signals was trained and tested on a single GPU using Nvidia GTX20708GB and the TensorFlow framework. The number of trainable parameters was 1000, and the training was performed for 3000 epochs with a training time of about 30 minutes. The prediction time for a one-second EEG data segment was about 0.05 seconds. For all experiments, the embodiment of the present invention used the following training configuration. The weights were initialized using the Glorot uniform initializer, and the training was performed using the Adam optimizer with a learning rate of 0.0009, a batch size of 64, and categorical cross entropy loss was used for training within 1000 epochs with a stopping condition of 300. The hyperparameters are shown in Table 1. The hyperparameter settings apply to all subjects, whether in a specific subject scenario or an independent subject scenario. Where C = the number of EEG channels, T = the number of time samples, F1 = the number of temporal filters, F2 = the number of spatial filters, K1 = the convolution kernel size of the first convolution layer, K T = convolution kernel size of TCN module, F T = the number of filters in the TCN module. Here, T = 1125 time points, C = 22 EEG channels, Nc = 3 motor imagery categories, and I = 5184 motor imagery epochs.

[0111] Table 1

[0112]

[0113] Specifically, the self-attention layer adopts a multi-head attention mechanism, performs a single attention head in different spaces, maps Q, K and V to different spaces through linear transformation, splices the attention results of different spaces, and performs a linear transformation to map them to the output space to obtain the final result.

[0114] The steps of the whole process are as follows:

[0115] (1) Apply the projection matrix of each attention head to the inputs Q, K, and V to obtain h different queries Q, keys K, and values ​​V; the parameter matrix of the projection matrix includes

[0116] (2) Perform self-attention calculation on each attention head to generate the output of h attention heads, namely head1 to head h , the formula is as follows:

[0117]

[0118] Where, i∈(1,h); and Represents the projection matrix of each attention head applied to the inputs Q, K, and V respectively;

[0119] (3) Concatenate the outputs of these attention heads together, which increases the dimension of the data. The concatenated result is a tensor with dimensions h*d_model. The concatenated result is combined with the weight matrix W. O Multiply them together to perform a linear transformation and obtain the final output. The formula is as follows:

[0120] MultiHead(Q,K,V)=Concat(head1,...,head h )W O .

[0121] Among them, W O represents the weight matrix.

[0122] Specifically, the parameter matrix contains

[0123] represents the query projection matrix of the i-th attention head is a matrix over the real field with dimension d model ×d k , where d model is the hidden layer dimension of the model, and d k is the dimension of the key vector; this projection matrix is ​​used to map the input query vector Q to a specific subspace for attention calculation;

[0124] represents the key projection matrix of the i-th attention head; is a matrix over the real field with dimension d model ×d k , where d model is the hidden layer dimension of the model, and d k is the dimension of the key vector; this projection matrix is ​​used to map the input key vector K to a specific subspace for attention calculation;

[0125] represents the numerical projection matrix of the i-th attention head; is a matrix over the real field with dimension d model ×d k , where d model is the hidden layer dimension of the model, and d k Is the dimension of the key vector. This projection matrix is ​​used to map the input numerical vector V to a specific subspace for attention calculation;

[0126] Represents the output weight matrix W O is a matrix over the real field with dimension hd v ×d model , where hd v is the dimension of the numeric vector, and d model is the hidden layer dimension of the model; this weight matrix is ​​used to map the output of the multi-head attention to the final model output dimension.

[0127] Specifically, the Transformer model is a model for machine translation proposed by Google in 2017. It completely abandons the structure of traditional recurrent neural networks and adopts a structure based entirely on the attention mechanism, which has achieved quite remarkable results. Since then, it has made the full attention mechanism model design model spread from the field of NLP to the field of computer vision. The Transformer model structure is as follows: Figure 5 As shown in the figure, the encoder and decoder have similar structures. The encoder operates from the bottom up: the input first passes through the embedding layer, followed by a positional encoding. After multi-head attention, it passes through a residual connection and LayerNorm, and finally passes through a feedforward neural network. The block consisting of the positional encoding, multi-head attention, and feedforward neural network is called a Transformer block, and the encoder can be composed of several Transformer blocks. The decoder is similar. Because the Transformer uses a self-attention mechanism, it lacks positional information compared to traditional recurrent neural networks such as RNNs (Recurrent Neural Networks), LSTMs (Long Short Term Memory), and GRUs (Gate Recurrent Units). Because the attention mechanism calculates the attention weight between the current word and every character in the entire sentence, the calculated attention weight remains the same even if the entire sentence is shuffled. Therefore, to assign positional information, the input information needs to be positionally encoded.

[0128] Thanks to its innovative self-attention and multi-head attention mechanisms, the Transformer has achieved revolutionary results in natural language processing, including models such as BERT (Bidirectional Encoder Representation from Transformers) and GPT (Generative Pre-Training). In addition to NLP (natural language processing), the Transformer has also been applied to other fields such as computer vision and speech recognition.

[0129] Specifically, ① Attention mechanism

[0130] The principle of attention mechanism is as follows Figure 6 As shown, the weighted average is calculated using the input (query, q) and the output (key-value, kv) in the model, and the output matrix is:

[0131]

[0132] ②Multi-head attention mechanism

[0133] The principle of Multi-Head Attention is as follows Figure 7 As shown in Figure 2, multiple attention heads are used simultaneously during the attention calculation process to learn different attention weights from different projection subspaces. This helps the model capture information in the input sequence in different semantic spaces and provide a more comprehensive understanding of the context.

[0134] Specifically, the TCN layer captures all available temporal information, flattens it, and outputs the extracted deep features for classification, including:

[0135] Perform preprocessing on the EEG signal; for example, filtering and denoising to reduce the impact of noise on classification performance.

[0136] Cutting the EEG signal into time windows and labeling each time window with a corresponding movement intention category;

[0137] Construct a TCN model, which consists of multiple temporal convolutional layers, each of which uses convolution kernels of different sizes to capture features at different time scales;

[0138] Apply global average pooling or global maximum pooling to capture the information of the entire sequence;

[0139] Connect the pooling output to the fully connected layer to perform the classification task and output predictions for different motion intent categories;

[0140] The TCN model is trained using labeled training data and optimized using a loss function and an optimization algorithm.

[0141] Specifically, a temporal convolutional network (TCN) refers to a temporal convolutional network, a new type of algorithm that can be used to solve time series prediction. This algorithm was first proposed by Lea et al. in 2016 while researching video action segmentation. Generally speaking, this conventional process involves two steps: first, low-level features are computed using a CNN (usually encoding spatiotemporal information). Second, these low-level features are fed into a classifier (usually a recurrent neural network) that captures high-level temporal information. The main drawback of this approach is that it requires two separate models.

[0142] The TCN model constructs a residual block to replace the convolution layer. Figure 8 As shown in Figure 1, a residual block contains two layers of convolution and nonlinear mapping, and WeightNorm and Dropout are added to each layer to regularize the network.

[0143] Dilated causal convolution Figure 9 As shown in the figure, dilated convolution allows for staggered sampling of the input during convolution, with the sampling rate controlled by d in the figure. d = 1 in the bottom layer means that every point in the input is sampled, while d = 2 in the middle layer means that every second point in the input is sampled. Generally speaking, the value of d increases with the number of layers. Therefore, dilated convolution causes the effective window size to grow exponentially with the number of layers. This approach allows convolutional networks to achieve a large receptive field using fewer layers.

[0144] The advantage of TCN is that it can automatically learn time-dependent features from sequential data, eliminating the need for manual feature extraction. Therefore, it is suitable for EEG signal classification. The implementation process begins with signal preprocessing, such as filtering and denoising, to reduce the impact of noise on classification performance. Second, data segmentation and labeling are performed, segmenting the EEG signals into appropriate time windows and labeling each window with the corresponding motion intent category. Third, the TCN model is constructed, consisting of a series of temporal convolutional layers. Each convolutional layer uses kernels of different sizes to capture features at different time scales. Convolutional layers may be interspersed with pooling layers or other nonlinear activation functions to extract and learn features. Fourth, global average pooling: After the convolutional layers, global average pooling or global max pooling can be applied to capture information from the entire sequence. Fifth, the classification layer connects the pooled output to a fully connected layer, which performs classification and outputs predictions for different action categories. Sixth, training and optimization are performed: the TCN model is trained using labeled training data, employing an appropriate loss function and optimization algorithm. Finally, evaluation and testing are performed.

[0145] DeepConvNet and ShallowConvNet are both deep convolutional neural network architectures used for EEG classification tasks. They represent deeper and shallower network structures, respectively, for extracting features from EEG signals and performing classification.

[0146] "DeepConvNet" refers to a deep convolutional neural network, first proposed in a paper by Schirrmeister et al. DeepConvNet typically consists of multiple convolutional and pooling layers, as well as several fully connected layers and activation functions. This architecture utilizes convolutional layers to capture local features in sequential data, while pooling layers reduce the data dimensionality while preserving key information. DeepConvNet's primary application in brain-computer interfaces is EEG signal classification, such as classifying different motor intentions into distinct categories.

[0147] ShallowConvNet is a relatively shallow convolutional neural network architecture proposed by Schirrmeister et al. in their paper. Compared to DeepConvNet, it uses fewer convolutional layers and parameters and focuses primarily on the local spatiotemporal features of EEG signals. This network structure is designed to improve computational efficiency while maintaining reasonable classification performance. ShallowConvNet has also been applied to EEG signal classification tasks, such as classifying different motor intentions.

[0148] In summary, both DeepConvNet and ShallowConvNet are deep convolutional neural network architectures designed for EEG signal classification tasks. They differ in network depth and structure, but both aim to extract useful time-frequency features from EEG signals for classification. The choice of architecture depends on the task requirements, computing resources, and performance requirements.

[0149] Specifically, it also includes: using LOSO cross-validation evaluation technique to evaluate performance;

[0150] Accuracy and Cohen's Kappa coefficient are used as performance indicators, as shown in formula (1) and formula (2), and the performance of the algorithm is evaluated by statistically analyzing the Cohen's Kappa coefficient k;

[0151]

[0152] Among them, TP represents a true positive example of a certain class, FP represents a false positive example, FN represents a false negative example, and TN represents a true negative example;

[0153]

[0154] Among them, P oRepresents the classification accuracy, P e Represents the results of random classification.

[0155] Specifically, the method proposed in the embodiment of the present invention has been examined in two evaluation modes: subject-specific and subject-independent. In the subject-specific analysis, the embodiment of the present invention uses the first sequence data of each subject to train the model, and tests it on the data of the second and third sequences of the same subject. As for the independent evaluation, it is usually called "cross-subject validation", or "cross-subject evaluation", that is, the "Leaving One Subject Out" (LOSO) evaluation method, which is used for each sequence. The model is trained and evaluated through several folds, and the number of folds is equal to the number of subjects. For each fold, one subject is used for evaluation, and the other subjects are used for training. The LOSO evaluation technique ensures that the model is evaluated on different subjects that do not appear in the training data. It is worth noting that in LOSO, it is necessary to avoid using cross-sequence data to prevent mixed noise across sequences.

[0156] Specifically, the performance evaluation of the method proposed in the embodiment of the present invention uses accuracy and Cohen's Kappa coefficient as performance indicators, as shown in Formula (1) and Formula (2). In addition, the embodiment of the present invention also performs a statistical analysis on the Cohen's Kappa coefficient k to evaluate the performance of the algorithm.

[0157]

[0158] Among them, TP represents a true positive example (True Positive) of a certain class, FP represents a false positive example (False Positive), FN represents a false negative example (False Negative), and TN represents a true negative example (True Negative).

[0159]

[0160] Among them, P o Represents the classification accuracy, P e Represents the results of random classification.

[0161] First, the present invention evaluated the learning effect to determine whether there was a significant performance improvement with the increase in training sequences. Furthermore, the present invention performed feature analysis to assess possible differences in the discriminative features between the two feedback modalities. To estimate the impact of EEG frequency band range, the present invention calculated the scalp energy distribution by averaging the discriminative energy across different frequency bands in the two sequences.

[0162] Next, the proposed method was compared with the performance of a classic machine learning algorithm, FBCSP-LDA, and three advanced convolutional neural network architectures: EEGNet, EEG-TCNet, and ATCNet. Because the proposed method in this embodiment consists of both shallow and deep layers, a culling analysis was performed to measure the effectiveness of each algorithm module.

[0163] Finally, a binary classification of motor imagery and resting (baseline) EEG signals was performed to assess learning outcomes. Classifiers were used to identify features (channel and frequency pairs) that distinguished motor imagery from resting under both feedback conditions. An optimized classifier was constructed for each subject in each sequence to evaluate these features and EEG modalities. Furthermore, paired-sample t-tests were performed to assess significant differences in online decoding performance. The significance level was set at 0.05. In the event of significant differences, a post-hoc Tukey correction was applied to correct for statistical errors. This correction helps control the false positive rate that may occur when multiple comparisons are made, ensuring the statistical credibility of the conclusions.

[0164] Specifically, Table 2 shows the classification accuracy (%) and kappa values ​​for all subjects in the specific subject and independent subject scenarios. Table 2 shows the real-time brain-computer interface (BMI) performance of the participants in the online sequence. The average accuracy across all subjects was 56.84 ± 4.14% in the visual feedback sequence and 60.27 ± 3.51% in the proprioceptive feedback sequence. The accuracy was significantly higher in the proprioceptive feedback sequence than in the visual feedback sequence (one-tailed two-sample t-test, p < 0.05, Bonferroni correction).

[0165] Furthermore, the average accuracy across all subjects was 53.97 ± 3.07% in LOSO 1, 54.54 ± 1.79% in LOSO 2, and 55.39 ± 2.49% in LOSO 3. There was no statistically significant difference in accuracy between the various LOSO methods (two-sample t-test, p > 0.05, Bonferroni correction).

[0166] Furthermore, the present invention calculated the randomness level based on the three-category classification. The probability of each class being guessed as a third class was 0.33, resulting in a randomness level accuracy of 33.3%. The classification accuracy of sequences V and P was significantly higher than the randomness level (one-tailed two-sample t-test, p < 0.05, Bonferroni correction). The performance of the LOSO session was also significantly higher than the randomness level (one-tailed two-sample t-test, p < 0.05, Bonferroni correction).

[0167] Table 2

[0168]

[0169] As shown in Table 3, the three-class classification accuracy and kappa values ​​(3 categories, i.e., left, right, and resting) of the proposed model and other methods were averaged for all subjects in each specific subject and independent subject context, respectively. Table 3 summarizes the accuracy and K value of the proposed model, compared with the reproduced EEGNet, EEG-TCNet, and ATCNet. The results of the reproduced model are based on the hyperparameters given in the original article, while the preprocessing, training, and evaluation followed the steps proposed in the embodiments of the present invention. The performance of all subjects was better than EEGNet, EEG-TCNet, and ATCNet, with an improvement of up to 4% and 2% in the individual-specific and individual-independent modes, respectively. The performance of all methods was also significantly higher than the random level (one-tailed two-sample t-test, p < 0.05, Bonferroni correction). In addition, the average confusion matrix of ATCNet and the reproduced model is shown in Figure 3. Figure 11 As shown in Figure 2, ATCNet performs better than other models in decoding all MI categories.

[0170] Table 3

[0171]

[0172] Table 4 shows the accuracy and kappa scores for the two-category classification (i.e., imagery and resting) replicated for the proposed model and other methods. These metrics are averaged across all subjects in both the individual subject context and the independent subject context. Table 4 summarizes the accuracy and kappa scores for the proposed model compared with the replicated EEGNet, EEG-TCNet, FBCSP, and ATCNet for both resting and moving motion classification. Across all subjects, the average accuracy for the visual feedback sequences was 55.42%, and the average accuracy for the proprioceptive feedback sequences was 64.20%. The chance level was calculated based on a binomial distribution, resulting in a 50% chance accuracy. Classification accuracy for both the visual feedback and proprioceptive feedback sequences was significantly above chance (one-tailed two-sample t-test, p < 0.05, Bonferroni correction). The performance of the leave-one-out sequence, both in terms of accuracy and kappa, was also significantly above chance (one-tailed two-sample t-test, p < 0.05, Bonferroni correction). As shown in Figure 12(a) and Figure 12(b), the average EEG signals of the Cz channel in the two feedback modes change over time.

[0173] Table 4

[0174]

[0175] To realize natural, intuitive, and user-friendly lower-limb exoskeleton brain-machine interface (BMI) systems, a comprehensive understanding of the neural mechanisms of lower-limb movement is essential. Neural signatures can be decoded from movement execution or motor imagery. Movement-related cortical potentials (MRCPs) are commonly used to detect motor intention during movement planning and execution, and these intentions can be detected approximately 1.5 to 2 seconds before the actual movement begins. Existing techniques have detected ankle dorsiflexion movement execution using manifold methods and linear discriminant analysis (LDA) as classifiers. Further studies have investigated factors influencing lower-limb movement intention, including movement type (dorsiflexion or plantar flexion), limb side (left or right leg), processing method (MRCP-based time series analysis, i.e., movement-related cortical potentials, or SMR-based frequency domain estimation, i.e., sensorimotor rhythms), and frequency bands (e.g., delta, theta, mu, beta, and the MRCP band at [0.11] Hz). During the offline-to-online BMI conversion process, detection delay is a common problem for this EEG modality. This is primarily due to the use of causal spectral filters, which often introduce significant system latency. In rehabilitation systems, short latency is crucial to enhance neural repair effectiveness.

[0176] Motor imagery (MI) activity is often used to control brain-computer interface (BMI) systems. Motor imagery occurs when imagining performing an action, but it can also be observed when the action is actually performed. Movement-related cortical potential (MRCP) activity occurs in the frequency band of 0-5Hz, while motor imagery activity is observed between 8-30Hz. These frequencies fall in the delta (<4Hz), theta (4-7Hz), mu (8-12Hz), and beta (13-30Hz) frequency bands, respectively. Therefore, for a specific user, it is necessary to first evaluate whether the user's brain activity frequency band is most intense in the MRCP frequency or in the MI frequency, which means that each time the user's intention is identified, the frequency band, i.e., the feature, needs to be selected based on the user's performance.

[0177] For controlling a rehabilitation exoskeleton, identifying the EEG intention of the lower-limb exoskeleton (i.e., imagining the movements of the left and right legs) and using it to control the device is the most intuitive and natural approach. Motor imagery is a cognitive activity that involves a virtual rehearsal of actual movement by internally reactivating any first-person motor representation. In this process, individuals reproduce and simulate various aspects of actual movement, including the details, sequence, and movement path, through internal neural activation and brain simulation, without producing explicit motor output. This cognitive rehearsal mechanism enables individuals to simulate and recreate motor behaviors in their brains, thereby enabling mental training and preparation for motor planning, coordination, and skill without the need for actual muscle movement. Furthermore, in the cerebral cortex, neuronal clusters specific to lower-limb movement are relatively limited in space. This may be because lower-limb movement does not involve complex neural networks and coordinated motor control mechanisms, resulting in a small number of neurons in the neural regions specific to each leg. Therefore, it is difficult to identify and capture specific lower limb movement intentions in a relatively small brain area to achieve more precise movement control and guidance. In the prior art, it has been proven that it is possible to distinguish between imagined left and right lower limb movements, with a classification accuracy of 63%. The main information comes from the alpha and beta frequency bands of electrodes C1, C2, PO3 and PO4, and the accuracy obtained in the embodiment of the present invention is within this recognition range. The introduction of more advanced methods may bring further performance improvements. Other lower limb experimental paradigms include pre-gait movement, gait cycle and movement coordination. Decoding kinematic parameters such as velocity information from lower limb joint movement can achieve continuous trajectory reconstruction.

[0178] Another important topic involves lower-limb powered robotics. Repeated rehabilitation training is crucial for the brain's ability to reshape neural circuits and adapt to new situations. The lower-limb exoskeleton devices proposed in the present invention have been shown to significantly improve walking ability, balance, and muscle strength in individuals with lower-limb injuries or disabilities.

[0179] Finally, individual differences in electroencephalogram (EEG) data affect the reusability and generalization ability of the model. Within the same subject, the performance of the model is significantly better than the LOSO method (leave one out of the group). Transfer learning can adjust the model to be more suitable for small-scale data sets while maintaining the ability to learn individual differences. Commonly used transfer learning methods in EEG signal analysis include domain adaptation, improved CSP algorithm, deep neural network (DNN) algorithm and subspace learning. It should be noted that in addition to being applied to processing raw EEG data, deep learning can also be further optimized to achieve artifact detection and removal of more neural signals.

[0180] Specifically, an integrated lower limb rehabilitation exoskeleton EEG decoding system includes:

[0181] EEG equipment, used to obtain EEG signals from the user's motor cortex;

[0182] A first processing module is used to perform shallow processing on the EEG signal to obtain a first type of feature, i.e., shallow feature;

[0183] The second processing module is used to build a deep processing network model of the EEG signal, perform deep processing on the EEG signal, and obtain the second type of features, i.e., deep features;

[0184] A decoding module is used to combine the first and second features and feed them into a trained classifier (the first sequence of EEG signals is used to train the classifier) ​​to decode the motor intention of the EEG signal.

[0185] The feedback module is used to provide visual feedback to the user based on the decoded movement intention, and then the decoded movement is executed on the lower limb exoskeleton device to obtain proprioceptive feedback.

[0186] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0187] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for EEG decoding of lower limb exoskeleton based on ensemble learning, characterized in that: The following steps are involved: Obtain EEG signals from the user's motor cortex through an EEG device; Perform shallow processing on the EEG signal to obtain the first type of features; Construct a deep processing network model of EEG signals, perform deep processing on EEG signals, and obtain the second type of features; Combine the first and second category features and train a classifier to decode the movement intention of the EEG signal; The user is given visual feedback based on the decoded movement intention, and then the decoded movement is executed on the lower limb exoskeleton device to obtain proprioceptive feedback; The deep processing network model includes a TCN layer, which is used to automatically learn time-related features from sequence data, capture all available temporal information, and output the extracted deep features after flattening for classification. The TCN layer is a temporal convolutional network that replaces one convolution layer with a residual block. A residual block contains two layers of extended causal convolution and nonlinear mapping. WeightNorm and Dropout are added to each layer to regularize the network. The deep processing network model of EEG signals also includes: deep convolution layer, first temporal convolution layer, second temporal convolution layer, and self-attention layer; Using deep convolution to learn spatial filters through deep convolutional layers to reduce the number of trainable parameters; Extract feature maps of EEG signals in different frequency bands through the first temporal convolutional layer; The second temporal convolution layer extracts feature maps representing data at different time scales; Determine the most effective motor imagery information through the self-attention layer; All available temporal information is captured through the TCN layer, and after flattening, the extracted deep features are output and classified, including: Preprocessing of EEG signals; Cut the EEG signal into time windows and label each time window with the corresponding movement intention category; Construct a TCN model, which consists of multiple temporal convolutional layers, each of which uses convolution kernels of different sizes to capture features at different time scales; Apply global average pooling or global maximum pooling to capture the information of the entire sequence; Connect the pooling output to the fully connected layer to perform the classification task and output predictions for different motion intent categories; Use labeled training data to train the TCN model, and optimize the TCN model through loss function and optimization algorithm.

2. The EEG decoding method for lower limb rehabilitation exoskeleton based on ensemble learning according to claim 1 is characterized in that: Shallow processing of EEG signals includes: Filter and wavelet transform the EEG signal to obtain different frequency components of the EEG signal; The different frequency components were processed by cospatial pattern analysis to separate the EEG signal into additional subcomponents with the largest variance differences between different categories; The features processed by co-spatial pattern analysis in different frequency components are combined to obtain the first type of features.

3. The EEG decoding method for lower limb rehabilitation exoskeleton based on ensemble learning according to claim 1, characterized in that: The EEG signal of the user's motor cortex includes: offline sequence data, first online sequence data, and second online sequence data; Train the deep processing network model of EEG signals through offline sequence data; The deep processing network model of the EEG signal is tested using the first online sequence data and the second online sequence data.

4. The EEG decoding method for lower limb rehabilitation exoskeleton based on ensemble learning according to claim 1, characterized in that: The self-attention layer adopts a multi-head attention mechanism, which performs linear transformation on Q, K and V in different spaces for a single attention head. The attention results in different spaces are concatenated and then linearly transformed and mapped to the output space to obtain the final result.

5. The method for EEG decoding of a lower limb exoskeleton based on ensemble learning according to claim 4, characterized in that: The process of concatenating the attention results of different spaces and performing a linear transformation to map them to the output space is as follows: Apply the projection matrix of each attention head to the input Q, K and V respectively to obtain h different queries Q, keys K and values ​​V; the parameter matrix of the projection matrix includes Perform self-attention calculation on each attention head to produce the output of h attention heads, as follows: Where, i∈(1,h); and Represents the projection matrix of each attention head applied to the inputs Q, K, and V respectively; The outputs of these attention heads are concatenated together and weighted to perform a linear transformation to obtain the final output. The formula is as follows: MultiHead(Q,K,V)=Concat(head1,…,head h )W O ; Among them, W O represents the weight matrix.

6. The method for EEG decoding of a lower limb exoskeleton based on ensemble learning according to claim 3, characterized in that: Offline sequence data acquisition includes: Preset the first prompt time, rest time period, visual prompt time period, motor imagery time period and second prompt time; The user is prompted via the display screen. Starting from the first prompt time, the user goes through a rest period, and then is instructed by visual prompts in the direction of movement. The user performs offline motor imagery training based on the visual prompts. After the motor imagery period ends, a second prompt is given. The acquisition of offline sequence data for one trial is completed. The first online sequence data is based on visual feedback, and an online test of visual feedback is performed during the motor imagery period to obtain discrete visual feedback; The second online sequence data is based on proprioceptive feedback, with visual feedback during the motor imagery period, and then the decoded movement is executed on the lower limb exoskeleton device to obtain proprioceptive feedback.

7. The EEG decoding method for lower limb rehabilitation exoskeleton based on ensemble learning according to claim 1, characterized in that: Also includes: The LOSO cross-validation evaluation technique was used for performance evaluation; Accuracy and Cohen's Kappa coefficient are used as performance indicators, as shown in formula (1) and formula (2), and the performance of the algorithm is evaluated by statistically analyzing the Cohen's Kappa coefficient k; Among them, TP represents a true positive example of a certain class, FP represents a false positive example, FN represents a false negative example, and TN represents a true negative example; Among them, P o Represents the classification accuracy, P e Represents the results of random classification.

Images