Hand rehabilitation method and system based on EEG-sEMG signal combined control

Abstract

This application discloses a hand rehabilitation method and system based on joint control of EEG-sEMG signals. The method includes: simultaneously acquiring and preprocessing EEG and sEMG signals; feeding the EEG signals into a multi-scale adaptive temporal convolutional network, and obtaining EEG features through spatial and temporal attention enhancement; inputting the sEMG signals into fatigue-invariant pathways to extract stable movement features and into fatigue-sensitive pathways to estimate the degree of fatigue; bidirectionally enhancing the bimodal features through enhanced co-attention; calculating fusion weights based on fatigue estimates and bimodal confidence levels using dynamic weighted gating, while simultaneously using a cross-modal compensation network to predict and compensate for degenerative modal features, and outputting the weighted fusion and classified movement intentions; and a lower-level controller adjusting the air chamber pressure of each finger of a pneumatic soft glove according to the movement intention using a PID closed-loop strategy to execute corresponding rehabilitation movements. This application can achieve robust recognition and precise rehabilitation training of typical hand grasping movement intentions.

Description

Technical Field

[0001] This invention relates to the field of medical rehabilitation technology, specifically to a hand rehabilitation method and system based on EEG-sEMG signal combined control, which is suitable for active rehabilitation training of patients with hand motor dysfunction caused by neurological diseases such as stroke. Background Technology

[0002] In recent years, the number of patients with hand motor dysfunction due to neurological diseases such as stroke has been increasing year by year. The loss of fine motor function in the hand seriously affects patients' daily living abilities and social participation. Clinical studies have shown that introducing patients' voluntary motor awareness in the early stages of rehabilitation training can significantly improve the efficiency of neural function reorganization through the mechanism of neuroplasticity.

[0003] In motor intention recognition, electroencephalography (EEG) and surface electromyography (sEMG) are currently the two mainstream types of bioelectrical signals. They reflect the human motor control process from the central nervous system and peripheral muscles, respectively, but each has inherent limitations when used alone. A single EEG signal has weak amplitude and low signal-to-noise ratio, making it susceptible to interference from various artifacts such as eye movements and electrocardiograms, resulting in a significant upper limit to recognition accuracy. While a single sEMG signal has high accuracy in recognizing motor execution states, its effective signal amplitude is severely insufficient for patients with severe movement disorders. Furthermore, muscle fatigue during prolonged rehabilitation training leads to signal amplitude attenuation and spectral drift, significantly reducing recognition performance. Existing EEG-sEMG multimodal fusion methods generally assume stable signal distribution and lack adaptive adjustment mechanisms to address the modal reliability imbalance caused by physiological fatigue.

[0004] In terms of rehabilitation implementers, traditional rigid hand exoskeletons are uncomfortable to wear and pose a risk of secondary injury. Pneumatic soft gloves based on the principle of pneumatic artificial muscle (PAM) are made of highly flexible materials and can conform well to the natural shape of the hand. However, existing systems have problems such as insufficient precision in air chamber pressure control, slow dynamic response, and significant overshoot during inflation and deflation.

[0005] Therefore, there is an urgent need for an intelligent hand rehabilitation solution that can sense and respond to the patient's active movement intentions, has fatigue self-adaptation capabilities, and can accurately execute rehabilitation movements. Summary of the Invention

[0006] The purpose of this invention is to provide a hand rehabilitation method and system based on EEG-sEMG signal joint control, to solve the technical problems of insufficient accuracy of single-modal signal recognition, performance degradation due to muscle fatigue, lack of adaptive adjustment in multimodal fusion, and poor control precision of rehabilitation actuators in existing technologies. Through the synergy of EEG and sEMG dual-modal fusion, fatigue-sensing dynamic weighted fusion decision-making, and PID closed-loop pressure control with a pneumatic soft glove, robust recognition and precise rehabilitation training of typical hand grasping movements are achieved.

[0007] To achieve the above objectives, this application adopts the following technical solution:

[0008] In a first aspect, this application provides a hand rehabilitation method based on combined control of EEG-sEMG signals, comprising the following steps: Simultaneously collect electroencephalogram (EEG) and electromyography (EMG) signals during the patient's hand movements, and preprocess the collected EEG and EMG signals respectively; The preprocessed EEG signal is input into a multi-scale adaptive temporal convolutional network to extract EEG temporal features at multiple time scales. The EEG temporal features are then enhanced through spatial attention and temporal attention mechanisms to obtain EEG feature vectors. The preprocessed surface electromyography (EMG) signals are input into the fatigue-invariant pathway and the fatigue-sensitive pathway, respectively. The fatigue-invariant pathway extracts temporal features and performs L2 normalization to obtain stable movement features that are insensitive to fatigue. The fatigue-sensitive pathway extracts temporal features that retain amplitude information and estimates the current fatigue level based on the energy benchmark established during the training phase, and outputs a fatigue estimate. The enhanced EEG feature vector and the stable motion feature are enhanced by bidirectional cross-attention enhancement through an enhanced co-attention module, resulting in enhanced EEG features and enhanced electromyographic features. Based on the fatigue estimate, the confidence level of the enhanced EEG features, and the confidence level of the enhanced EMG features, the EEG modality fusion weights and EMG modality fusion weights are calculated through a dynamic weighted gating mechanism. The degenerative modality features are predicted and compensated using a cross-modal compensation network. The compensated EMG features and EEG features are then weighted and fused according to the fusion weights to obtain the fused features. The hand movement intention recognition result is then output through a classifier. The hand movement intention recognition result is sent to the lower-level controller. The lower-level controller determines the corresponding target air pressure value based on the recognition result, and uses a PID closed-loop control strategy to adjust the pressure of the corresponding finger air chamber in the pneumatic soft glove, driving the glove to perform hand rehabilitation movements corresponding to the movement intention.

[0009] Optionally, the simultaneous acquisition of electroencephalogram (EEG) signals and surface electromyogram (EMG) signals includes: EEG signals from 19 motor cortex-related channels were collected using an EEG cap conforming to the international 10-20 standard. Reference electrodes were placed on both mastoid processes. The original samples were then resampled to 500 Hz. Electromyography (EMG) signals from six muscle sites related to hand grasping motion were acquired using surface EMG electrodes, including extensor carpi ulnaris, extensor digitorum, flexor carpi radialis, flexor carpi ulnaris, biceps brachii, and triceps brachii. Reference electrodes were placed at bony landmarks on the lateral aspect of the upper arm. The original samples were then resampled to 500 Hz. A multi-parameter synchronization box is used to achieve time synchronization between EEG signals and surface electromyography signals, ensuring that the bimodal data has a unified event label.

[0010] Optionally, preprocessing of the EEG signals includes: A fourth-order Butterworth bandpass filter is used for bandpass filtering from 0.1 to 50 Hz to remove low-frequency baseline drift and high-frequency electromagnetic interference. A notch filter with a center frequency of 50Hz and a quality factor of Q=30 is used to eliminate power frequency interference; The baseline mean is corrected by taking 0.2 seconds before the start of each task as the baseline window to eliminate DC offset of the signal; Each EEG channel was z-score standardized to achieve a mean of 0 and a standard deviation of 1.

[0011] Optionally, a Butterworth bandpass filter is used for bandpass filtering in the 20–200 Hz range to filter out low-frequency motion artifacts and ECG interference. A 50Hz notch filter is used to eliminate power frequency interference; The preprocessed surface electromyography (EMG) signals were divided into two paths: one path was z-score normalized for fatigue-invariant pathways, and the other path was mean-centered but not variance-normalized for fatigue-sensitive pathways, in order to preserve the relative amplitude differences.

[0012] Optionally, the multi-scale adaptive temporal convolutional network includes three parallel branches: The short-time scale branch employs dilated convolutions of the dilation rate sequence [1,2,4] to capture rapid fluctuations and transient event-related potentials in EEG signals; The mesoscale branch uses dilated convolutions of the dilation rate sequence [2,4,8] to model the continuous neural oscillations during the grasping execution phase; The long-term branch uses dilated convolution of the dilation rate sequence [4,8,16] to extract long-term temporal dependencies; The outputs of the three branches are integrated through a learnable weighted fusion strategy. The fused feature maps are then enhanced by spatial attention and temporal attention mechanisms to obtain EEG feature vectors.

[0013] Optionally, the fatigue estimate is calculated as follows: Global average pooling is performed on the temporal features extracted from fatigue-sensitive pathways to obtain energy statistics; Calculate the ratio of this energy statistic to the operating mean energy baseline established during the training phase; Subtracting this ratio from 1 and truncating to the [0,1] interval, we obtain the fatigue estimate. The closer the fatigue estimate is to 1, the more severe the fatigue condition.

[0014] Optionally, the calculation of the fusion weights through the dynamic weighted gating mechanism includes: Calculate the confidence scores for the enhanced EEG features and the enhanced EMG features, respectively; The two enhancement features are concatenated after global average pooling and then input into a gated multilayer perceptron to generate the original fusion weights. The original weights are modulated using confidence levels, and then the product of the fatigue estimate and the fixed bias coefficient is superimposed as the explicit bias. After applying range constraints to the modulated weights, they are renormalized to obtain the final EEG modality fusion weights and EMG modality fusion weights.

[0015] Optionally, the cross-modal compensation network predicts and compensates for degenerate modal features by: Using enhanced EEG features as input, a two-level structure of coarse predictor and fine predictor is used to generate a predictive estimate of electromyographic features; Calculate the overall quality score, which is the average of the EEG confidence score and the EMG confidence score; The compensation intensity is dynamically calculated based on the overall mass fraction. The compensation intensity increases as the mass fraction decreases and has an upper limit. The weighted sum of the predicted estimate and the original electromyographic features is used as the compensated electromyographic feature. The weighting adopts a residual connection method, and a fixed residual weight is introduced to prevent overcompensation.

[0016] Optionally, the step of adjusting the pressure of the corresponding finger air chamber in the pneumatic soft glove using a PID closed-loop control strategy includes: The current pressure value of each finger's air cavity is collected in real time by a pressure sensor, and the error between the current pressure and the target air pressure value is calculated. The control output is calculated based on the proportional coefficient, integral coefficient, and derivative coefficient, and the working status of the air pump and solenoid valve is adjusted accordingly. Different PID parameters are used during the inflation and deflation processes to ensure zero overshoot in both processes and to quickly and stably track the target pressure value. Secondly, this application provides a hand rehabilitation system based on EEG-sEMG signal joint control, used to implement the method described in the first aspect, the system comprising: The signal acquisition module is used to simultaneously acquire electroencephalogram (EEG) signals and surface electromyography (EMG) signals during the patient's hand movements, and to preprocess the acquired EEG signals and EMG signals respectively. The signal processing module is used to input the preprocessed EEG signal into a multi-scale adaptive temporal convolutional network, extract multi-timescale EEG temporal features and perform spatial and temporal attention enhancement, and input the preprocessed surface electromyography signal into the fatigue-invariant pathway and the fatigue-sensitive pathway respectively to obtain stable movement features and fatigue estimates that are insensitive to fatigue. The fusion decision module is used to perform bidirectional cross-attention enhancement on EEG feature vectors and stable action features through the enhanced co-attention module. Based on fatigue estimates and bimodal confidence, the fusion weight is calculated through a dynamic weighted gating mechanism. The degenerative modality features are predicted and compensated using a cross-modal compensation network. The compensated electromyographic features and EEG features are weighted and fused, and the hand movement intention recognition result is output by a classifier. The motion control module includes a lower-level controller and a pneumatic soft glove. The lower-level controller determines the corresponding target air pressure value based on the hand movement intention recognition result, and uses a PID closed-loop control strategy to adjust the pressure of the corresponding finger air chamber in the pneumatic soft glove to drive the glove to perform corresponding hand rehabilitation actions.

[0017] Compared with the prior art, this application has the following beneficial effects: 1. By integrating EEG (central intention) and sEMG (peripheral execution), the EEG-driven system can be utilized in the early stages of severe muscle weakness, and EEG can still maintain stable recognition even when muscles are fatigued, achieving complementary advantages.

[0018] 2. The dual-path sEMG encoder completely decouples motion features from fatigue state; the fatigue-sensing dynamic weighted gating mechanism adaptively adjusts the fusion weights according to the patient's real-time fatigue state, maintaining stable recognition performance during long-term training.

[0019] 3. When sEMG degrades severely due to fatigue, the compensation network predicts and compensates for sEMG features from EEG features, effectively mitigating performance degradation under extreme fatigue.

[0020] 4. The lower-level controller, combined with PID closed-loop control, eliminates pressure overshoot, achieves independent and precise control of all five fingers, and supports multiple rehabilitation training modes. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. Furthermore, these drawings and textual descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to specific embodiments.

[0022] Figure 1 A schematic flowchart illustrating a hand rehabilitation method based on combined control of EEG-sEMG signals, provided for an embodiment of this application; Figure 2 An overall architecture diagram of a hand rehabilitation system based on EEG-sEMG signal joint control provided in this application embodiment; Figure 3 This is a schematic diagram of a hand rehabilitation system based on EEG-sEMG signal joint control, provided as an embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the embodiments of this application. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0024] Example 1 This embodiment provides a hand rehabilitation method based on combined control of EEG-sEMG signals. For example... Figure 1 and Figure 2 As shown, the method specifically includes the following steps: Step S1: Simultaneously collect EEG and surface electromyography (EMG) signals during the patient's hand movements, and preprocess the collected EEG and EMG signals respectively.

[0025] This step involves simultaneously acquiring bioelectrical signals from both the patient's central nervous system and peripheral muscles. After preprocessing, high-quality, synchronously aligned dual-modal raw data is provided for subsequent feature extraction, ensuring the reliability and consistency of signal acquisition.

[0026] Among them, surface electromyography (sEMG) signals can reflect information such as the patient's muscle activation level and exertion intention. Electroencephalography (EEG) signals can reflect information such as the patient's attention level and central fatigue. By fusing the two types of signals, the system can simultaneously obtain "what the subject wants to do (sEMG+EEG)" and "the subject's current central cognitive state (EEG)," thereby facilitating more comprehensive and accurate control decisions in subsequent steps.

[0027] Regarding the acquisition of the above two types of physiological signals, in some embodiments, the simultaneous acquisition of electroencephalogram (EEG) signals and surface electromyography (EMG) signals includes: (1) EEG signal acquisition: EEG signals from 19 motor cortex-related channels were acquired using an EEG cap conforming to the international 10-20 standard, with reference electrodes placed bilaterally on the mastoid process. Nineteen electrode channels were selected for the brain regions related to the hand motor cortex, including FC1, FC2, FC3, FC4, FC5, FC6, Cz, C1, C2, C3, C4, C5, C6, CP1, CP2, CP3, CP4, CP5, and CP6. The original sampling rate was 2500Hz, and after preprocessing, it was resampled to 500Hz to balance data accuracy and computational efficiency.

[0028] (2) Acquisition of surface electromyography (EMG) signals: EMG signals from six muscle sites related to hand grasping movements were acquired using surface EMG electrodes, including the extensor carpi ulnaris, extensor digitorum, flexor carpi radialis, flexor carpi ulnaris, biceps brachii, and triceps brachii. The reference electrode was placed at the bony landmarks on the lateral aspect of the upper arm. The original sampling rate was 2000 Hz, and after preprocessing, it was resampled to 500 Hz.

[0029] The system guides the patient to perform three typical hand grasping movements in sequence: ball grasp, cylindrical grasp, and hook grasp. Each movement lasts approximately 3 seconds, including a preparation phase (0.5 seconds), an execution phase (2 seconds), and a relaxation phase (0.5 seconds). A multi-parameter EEG synchronization box is used to synchronize the EEG signals with surface electromyography (EMG) signals, ensuring that the bimodal data have consistent event labeling.

[0030] Understandably, as an alternative, the number of EEG channels can be reduced or increased as needed; sEMG muscle sites can be adjusted according to specific training movements, such as adding the flexor pollicis longus; and the sampling rate can be adjusted between 250-1000Hz.

[0031] After acquiring EEG and surface electromyography signals, preprocessing was performed to improve data reliability.

[0032] In some embodiments, preprocessing of EEG signals includes: Step S11: Use a 4th-order Butterworth bandpass filter to perform bandpass filtering from 0.1 to 50 Hz to remove low-frequency baseline drift and high-frequency electromagnetic interference.

[0033] transfer function It can be represented as:

[0034] in, For the complex frequency variable in the Laplace transform, The cutoff angular frequency, The filter order ( =4).

[0035] Step S12: Use a notch filter with a center frequency of 50Hz and a quality factor of Q=30 to eliminate power frequency interference.

[0036] transfer function It can be represented as:

[0037] in, The suppressed center angular frequency (corresponding to 50 Hz). This is the quality factor.

[0038] Step S13: Extract 0.2 seconds before the start of each task as a baseline window for baseline mean correction to eliminate DC offset of the signal.

[0039] Step S14: Perform z-score standardization on each EEG channel to make the mean 0 and the standard deviation 1, thereby reducing the impact of individual differences.

[0040] In some embodiments, preprocessing of the surface electromyography signal includes: Step S15: Use a Butterworth bandpass filter to perform bandpass filtering from 20 to 200 Hz to filter out low-frequency motion artifacts and ECG interference, while retaining the main frequency bands that reflect muscle activation.

[0041] Step S16: Use a 50Hz notch filter (Q=30) to eliminate power frequency interference.

[0042] Step S17: The preprocessed surface electromyography signal is divided into two paths: one path is z-score normalized for the fatigue-invariant pathway, and the other path is mean-centered but not variance-normalized for the fatigue-sensitive pathway, in order to preserve the relative amplitude differences.

[0043] Through the scheme in step S1, EEG reflects the central generation process of motion intent, while sEMG reflects the actual execution state of the muscles; the two complement each other. By unifying the sampling rate, synchronizing event labeling, and employing a branching normalization strategy during the acquisition phase, a reliable input foundation is established for subsequent fatigue decoupling modeling and multimodal fusion.

[0044] In addition, in other embodiments, a data augmentation strategy combining mixed sample enhancement (Mixup, mixing coefficient α=0.1) and temporal random masking (SpecAug, masking ratio 1%) is used to augment the preprocessed EEG and sEMG signals, thereby increasing the number of training samples and improving the generalization ability and robustness of the subsequent model.

[0045] Step S2: Input the preprocessed EEG signal into a multi-scale adaptive temporal convolutional network to extract EEG temporal features at multiple time scales, and enhance the EEG temporal features through spatial attention and temporal attention mechanisms to obtain EEG feature vectors.

[0046] The purpose of this step is to design multiple parallel temporal convolutional branches to capture motor intention features at different time scales related to motor intention in EEG signals, and to combine spatial attention (highlighting key brain regions) and temporal attention (focusing on key time periods) to achieve efficient extraction and enhancement of multi-level motor intention features in EEG signals, providing a highly discriminative EEG representation for subsequent multimodal fusion.

[0047] Specifically, the multi-scale adaptive temporal convolutional network includes three parallel branches, each corresponding to a different time scale: (1) Short-term scale branch: Dilated convolution with dilation rate sequence [1,2,4] is used to capture rapid fluctuations and instantaneous event-related potentials (ERPs) in EEG signals; suitable for modeling early cognitive processes in the motor preparation stage.

[0048] (2) Mid-time scale branch: Dilated convolution of dilation rate sequence [2,4,8] is used to model the continuous neural oscillation during the grasping execution period; corresponding to the rhythmic activity of the motor cortex during the hand grasping action execution period.

[0049] (3) Long-term scale branch: Dilated convolution of the dilation rate sequence [4,8,16] is used to extract long-term temporal dependencies; used to capture the overall policy and state changes in motion tasks.

[0050] Each branch employs Adaptive Temporal Convolutional Blocks (ATCBs), which adaptively adjust the weights of each channel using a learnable channel-level scaling factor to enhance the contribution of key channels. The calculation process is as follows: ATCB(x) = sigma(BN(Conv_dil(x))) ⊙ Scale(x) Where Conv_dil represents dilated convolution, BN is batch normalization, sigma is the GELU activation function, Scale(x) is a learnable channel-level scaling factor generated by global average pooling and 1×1 convolution, and ⊙ represents element-wise multiplication.

[0051] The outputs of the three branches are integrated through a learnable weighted fusion strategy. The fused feature maps are then enhanced by spatial attention and temporal attention mechanisms to obtain EEG feature vectors.

[0052] Specifically, the outputs of the three branches are integrated through a learnable weighted fusion strategy. The normalized branch weights are generated by global average pooling and then by a multilayer perceptron (MLP), avoiding the limitations of manually setting weights.

[0053] This process can be represented as:

[0054]

[0055]

[0056] in, Indicates the first The feature map output from each branch (e.g., short-time, medium-time, and long-time branches). Its dimension is... ,in, This indicates the batch size, which is the number of samples processed at one time. This represents the number of feature channels, such as the number of convolutional kernels. This represents the number of time steps, which is the sequence length.

[0057] This represents global average pooling. It applies to the time dimension. The average value is calculated for each channel, and the feature map is then transferred from... Compress to That is, for each sample and each channel, the mean of the entire time series is taken to obtain an aggregated global feature description.

[0058] For the first The result of the branches after global average pooling has a dimension of It summarizes the overall response strength of the feature map of this branch in the time dimension.

[0059] This means concatenating the global feature vectors of the three branches along the channel dimension to obtain... The matrix.

[0060] A multilayer perceptron typically consists of one or more fully connected layers and a non-linear activation function (such as ReLU). Its input is the concatenated global features, and its output is three unnormalized weight values ​​(logits), corresponding to the importance scores of the three branches.

[0061] To convert the three weight values ​​logits output by the MLP into probability values ​​that sum to 1. This refers to the normalized branch weights. The higher the weight, the greater the contribution of that branch to the final fusion result in the current input sample.

[0062] For the first The fusion weights (scalars) of each branch belong to the interval [0,1]. Since the global features of each sample are different, It changes dynamically with the input, enabling learnable weighted fusion.

[0063] This is a weighted summation operation. The feature map of each branch... With the corresponding weights Multiply the results of the three branches, then add the results of the three branches element by element to obtain the fused feature map. The dimension remains the same. .

[0064] This mechanism extracts global semantic information for each branch through global average pooling, then dynamically learns the importance weights of each branch using MLP+Softmax, and finally fuses multi-scale temporal features in a weighted sum. Compared to fixed weights or simple concatenation, learnable weighted fusion can adaptively adjust branch contributions based on the characteristics of the input signal, improving the model's ability to capture features at different time scales. This is particularly important in EEG signal processing because the energy distribution of EEG patterns from different subjects or different actions may differ at short, medium, and long time scales.

[0065] The fused feature maps are enhanced using both spatial attention and temporal attention mechanisms. Spatial attention highlights pathways in motor cortex-related brain regions while suppressing noise from irrelevant pathways. Temporal attention focuses on time windows strongly correlated with the execution of grasping actions (e.g., a 2-second execution period).

[0066] Specifically, regarding the spatial attention mechanism: after global average pooling along the time dimension, two layers of 1×1 convolutions and activation functions are used to generate channel-level weights W_spatial, which can be expressed as: W_spatial=sigma(Conv_{1x1}(GELU(Conv_{1x1}(GAP_T(F))))) Where F is the feature map after multi-scale branch fusion, W_spatial is the channel-level weight vector, Conv_ is the convolution operation, and 1x1 is the convolution kernel size.

[0067] For the temporal attention mechanism: after mean pooling (CAP) along the channel dimension, a 1×1 convolution and sigmoid activation are performed to generate the temporal weight W_temporal, which can be expressed as: W_temporal=sigma(Conv_{1x1}(CAP(F))) The final EEG encoded output is: F_EEG=F⊙W_spatial⊙W_temporal Through the approach in step S2, the multi-scale architecture can simultaneously capture fast transients and slow evolutions, while the dual attention mechanism further refines features from both spatial and temporal dimensions, significantly improving the accuracy of EEG decoding.

[0068] Understandably, as alternatives, a pure convolutional network or a Transformer encoder can be used instead of MS-ATCN; in addition, the dilation rate sequence can be adjusted according to the sampling rate; and the attention mechanism can be replaced by multi-head self-attention.

[0069] Step S3: Input the preprocessed surface electromyography signal into the fatigue-invariant pathway and the fatigue-sensitive pathway respectively; the fatigue-invariant pathway extracts temporal features and performs L2 normalization to obtain stable movement features that are not sensitive to fatigue; the fatigue-sensitive pathway extracts temporal features that retain amplitude information and estimates the current fatigue level based on the energy benchmark established during the training phase, and outputs the fatigue estimate.

[0070] It is understood that this step and step S2 can be executed synchronously or asynchronously.

[0071] During rehabilitation training, human muscles gradually fatigue. sEMG signals experience amplitude attenuation, spectral shift, and signal-to-noise ratio reduction during muscle fatigue, causing traditional encoders to extract features that are simultaneously mixed with motion information and fatigue noise. To address these issues, this application designs a dual-path parallel architecture: a fatigue-invariant path projects features onto a unit hypersphere using L2 normalization to eliminate amplitude variation interference and extract motion type discrimination features that are insensitive to fatigue; the fatigue-sensitive path retains the original amplitude information and quantifies the current fatigue level based on an energy benchmark established from the training set. These two paths achieve complete decoupling of motion features and fatigue state, providing clean motion features and a clear fatigue state reference for subsequent fusion decisions.

[0072] The core objective of fatigue-invariant pathways is to extract stable features that are strongly correlated with specific action categories and insensitive to fatigue levels, providing a reliable basis for motion intention recognition. The main structure of fatigue-invariant pathways includes: (1) Temporal Convolutional Network: Deep temporal features are extracted using a multi-layer TCNBlock (temporal convolutional residual block) with an inflation rate sequence [1,2,4,8]. This can be represented as: TCN(x)=x+BN(Conv_dil2(Dropout(ReLU(BN(Conv_dil1(x)))))) Where x is the input feature, and Conv_dil1 is the first dilated convolutional layer, which convolves the time dimension with a given dilation rate (e.g., 1, 2, 4, 8) to increase the receptive field without increasing the parameters.

[0073] Dropout is a random deactivation layer used to randomly set the output of some neurons to 0 with a certain probability, thus preventing overfitting.

[0074] Conv_dil2 is the second dilated convolutional layer used to further extract deeper features. Typically, the number of output channels is restored to be consistent with the input x, which facilitates residual connections.

[0075] This residual block extracts multi-scale temporal context information through two dilated convolutions, while using the residual structure to ensure lossless transfer of features from deep networks, making it suitable for processing long sequence signals such as electromyography and electroencephalography.

[0076] (2) L2 normalization and adaptive scaling module: First, L2 normalization is used to project the features onto a unit hypersphere to eliminate the influence of amplitude changes caused by fatigue. F_norm_inv=F_inv / ||F_inv||2·sqrt(C·T) Where F_norm_inv is the normalized output. C and T are the number of channels and time steps, respectively. F_inv is the original input feature, which is the output TCN(x) of the sequential convolutional network. ||||2 is the L2 norm, used to calculate the square root of the sum of squares of all elements of each sample. F_inv / ||F_inv||2 is responsible for projecting the features of each sample onto the unit hypersphere.

[0077] The optimized features were then further optimized using the learnable channel scaling parameter S. F_scaled_inv=F_norm_inv⊙S It is evident that L2 normalization is the core mechanism for fatigue decoupling in the invariant pathway: fatigue leads to an overall decrease in sEMG amplitude, but the relative activation patterns (feature directions) between different action categories remain relatively stable before and after fatigue. L2 normalization projects the features onto a unit hypersphere, retaining only the feature direction information and removing the influence of amplitude magnitude, making the output features of this pathway insensitive to fatigue while maintaining discriminative power over action categories.

[0078] The fatigue-sensitive pathway focuses on capturing the dynamic changes in muscle fatigue state and outputs an implicit estimate of fatigue level, providing a core state basis for subsequent fusion modules. The calculation method for the fatigue estimate can be summarized as follows: Global average pooling is performed on the temporal features extracted by the fatigue-sensitive pathway to obtain an energy statistic; the ratio of this energy statistic to the average energy baseline established during the training phase is calculated; this ratio is subtracted from 1 and truncated to the [0,1] interval to obtain the fatigue estimate. The closer the fatigue estimate is to 1, the more severe the fatigue state. Specific explanations are as follows: (1) Convolutional feature extraction with preserved amplitude: This pathway does not perform amplitude normalization and fully preserves the sEMG amplitude information, which can be expressed as: F_sens=TCN(Conv_{1x1}(X_EMG^{sens})) Where X_EMG^{sens} represents the input surface electromyography signal of the fatigue-sensitive pathway. TCN is a temporal convolutional network. F_sens represents the output features.

[0079] This pathway fully preserves the amplitude information of the sEMG signal, enabling subsequent fatigue estimation to accurately capture the energy decrease caused by muscle fatigue.

[0080] (2) Fatigue state estimation based on training set energy benchmark: Global average pooling is performed on the TCN features of fatigue-sensitive pathways to obtain the statistic E_sens reflecting the current overall energy level of sEMG: E_sens = ||GAP(F_sens)||2 During training, cumulative statistics are performed on E_sens for each batch (e.g., calculating a moving average or taking the average of all training samples) to obtain the running mean energy E_ref of clean (fatigue-free) sEMG; after training, E_ref is stored in the model as a fixed baseline. The fatigue level estimate is calculated as follows: f_hat=clip(1-E_sens / E_ref,0,1) Where f_hat is the fatigue estimate. clip( The function (,0,1) is a phase function that restricts the result to the interval [0,1].

[0081] The physical meaning of the above formula is as follows: During the training phase, the clean sEMG has E_sens≈E_ref, so f_hat≈0 (low fatigue); During the testing phase, when the sEMG amplitude decays due to fatigue, E_sens is significantly lower than E_ref, and f_hat increases monotonically with the degree of decay, approaching 1 to indicate severe fatigue.

[0082] In summary, the fatigue-invariant and fatigue-sensitive pathways output two differentiated features: the fatigue-invariant pathway outputs F_inv∈R^{B×C×T} for action classification; the fatigue-sensitive pathway outputs f_hat∈R^B (scalar) for fatigue perception fusion. This decoupled design fundamentally avoids the contamination of action features by fatigue noise, while providing a clear fatigue state reference for multimodal fusion.

[0083] Step S4: The EEG feature vector and stable action feature are enhanced by bidirectional cross-attention through the enhanced co-attention module to obtain the enhanced EEG features and enhanced EMG features.

[0084] The purpose of this step is to enhance the discriminative power of single-modal features by first achieving bidirectional interactive enhancement of EEG and sEMG features through an enhanced co-attention module before performing modality weighted fusion.

[0085] Specifically, multidimensional statistics are calculated for the EEG feature F_EEG (electroencephalogram feature vector) and the sEMG invariant feature F_inv (stable action feature): g_EEG=0.5·GAP(F_EEG)+0.3·GMP(F_EEG)+0.2·STD(F_EEG) g_EMG=0.5·GAP(F_inv)+0.3·GMP(F_inv)+0.2·STD(F_inv) Where GAP, GMP, and STD represent global average pooling, global max pooling, and standard deviation statistics, respectively. That is, the weighted sum of global average pooling, global max pooling, and standard deviation is calculated for EEG features and sEMG invariant features, respectively, to comprehensively characterize the overall level, peak value, and dispersion of the features.

[0086] The cross-attention weights are calculated as follows: W_EMG=alpha_min+(alpha_max-alpha_min)·sigma(MLP_{E2M}(LN(g_EEG))) W_EEG=alpha_min+(alpha_max-alpha_min)·sigma(MLP_{M2E}(LN(g_EMG))) Where LN is layer normalization, alpha_min=0.1, alpha_max=1.5, that is, the weights are mapped to the interval [0.1, 1.5] after Sigmoid activation, allowing for moderate enhancement but preventing a certain mode from being completely suppressed. The enhanced feature is: original feature ⊙ cross-attention weights + 0.1 × original feature (residual connection), that is: Enhanced EEG characteristics: F_tilde_EEG = F_EEG ⊙ W_EEG + 0.1·F_EEG Enhanced surface electromyography (EMG) characteristics: F_tilde_EMG = F_inv⊙W_EMG + 0.1·F_inv Step S5: Based on the fatigue estimate, the confidence of the enhanced EEG features, and the confidence of the enhanced EMG features, calculate the EEG modality fusion weights and EMG modality fusion weights through a dynamic weighted gating mechanism, and use a cross-modal compensation network to predict and compensate for the degenerative modality features. Then, the compensated EMG features and EEG features are weighted and fused according to the fusion weights to obtain the fused features. Finally, the hand movement intention recognition result is output through a classifier.

[0087] The purpose of this step is to dynamically adjust the fusion weights of EEG and sEMG through a two-layer perception mechanism: first, based on feature quality self-evaluation, the confidence level of each modality is obtained; then, combined with an explicit bias of fatigue estimates, the fusion weights of EEG and sEMG are dynamically adjusted (sEMG is dominant for mild fatigue, and gradually switches to EEG dominance for moderate to severe fatigue); simultaneously, when a modality degrades severely due to fatigue, a cross-modal compensation network predicts the features of the degraded modality (sEMG) from the high-quality modality (EEG) and performs residual compensation to effectively repair missing information. The two mechanisms work together to ensure that the system can output stable and reliable hand movement recognition results throughout the entire fatigue range.

[0088] In some embodiments, the EEG modality fusion weights and EMG modality fusion weights are calculated using a dynamic weighted gating mechanism, including: Step S51: Calculate the confidence levels of the enhanced EEG features and the enhanced EMG features, respectively.

[0089] The calculation formula can be expressed as: The confidence score of the EEG feature is calculated as c_EEG = sigma(MLP_EEG(GAP(F_tilde_EEG))). The confidence level of the electromyographic features is calculated as c_EMG = sigma(MLP_EMG(GAP(F_tilde_EMG))). Step S52: After globally averaging the two enhanced features, concatenate them and input them into a gated multilayer perceptron (MLP_gate) to generate the original fusion weights.

[0090] The formula can be expressed as: [omega_raw_EEG,omega_raw_EMG]=Softmax(MLP_gate([GAP(F_tilde_EEG),GAP(F_tilde_EMG)])) Among them, omega_raw_EEG and omega_raw_EMG are the original fusion weights.

[0091] Step S53: Modulate the original weights with confidence levels, and then superimpose the product of the fatigue estimate and the fixed bias coefficient as the explicit bias.

[0092] The confidence modulation process can be represented as: omega_EEG=(c_EEG·omega_raw_EEG) / (c_EEG·omega_raw_EEG+c_EMG·omega_raw_EMG+epsilon) Where omega_EEG is the modulated EEG weight.

[0093] After confidence modulation, an explicit bias correction based on fatigue perception is introduced: omega_tilde_EEG=omega_EEG+gamma_f·f_hat Where omega_tilde_EEG is the unnormalized EEG weight after adding fatigue bias, and gamma_f=0.15 is the fixed bias coefficient.

[0094] Step S54: Apply range constraints to the modulated weights and then renormalize them to obtain the final EEG modality fusion weights and EMG modality fusion weights.

[0095] After applying range constraints to the intermediate weights (using [0.25, 0.85] as an example below) and renormalizing, ensure that both modes retain a contribution no less than a set value (e.g., 25%) to prevent extreme biases. The weights for EEG modality fusion are omega_EEG=clip(omega_tilde_EEG,0.25,0.85). The weights for electromyography modal fusion are omega_EMG = 1 - omega_EEG It should be noted that when muscle fatigue leads to a decline in sEMG signal quality, the fatigue estimate f_hat increases, and omega_tilde_EEG increases accordingly. The system actively increases the fusion weight of the EEG mode, achieving an adaptive and smooth switch from sEMG-dominated to EEG-dominated mode. This explicit bias compensates for the potential lag in fatigue response that may occur with purely data-driven confidence assessment.

[0096] Furthermore, when the feature quality of a certain modality drops sharply due to fatigue degradation, the cross-modal compensation network is responsible for predicting and compensating for the missing information of the low-quality modality from the high-quality modality, effectively mitigating the impact of single modality degradation on the overall recognition performance.

[0097] In some embodiments, the cross-modal compensation network predicts and compensates for degenerate modal features, specifically including: Step S55: Using the enhanced EEG features as input, generate a predictive estimate of the electromyographic features through a two-level structure of coarse predictor and fine predictor.

[0098] Specifically, the cross-modal compensation network takes the EEG-enhanced feature F_tilde_EEG as input and generates a predictive estimate of the sEMG feature through a two-level structure: a coarse predictor (two convolutional blocks with kernel sizes of 1 and 5) and a fine predictor (three fine convolutional layers). The formula is expressed as: F_hat_M=Fine(Coarse(F_tilde_EEG)) Here, Coarse() is the coarse predictor, a shallow convolutional block typically containing two convolutional layers (kernel sizes 1 and 5 respectively), used to quickly extract general structural features from the EEG and generate a preliminary sEMG prediction. Fine() is the fine predictor, a more refined convolutional network, typically containing three fine convolutional layers (e.g., kernel sizes 3, 3, 3), used to refine the details of the coarse prediction and output the final predicted features. F_hat_M is the sEMG feature output from the cross-modal compensation network, obtained from the EEG prediction.

[0099] Step S56: Calculate the overall quality score Q, which is the mean of the EEG confidence score and the EMG confidence score. The formula is as follows: Q=(c_EEG+c_EMG) / 2 Step S57: Dynamically calculate the compensation intensity w_comp based on the overall quality score. The compensation intensity w_comp increases as the quality score decreases and has an upper limit (the formula below uses 0.8 as an example). The upper limit is retained to prevent over-reliance on prediction.

[0100] w_comp=clip(1-Q,0,0.8) F_M^comp=F_tilde_EMG·(1-w_comp)+F_hat_M·w_comp Where w_comp represents the compensation intensity, and F_M^comp represents the compensated electromyographic characteristics.

[0101] Step S58: Use the weighted sum of the predicted estimate and the original EMG feature as the compensated EMG feature. The weighting adopts a residual connection method, and a fixed residual weight r_w (e.g., r_w=0.3) is introduced to prevent overcompensation.

[0102] F_M^final=F_M^comp·(1-r_w)+F_tilde_EMG·r_w Wherein, F_M^final represents the compensated electromyographic feature. When w_comp=0 (normal signal quality), F_M^final degenerates into F_tilde_EMG, and the compensation network is automatically deactivated; when w_comp increases, the predicted features proportionally fill the gap in degenerate information, and the residual connections maintain system stability.

[0103] Then, the compensated electromyographic features and electroencephalographic features are weighted and fused according to the fusion weights obtained in the previous steps to obtain fused features, which are then output by a classifier to identify hand movement intentions.

[0104] The formula for weighted fusion is expressed as: F_fused=omega_EEG·F_tilde_EEG+omega_EMG·F_M^final Here, F_fused represents the fusion feature.

[0105] The fused features are then processed through a fully connected layer and Softmax to output the predicted probabilities of three types of hand gripping actions (spherical grip, cylindrical grip, and hook grip). The category with the highest probability is taken as the final motion intent recognition result.

[0106] Additionally, it should be noted that the total training loss function L_total can be: L_total=L_cls+lambda_comp·L_comp Where L_cls is the cross-entropy classification loss, L_comp is the cross-modal compensation loss, and lambda_comp=0.35 is the fixed weight coefficient.

[0107] In summary, this application's solution achieves adaptive weighting based on patient condition through a dual adjustment of self-perceived confidence and explicit fatigue bias; the cross-modal compensation network further alleviates the problem of complete sEMG failure under extreme fatigue. The entire mechanism enables the system to maintain stable recognition performance during long-term rehabilitation training.

[0108] Step S6: The hand movement intention recognition result is sent to the lower-level controller. The lower-level controller determines the corresponding target air pressure value based on the recognition result, and uses a PID closed-loop control strategy to adjust the pressure of the corresponding finger air chamber in the pneumatic soft glove, driving the glove to perform hand rehabilitation movements corresponding to the movement intention.

[0109] The purpose of this step is to convert the motion intention category (spherical grip / cylindrical grip / hook grip, etc.) output by the host computer into the target air pressure command of each finger cavity of the pneumatic soft glove. The PID closed-loop control algorithm is used to adjust the air cavity pressure of each finger in real time, so that the glove movement accurately reproduces the patient's intention, realizes the seamless transformation from "intention" to "movement", and supports a variety of refined rehabilitation training modes.

[0110] First, it should be noted that a pneumatic soft glove is used as the actuator for hand rehabilitation. The pneumatic glove is constructed based on the principle of pneumatic artificial muscles (PAM), made of highly flexible material, with independent airbags arranged along each of the five fingers. Inflating / deflating these airbags drives the fingers to flex or extend, simulating natural grasping and opening movements. Compared to traditional rigid exoskeletons, this actuator offers better wearing comfort, stronger conformity to the natural bending shape of the hand, and is suitable for long-term rehabilitation training.

[0111] The overall architecture of the pneumatic soft glove control system is divided into two main modules: feedback control and pneumatic drive. The two work together to achieve precise control and real-time adjustment of hand rehabilitation movements.

[0112] (1) Lower-level control core: The system uses the STM32F103ZET6 development board as the lower-level control platform. This development board is based on the ARM Cortex-M3 core with a main frequency of up to 72MHz, which can efficiently execute multi-threaded control logic and complex feedback algorithms. The development board has multiple communication interfaces such as GPIO, USART, SPI, I²C, and CAN, which support stable communication with peripherals such as the host computer, pressure sensor, and relay control module; it integrates ADC (analog-to-digital converter) and DAC (digital-to-analog converter) to facilitate accurate acquisition of analog signals from the pressure sensor; it has a low-power design and complete interrupt and timing functions to meet the dual requirements of real-time performance and reliability for long-term continuous operation of rehabilitation training. The STM32 microcontroller directly controls the relay module by outputting PWM signals, thereby manipulating the air path of the solenoid valve to achieve precise independent control of the air chambers of each finger.

[0113] (2) Air source drive device: The system is equipped with two AP40 miniature oil-free silent air pumps (DC 24V, rated current 600mA, power 12W, no-load flow 12L / min, positive pressure >200kPa, negative pressure -70kPa, noise <60dB), one for inflation and the other for depressurization, respectively realizing the forward and reverse drive functions of the pneumatic soft glove. The two air pumps can work alternately or simultaneously as needed according to the training mode.

[0114] (3) Pneumatic circuit control components: The system uses a FA0520E two-position three-way solenoid valve with a response time of <5ms, power of <2W, and support for a maximum load air pressure of 350mmHg. Each finger is equipped with an independent solenoid valve to achieve individual control of the five-finger air circuit without interference. The solenoid valve is driven by an 8-channel relay module (rated voltage 5V, which can be directly powered by a microcontroller). The relays serve as electrical switches between the solenoid valves and the air pump. The conduction state of the relays is controlled by the PWM signal of the STM32, which indirectly achieves precise control of the opening and closing of the solenoid valves, while providing electrical isolation protection.

[0115] (4) Pressure sensor: The system uses an XGZP6847D pressure sensor module (range 0~200kPa, overload pressure 500kPa, power supply voltage 3.3V, accuracy ±0.5%Span), integrated into each finger air path branch, for real-time monitoring of pressure changes inside the air chamber. The sensor module supports digital correction of parameters such as zero-point offset, sensitivity, temperature drift, and nonlinear error, and outputs a calibrated and temperature-compensated standard voltage signal to provide accurate real-time feedback data for closed-loop PID control.

[0116] (5) Pneumatic circuit architecture: The gas output from the air pump first enters the six-way pneumatic circuit conversion block, and then is distributed to five two-position three-way solenoid valves. Each solenoid valve controls the pneumatic circuit of one finger, ensuring that the movement of the five fingers can be adjusted independently. The outlet of the solenoid valve is connected to a pressure sensor module for real-time monitoring of the pneumatic circuit pressure, and finally connected to the five finger pneumatic channels.

[0117] To achieve precise control and dynamic adjustment of the pressure in each air chamber of the pneumatic soft glove, this invention employs a PID (Proportional-Integral-Derivative) closed-loop feedback control strategy. This method continuously monitors real-time pressure errors and dynamically adjusts the operating states of the air pump and solenoid valve to ensure that the pressure in each finger's air chamber stably tracks the target setpoint.

[0118] The PID controller integrates three control actions: proportional (P), integral (I), and derivative (D). The proportional action quickly adjusts the output based on the current error value; the integral action eliminates steady-state error and improves the long-term stability of the system; and the derivative action predicts the error change trend, improving the system's response speed and vibration resistance. Its mathematical expression is as follows: u(t)=K_p·e(t)+K_i·Integral[0,t]{e(tau)d(tau)}+K_d·de(t) / dt in: u(t) is the control output (adjusting the inflation or deflation pressure); e(t) = P_set - P(t) is the error (i.e. the deviation between the set pressure and the current pressure); K_p is a proportional coefficient used to adjust the system's response speed; K_i is the integral coefficient, used to eliminate steady-state error; K_d is the differential coefficient, used to enhance the dynamic response of the system and suppress oscillations.

[0119] Based on this, the PID closed-loop control strategy for adjusting the pressure of the corresponding finger air chambers in the pneumatic soft glove specifically includes: The current pressure value P(t) of each finger's air chamber is collected in real time by a pressure sensor, and the error e(t) between the current pressure P(t) and the target air pressure value P_set is calculated. The control output u(t) is calculated based on the proportional coefficient K_p, integral coefficient K_i, and derivative coefficient K_d to adjust the working state of the air pump and solenoid valve. Different PID parameters are used for the inflation and deflation processes to ensure no overshoot during inflation and deflation, and to quickly and stably track the target pressure value.

[0120] In practical applications, the Ziegler-Nichols method combined with the Simulink simulation platform can be used for PID parameter tuning: First, the proportional gain K_p is initially set to enable the system to have a fast response capability; then, the integral gain K_i is introduced to eliminate steady-state error; finally, the derivative gain K_d is set to suppress oscillation and overshoot. The optimal control parameters obtained after automatic optimization by Simulink in this embodiment are shown in the table below:

[0121] Based on the above PID parameters, there is no significant overshoot during the inflation process, the system response time is significantly shortened, and the pressure is maintained stably after reaching the set pressure; there is also no overshoot during the deflating process, and the system can quickly and stably track the target pressure value, achieving precise and independent adjustment of the air chamber pressure of each finger.

[0122] Based on the above solution, compared with the prior art, this application has the following beneficial effects: 1. By integrating EEG (central intention) and sEMG (peripheral execution), the EEG-driven system can be utilized in the early stages of severe muscle weakness, and EEG can still maintain stable recognition even when muscles are fatigued, achieving complementary advantages.

[0123] 2. The dual-path sEMG encoder completely decouples motion features from fatigue state; the fatigue-sensing dynamic weighted gating mechanism adaptively adjusts the fusion weights according to the patient's real-time fatigue state, maintaining stable recognition performance during long-term training.

[0124] 3. When sEMG degrades severely due to fatigue, the compensation network predicts and compensates for sEMG features from EEG features, effectively mitigating performance degradation under extreme fatigue.

[0125] 4. The lower-level controller, combined with PID closed-loop control, eliminates pressure overshoot, achieves independent and precise control of all five fingers, and supports multiple rehabilitation training modes.

[0126] Example 2 This embodiment provides a hand rehabilitation system based on combined EEG-sEMG signal control, used to implement the method of Embodiment 1. For example... Figure 3 As shown, the system includes the following structure: The signal acquisition module 110 is used to simultaneously acquire the electroencephalogram (EEG) signals and surface electromyogram (EMG) signals during the patient's hand movements, and to preprocess the acquired EEG signals and EMG signals respectively. The signal processing module 120 is used to input the preprocessed EEG signal into a multi-scale adaptive temporal convolutional network, extract multi-timescale EEG temporal features and perform spatial and temporal attention enhancement, and input the preprocessed surface electromyography signal into the fatigue-invariant pathway and the fatigue-sensitive pathway respectively to obtain stable movement features and fatigue estimates that are insensitive to fatigue. The fusion decision module 130 is used to perform bidirectional cross-attention enhancement on EEG feature vectors and stable action features through an enhanced co-attention module. Based on fatigue estimates and bimodal confidence, the fusion weight is calculated through a dynamic weighted gating mechanism. The degenerative modality features are predicted and compensated using a cross-modal compensation network. The compensated electromyographic features and EEG features are weighted and fused, and the hand movement intention recognition result is output through a classifier. The motion control module 140 includes a lower-level controller and a pneumatic soft glove. The lower-level controller determines the corresponding target air pressure value based on the hand movement intention recognition result, and uses a PID closed-loop control strategy to adjust the pressure of the corresponding finger air chamber in the pneumatic soft glove to drive the glove to perform the corresponding hand rehabilitation movements.

[0127] The functional implementation details of each module correspond one-to-one with the aforementioned method steps, and will not be repeated here.

[0128] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0129] It should be noted that in the description of this application, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means at least two.

[0130] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0131] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0132] Those skilled in the art will understand that all or part of the steps of the methods implementing the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0133] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and used as an independent product salesperson or other means, it can be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a hard disk, or an optical disk, etc.

[0134] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0135] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A hand rehabilitation method based on EEG-sEMG signal combined control, characterized in that, Includes the following steps: Simultaneously collect electroencephalogram (EEG) and electromyography (EMG) signals during the patient's hand movements, and preprocess the collected EEG and EMG signals respectively; The preprocessed EEG signal is input into a multi-scale adaptive temporal convolutional network to extract EEG temporal features at multiple time scales. The EEG temporal features are then enhanced through spatial attention and temporal attention mechanisms to obtain EEG feature vectors. The preprocessed surface electromyography (EMG) signals are input into the fatigue-invariant pathway and the fatigue-sensitive pathway, respectively. The fatigue-invariant pathway extracts temporal features and performs L2 normalization to obtain stable movement features that are insensitive to fatigue. The fatigue-sensitive pathway extracts temporal features that retain amplitude information and estimates the current fatigue level based on the energy benchmark established during the training phase, and outputs a fatigue estimate. The enhanced EEG feature vector and the stable motion feature are enhanced by bidirectional cross-attention enhancement through an enhanced co-attention module, resulting in enhanced EEG features and enhanced electromyographic features. Based on the fatigue estimate, the confidence level of the enhanced EEG features, and the confidence level of the enhanced EMG features, the EEG modality fusion weights and EMG modality fusion weights are calculated through a dynamic weighted gating mechanism. The degenerative modality features are predicted and compensated using a cross-modal compensation network. The compensated EMG features and EEG features are then weighted and fused according to the fusion weights to obtain the fused features. The hand movement intention recognition result is then output through a classifier. The hand movement intention recognition result is sent to the lower-level controller. The lower-level controller determines the corresponding target air pressure value based on the recognition result, and uses a PID closed-loop control strategy to adjust the pressure of the corresponding finger air chamber in the pneumatic soft glove, driving the glove to perform hand rehabilitation movements corresponding to the movement intention.

2. The method according to claim 1, characterized in that, The synchronous acquisition of electroencephalogram (EEG) signals and surface electromyogram (EMG) signals includes: EEG signals from 19 motor cortex-related channels were collected using an EEG cap conforming to the international 10-20 standard. Reference electrodes were placed on both mastoid processes. The original samples were then resampled to 500 Hz. Electromyography (EMG) signals from six muscle sites related to hand grasping motion were acquired using surface EMG electrodes, including extensor carpi ulnaris, extensor digitorum, flexor carpi radialis, flexor carpi ulnaris, biceps brachii, and triceps brachii. Reference electrodes were placed at bony landmarks on the lateral aspect of the upper arm. The original samples were then resampled to 500 Hz. A multi-parameter synchronization box is used to achieve time synchronization between EEG signals and surface electromyography signals, ensuring that the bimodal data has a unified event label.

3. The method according to claim 1, characterized in that, Preprocessing of EEG signals includes: A fourth-order Butterworth bandpass filter is used for bandpass filtering from 0.1 to 50 Hz to remove low-frequency baseline drift and high-frequency electromagnetic interference. A notch filter with a center frequency of 50Hz and a quality factor of Q=30 is used to eliminate power frequency interference; The baseline mean is corrected by taking 0.2 seconds before the start of each task as the baseline window to eliminate DC offset of the signal; Each EEG channel was z-score standardized to achieve a mean of 0 and a standard deviation of 1.

4. The method according to claim 1, characterized in that, The preprocessing of the surface electromyography signal includes: A Butterworth bandpass filter was used for bandpass filtering in the 20–200 Hz range to filter out low-frequency motion artifacts and ECG interference. A 50Hz notch filter is used to eliminate power frequency interference; The preprocessed surface electromyography (EMG) signals were divided into two paths: one path was z-score normalized for fatigue-invariant pathways, and the other path was mean-centered but not variance-normalized for fatigue-sensitive pathways, in order to preserve the relative amplitude differences.

5. The method according to claim 1, characterized in that, The multi-scale adaptive temporal convolutional network includes three parallel branches: The short-time scale branch employs dilated convolutions of the dilation rate sequence [1,2,4] to capture rapid fluctuations and transient event-related potentials in EEG signals; The mesoscale branch uses dilated convolutions of the dilation rate sequence [2,4,8] to model the continuous neural oscillations during the grasping execution phase; The long-term branch uses dilated convolution of the dilation rate sequence [4,8,16] to extract long-term temporal dependencies; The outputs of the three branches are integrated through a learnable weighted fusion strategy. The fused feature maps are then enhanced by spatial attention and temporal attention mechanisms to obtain EEG feature vectors.

6. The method according to claim 1, characterized in that, The fatigue estimate is calculated as follows: Global average pooling is performed on the temporal features extracted from fatigue-sensitive pathways to obtain energy statistics; Calculate the ratio of this energy statistic to the operating mean energy baseline established during the training phase; Subtracting this ratio from 1 and truncating it to the [0,1] interval, we obtain the fatigue estimate. The closer the fatigue estimate is to 1, the more severe the fatigue state.

7. The method according to claim 1, characterized in that, The calculation of fusion weights through a dynamic weighted gating mechanism includes: Calculate the confidence scores for the enhanced EEG features and the enhanced EMG features, respectively; The two enhancement features are concatenated after global average pooling and then input into a gated multilayer perceptron to generate the original fusion weights. The original weights are modulated using confidence levels, and then the product of the fatigue estimate and the fixed bias coefficient is superimposed as the explicit bias. After applying range constraints to the modulated weights, they are renormalized to obtain the final EEG modality fusion weights and EMG modality fusion weights.

8. The method according to claim 1, characterized in that, The cross-modal compensation network predicts and compensates for degenerate modal features, including: Using enhanced EEG features as input, a two-level structure of coarse predictor and fine predictor is used to generate a predictive estimate of electromyographic features; Calculate the overall quality score, which is the average of the EEG confidence score and the EMG confidence score; The compensation intensity is dynamically calculated based on the overall mass fraction. The compensation intensity increases as the mass fraction decreases and has an upper limit. The weighted sum of the predicted estimate and the original electromyographic features is used as the compensated electromyographic feature. The weighting adopts a residual connection method, and fixed residual weights are introduced to prevent overcompensation.

9. The method according to claim 1, characterized in that, The method of adjusting the pressure of the corresponding finger air chamber in the pneumatic soft glove using a PID closed-loop control strategy includes: The current pressure value of each finger's air cavity is collected in real time by a pressure sensor, and the error between the current pressure and the target air pressure value is calculated. The control output is calculated based on the proportional coefficient, integral coefficient, and derivative coefficient, and the working status of the air pump and solenoid valve is adjusted accordingly. Different PID parameters are used for the inflation and deflation processes to ensure that there is no overshoot during inflation and deflation, and to quickly and stably track the target pressure value.

10. A hand rehabilitation system based on EEG-sEMG signal joint control, characterized in that, The system for implementing the method according to any one of claims 1 to 9, the system comprising: The signal acquisition module is used to simultaneously acquire electroencephalogram (EEG) signals and surface electromyography (EMG) signals during the patient's hand movements, and to preprocess the acquired EEG signals and EMG signals respectively. The signal processing module is used to input the preprocessed EEG signal into a multi-scale adaptive temporal convolutional network, extract multi-timescale EEG temporal features and perform spatial and temporal attention enhancement, and input the preprocessed surface electromyography signal into the fatigue-invariant pathway and the fatigue-sensitive pathway respectively to obtain stable movement features and fatigue estimates that are insensitive to fatigue. The fusion decision module is used to perform bidirectional cross-attention enhancement on EEG feature vectors and stable action features through the enhanced co-attention module. Based on fatigue estimates and bimodal confidence, the fusion weight is calculated through a dynamic weighted gating mechanism. The degenerative modality features are predicted and compensated using a cross-modal compensation network. The compensated electromyographic features and EEG features are weighted and fused, and the hand movement intention recognition result is output by a classifier. The motion control module includes a lower-level controller and a pneumatic soft glove. The lower-level controller determines the corresponding target air pressure value based on the hand movement intention recognition result, and uses a PID closed-loop control strategy to adjust the pressure of the corresponding finger air chamber in the pneumatic soft glove to drive the glove to perform corresponding hand rehabilitation actions.

Images