Control strategy generation method and device for brain-controlled rehabilitation equipment, equipment and storage medium

Abstract

The invention discloses a brain-controlled rehabilitation equipment-oriented control strategy generation method, device and equipment and a storage medium, and relates to the technical field of signal processing, and the method comprises the following steps: acquiring a multi-channel electroencephalogram signal, and preprocessing the multi-channel electroencephalogram signal to obtain a target electroencephalogram signal comprising a steady-state visual evoked potential signal and a motor imagery signal, the target electroencephalogram signal corresponds to a preset action category; decoding the steady-state visual evoked potential signal by adopting filter group task related component analysis to obtain a correlation score vector; decoding the motor imagery signal by adopting a Mangban dynamic routing space-time network model to obtain a classification score vector; performing posterior probability distribution conversion and weighted fusion on the correlation score vector and the classification score vector to obtain fusion probability distribution; and generating a target control strategy according to the action category corresponding to the highest probability value in the fusion probability distribution. The control strategy obtained by the invention can consider both intention recognition precision and rehabilitation nerve activation effect.

Description

Technical Field

[0001] This application relates to the field of signal processing technology, and in particular to a method, apparatus, device, and storage medium for generating control strategies for brain-controlled rehabilitation devices. Background Technology

[0002] Upper limb motor dysfunction (such as loss of hand function after stroke or spinal cord injury) is one of the most challenging problems in neurorehabilitation. Traditional treatments rely on manual techniques, robotic assistance, or electrical stimulation, but these methods suffer from low patient participation, rigid training processes, and a lack of personalized feedback, making it difficult to meet the needs for continuous, efficient, and family-based rehabilitation. Brain-computer interface (BCI) technology, through a closed loop of "brain-device-feedback," holds promise for reconstructing neuroplasticity. Steady-state visual evoked potentials (SSVEPs) offer advantages such as high recognition rate, fast response, and short training cycles, while motor imagery (MI) can directly activate the motor cortex, exhibiting a significant neural activation effect. Simultaneously utilizing the high control precision of SSVEP and the neural activation effect of MI will greatly enhance rehabilitation outcomes.

[0003] Currently, most brain-controlled rehabilitation devices adopt a single paradigm: (1) SSVEP single paradigm system: under visual stimulation, the occipital region EEG is collected, and after decoding by algorithms such as Filter Bank Task-Related Component Analysis (FBTRCA), high-precision control commands are output to drive the exoskeleton or rehabilitation gloves to complete the action; (2) MI single paradigm system: under motor imagery task, the sensorimotor cortex EEG is collected, and after decoding the intention by networks such as Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) or Transformer, the rehabilitation device is driven; (3) Simple cascade or hard decision fusion: a few studies have tried to decode SSVEP first and then decode MI, or use rule-level fusion such as voting and gating to achieve dual-mode control.

[0004] The current problems with brain-controlled rehabilitation devices are: (1) Inability to balance accuracy and neural activation effect: The SSVEP system has precise control but lacks active activation of the motor cortex; the MI system has neural activation effect, but recognition is unstable due to low signal-to-noise ratio, nonlinearity and individual differences. (2) Bottleneck in MI signal decoding: Traditional CNN / RNN / Transformer cannot simultaneously capture the long-term temporal dependence and spatial features of electroencephalogram (EEG) signals, and has high computational complexity and poor individual generalization. (3) The fusion strategy is too simple: The existing cascade or hard decision mechanism has low information utilization rate, poor robustness in the face of signal conflict, noise or quality fluctuation, and cannot dynamically balance the confidence of the two paths. Therefore, how to obtain a control strategy that balances the accuracy of intention recognition and the effect of rehabilitation neural activation has become an urgent problem to be solved.

[0005] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The purpose of this application is to provide a method, device, equipment and storage medium for generating control strategies for brain-controlled rehabilitation devices, aiming to solve the technical problem of how to obtain control strategies that take into account both the accuracy of intention recognition and the effect of rehabilitation neural activation.

[0007] To achieve the above objectives, this application proposes a method for generating control strategies for brain-controlled rehabilitation devices, the method comprising: Multi-channel EEG signals are acquired and preprocessed to obtain target EEG signals, which correspond to preset action categories. The target EEG signals include steady-state visual evoked potential signals and motor imagery signals. The steady-state visual evoked potential signal is decoded through the first decoding path to obtain a correlation score vector. The first decoding path uses task correlation component analysis of the filter group. The motion image signal is decoded through a second decoding path to obtain a classification score vector. The second decoding path adopts the Mamba dynamic routing spatiotemporal network model. The relevance score vector and the classification score vector are transformed by posterior probability distribution and weighted fusion to obtain a fused probability distribution. A target control strategy is generated based on the action category corresponding to the highest probability value in the fusion probability distribution.

[0008] Furthermore, to achieve the above objectives, this application also proposes a control strategy generation device for brain-controlled rehabilitation devices, the device comprising: A preprocessing module is used to acquire multi-channel EEG signals and preprocess the multi-channel EEG signals to obtain target EEG signals. The target EEG signals correspond to preset action categories and include steady-state visual evoked potential signals and motor imagery signals. The first decoding module is used to decode the steady-state visual evoked potential signal through a first decoding path to obtain a correlation score vector. The first decoding path adopts task correlation component analysis of the filter group. The second decoding module is used to decode the motion image signal through the second decoding path to obtain a classification score vector. The second decoding path adopts the Mamba dynamic routing spatiotemporal network model. The weighted fusion module is used to perform posterior probability distribution transformation and weighted fusion on the relevance score vector and the classification score vector to obtain a fused probability distribution. The strategy generation module is used to generate a target control strategy based on the action category corresponding to the highest probability value in the fusion probability distribution.

[0009] Furthermore, to achieve the above objectives, this application also proposes a control strategy generation device for brain-controlled rehabilitation devices, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the control strategy generation method for brain-controlled rehabilitation devices as described above.

[0010] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the control strategy generation method for brain-controlled rehabilitation devices as described above.

[0011] One or more technical solutions proposed in this application have at least the following technical effects: First, the brain-controlled rehabilitation system acquires and preprocesses multi-channel EEG signals to obtain target EEG signals containing steady-state visual evoked potential (VEP) signals and motor imagery signals, ensuring the purity and independence of both signals and providing high-quality input for subsequent accurate decoding. Next, the system decodes the VEP signals using task-related component analysis with a filter group through a first decoding path, obtaining a correlation score vector. Using sub-band filtering and spatial projection templates, high-precision frequency identification of the VEP signals is achieved, providing a stable and reliable control signal for the system. Subsequently, the system decodes the motor imagery signals using a Mamba dynamic routing spatiotemporal network model through a second decoding path, obtaining a classification score vector. This model effectively improves the robustness and accuracy of decoding the motor imagery signals. Then, the system performs posterior probability distribution transformation and weighted fusion on the correlation score vector and classification score vector to obtain a fused probability distribution. Finally, the system generates a target control strategy based on the action category corresponding to the highest probability value in the fused probability distribution, driving the rehabilitation device to complete the corresponding action. This achieves a balance between the control precision of EEG signal decoding and the neural activation effect, providing an efficient and intelligent solution for neurological injury rehabilitation. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0013] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart illustrating an embodiment of the control strategy generation method for brain-controlled rehabilitation devices provided in this application. Figure 2 This is a schematic diagram of the preprocessing flow provided in Embodiment 1 of the control strategy generation method for brain-controlled rehabilitation equipment in this application; Figure 3 This is a schematic diagram of the structure of the brain-controlled rehabilitation system provided in Embodiment 1 of the control strategy generation method for brain-controlled rehabilitation equipment in this application; Figure 4 This is a flowchart illustrating the brain-controlled rehabilitation system provided in Embodiment 1 of the control strategy generation method for brain-controlled rehabilitation devices in this application. Figure 5 This is a schematic diagram of the fusion visual guidance process provided in Embodiment 1 of the control strategy generation method for brain-controlled rehabilitation equipment in this application; Figure 6 This is a flowchart illustrating Embodiment 2 of the control strategy generation method for brain-controlled rehabilitation devices in this application; Figure 7 This is a schematic diagram of the module structure of the control strategy generation device for brain-controlled rehabilitation equipment according to an embodiment of this application.

[0015] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0016] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application. To better understand the technical solutions of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0017] It should be noted that the executing entity of this application embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or brain-controlled rehabilitation system capable of realizing the above functions. The following uses a brain-controlled rehabilitation system as an example to describe this embodiment and the following embodiments.

[0018] Based on this, embodiments of this application provide a method for generating control strategies for brain-controlled rehabilitation devices, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the control strategy generation method for brain-controlled rehabilitation devices according to this application.

[0019] In this embodiment, the control strategy generation method for brain-controlled rehabilitation devices includes steps S10 to S50: Step S10: Acquire multi-channel EEG signals and preprocess the multi-channel EEG signals to obtain target EEG signals. The target EEG signals correspond to preset action categories and include steady-state visual evoked potential signals and motor imagery signals.

[0020] It should be noted that multichannel EEG signals refer to brain activity data collected simultaneously from different locations on the scalp using multiple electrodes, reflecting the synchronous firing information of neurons in multiple functional areas of the brain. Target EEG signals refer to the portion of EEG data retained after preprocessing and noise reduction; they contain both SSVEP and MI components and can reliably correspond to the user's specific intentions.

[0021] Preset action categories refer to several hand rehabilitation movements (such as grasping, single-finger flexion and extension, tapping, etc.) predefined by the system, used to establish a one-to-one correspondence with target EEG signals, realizing the mapping from intention to action. SSVEP signal refers to the periodic EEG response generated in the occipital cortex and phase-locked with the stimulation frequency when the brain continuously gazes at a visual stimulus flashing at a fixed frequency. MI signal refers to the EEG activity appearing in the motor sensory cortex with event-related synchronization / desynchronization characteristics when the subject only mentally simulates a limb movement without actually performing it.

[0022] It is understandable that different target EEG signals correspond to different action categories. The flashing frequency of SSVEP signals is different, which corresponds to different action categories. The spatiotemporal characteristics of EEG generated by MI signals are different, which correspond to different action categories.

[0023] Within the brain-controlled rehabilitation system, each type of hand rehabilitation movement is bound to the brain's intention through two parallel EEG "fingerprints": (1) Frequency fingerprint—SSVEP signal: The visual guidance interface uses small squares flashing at different fixed frequencies (such as 8Hz, 10Hz, 12Hz, etc.) to represent different movement categories; when the user gazes at a certain frequency block, the peak SSVEP energy generated in the occipital lobe is exactly at that frequency, and the system locks that frequency as the identifier of the corresponding movement category. (2) Feature fingerprint—MI signal: Motor imagery tasks (imagine right hand grasping, imagine right hand single finger flexion and extension, etc.) will induce specific spatial-temporal-spectral patterns in the motor-sensory cortex. Rhythmic event-related desynchronization / resynchronization (ERD / ERS). These patterns are mapped to unique feature vectors, each corresponding to a predefined action category.

[0024] Therefore, as long as a target EEG signal presents a specific SSVEP frequency or a specific MI feature, the system will uniquely classify it into the action category associated with it; if both occur simultaneously, the system will combine the two probabilities and ultimately point to a specific hand rehabilitation action with the highest confidence.

[0025] As an example, the target EEG signal includes a steady-state visual evoked potential signal and a motor imagery signal; the step of preprocessing the multi-channel EEG signal to obtain the target EEG signal includes: performing band separation filtering on the multi-channel EEG signal using a bandpass filter bank to obtain a band-separated signal; dividing the band-separated signal into channel subsets according to brain region functional localization to obtain a steady-state visual pathway signal and a motor imagery pathway signal; performing artifact recognition and removal on the motor imagery pathway signal using independent component analysis or an electrooculography artifact recognition algorithm based on template matching to obtain a clean motor imagery signal; and performing amplitude normalization processing on the steady-state visual pathway signal and the clean motor imagery signal to obtain the steady-state visual evoked potential signal and the motor imagery signal.

[0026] A bandpass filter bank is a collection of bandpass filters with different center frequencies and non-overlapping passbands, used to divide the raw EEG signal into several sub-bands according to frequency bands. For SSVEP signals, based on a preset stimulation frequency... Construct a set of narrowband bandpass filters to extract frequency bands from the signal: in, Indicates the first The SSVEP signal corresponding to a preset target frequency in time The value of is the sub-band signal obtained after bandpass filtering; BandPass indicates the bandpass filtering operation, which is used to extract components within a specific frequency range from the original signal; This indicates the original multichannel EEG signals in time. The value; It is the first The center frequency of the bandpass filter range of a preset target frequency.

[0027] For MI signals, an 8–30Hz bandpass filter is uniformly applied to extract the relevant data. and Rhythmic components: in, This indicates the time of the MI signal after bandpass filtering. The value of .

[0028] Bandwidth separation refers to the set of sub-band signals distributed across different frequency bands obtained after processing with a bandpass filter bank. Brain region functional localization involves dividing multi-channel EEG electrodes into spatial correspondences of visual and motor cortex regions based on the anatomical and functional distribution of the visual and motor cortex. Steady-state visual pathway signals (SSVEP-pathway signals) are EEG signals extracted from occipital lobe electrodes, bandwidth separated, and used for further decoding of SSVEP. Motor imagery pathway signals (MI-pathway signals) are EEG signals extracted from motor cortex electrodes, bandwidth separated, and used for further decoding of motor imagery features. Independent Component Analysis (ICA) is a blind source separation method that identifies and removes artifacts such as electrooculogram (EOG) artifacts by calculating statistically independent spatial components. Electrooculogram template-matching artifact removal is a technique that uses pre-established EOG artifact templates to detect correlations with signal segments, thereby locating and removing EOG artifacts. Pure motor imagery signals refer to the brain electrical activity of neural sources related to motor imagery that is retained after removing artifacts such as electrooculography (EOG) and electromyography (EMG). Motor imagery pathways (C3, C4, Cz) are susceptible to interference from EOG and EMG.

[0029] Please refer to Figure 2 , Figure 2 This diagram illustrates the preprocessing flow of the control strategy generation method for brain-controlled rehabilitation devices according to Embodiment 1 of this application. First, the raw EEG signal undergoes unified preprocessing, including filtering, channel selection and spatial partitioning, artifact removal, and signal standardization, to build a high-quality input foundation for decoding. Subsequently, the SSVEP signal branch and the MI signal branch undergo further processing: the SSVEP signal branch, after frequency filtering and channel selection, is partitioned into channels and space, ultimately outputting the preprocessed SSVEP signal; while the MI signal branch, after frequency filtering and channel selection, requires artifact removal, such as ICA or template-matching-based EEG artifact recognition algorithms, to eliminate non-neural source artifacts like EEG signals, and finally, signal standardization is performed, outputting the preprocessed MI signal. This process ensures the decoupling of the two signals in spatial distribution, which helps improve the discriminative ability of subsequent model feature extraction, while also guaranteeing the path independence and functional complementarity between the SSVEP and MI signals, laying a solid foundation for building a highly robust dual-mode brain-controlled signal base.

[0030] First, the brain-controlled rehabilitation system invokes a pre-set set of bandpass filters to filter the original multi-channel EEG signals channel by channel using non-overlapping narrow bands such as 8–30Hz and 30–45Hz. This breaks down the aliased broadband signals into multiple frequency bands, laying a clear foundation for the subsequent extraction of the required frequency bands for SSVEP and MI. Second, based on pre-calibrated spatial coordinates of the visual and motor cortex, the system assigns channel data from the occipital lobe region (O1, O2, Oz) to the steady-state visual pathway and channel data from the sensorimotor region (C3, C4, Cz) to the motor imagery pathway, ensuring that the two pathways do not interfere with each other spatially and improving feature purity. Then, the system centers and whitens the signal from the motor imagery pathway, and uses FastICA (Independent Component Analysis) to extract 20 independent components using symmetric approximation and a tanh nonlinear function. For each component, the system calculates its Pearson correlation coefficient with a pre-stored electrooculogram template (average blink waveform) using a 100ms sliding window. If the correlation coefficient is greater than 0.35, the column vector of the mixing matrix corresponding to that component is set to zero, while the remaining components remain unchanged. Finally, the retained components are used to reconstruct the time-domain signal through the inverse mixing matrix, completing the removal of electrooculogram artifacts and avoiding misinterpretation of intent. Finally, the system calculates the mean and standard deviation of the signals obtained from the two pathways channel by channel, and performs z-score normalization to unify the signal amplitude within the range of zero mean and unit variance, eliminating individual differences and channel gain differences, and ensuring the consistency and robustness of the input to the subsequent SSVEP and MI decoding models.

[0031] Step S20: Decode the steady-state visual evoked potential signal through the first decoding path to obtain the correlation score vector. The first decoding path uses task-related component analysis of the filter group.

[0032] It should be noted that the first decoding path refers to the dedicated algorithm pipeline within the system for processing SSVEP signals. Its input is the SSVEP data from the occipital channel, and its output is used for subsequent fusion decision-making. The correlation score vector is a multi-dimensional vector, with each dimension providing the correlation coefficient between the signal under test and the corresponding frequency template, fully preserving its matching strength with each candidate category. Filter Bank Task-Related Component Analysis (FBTRCA) is a multi-subband spatial filtering method. It first extracts the spatial projection that maximizes intra-class consistency from each subband signal, and then fuses the subband correlation coefficients to achieve high-precision identification of SSVEP frequencies.

[0033] Understandably, FBTRCA can effectively address the characteristic differences of SSVEP in different frequency bands. It enhances the sensitivity to each frequency component through subband filtering, and then uses Task-Related Component Analysis (TRCA) to extract spatial filters that are highly correlated with the target frequency, maximizing the consistency of intra-class signals. This allows for accurate identification of the target frequency under complex background noise, providing the system with high-precision and high-stability control signals and ensuring the reliability of subsequent fusion decisions.

[0034] Step S30: Decode the motion image signal through the second decoding path to obtain a classification score vector. The second decoding path adopts the Mamba dynamic routing spatiotemporal network model.

[0035] It should be noted that the second decoding path refers to the algorithm channel in the system dedicated to processing MI signals. It takes EEG data from the motor cortex channel as input and outputs a confidence vector for fusion decision-making. The second decoding path aims to address the inherent technical challenges of MI signals, such as low signal-to-noise ratio, complex long-range temporal dependencies, and significant individual variability.

[0036] The classification score vector is a vector with a dimension equal to the number of action categories. Each element represents the original logit value (log odds value) of the MI signal being classified as the corresponding category, which can be converted into a posterior probability using Softmax.

[0037] The Mamba-based Spatio-Temporal Attention Network (Mamba-STA-Net) is an end-to-end network that uses a one-dimensional convolutional neural network (1D-CNN) to extract shallow spatio-temporal features, a Mamba state-space module to capture long-range temporal dependencies, dynamic routing pooling to focus on key time points, and finally outputs classification scores through linear layers. This network model organically combines the local perceptual capabilities of convolutional neural networks, the long-range temporal modeling capabilities of the Mamba state-space model, and the original dynamic routing pooling mechanism described in this application through a "hierarchical feature extraction" strategy, achieving high robustness and high-precision decoding of MI signals.

[0038] As an example, the second decoding path employs the Mamba Dynamic Routing Spatiotemporal Network Model, which includes a convolutional neural network, a state-space model, a routing network, and a linear classifier. The step of decoding the motion image signal through the second decoding path to obtain a classification score vector includes: extracting shallow spatiotemporal features of the motion image signal using a convolutional neural network; performing long-range temporal dependency modeling on the shallow spatiotemporal features using a state-space model to obtain a deep feature sequence; calculating the importance score of each time point in the deep feature sequence using a routing network; generating dynamic attention weights based on the importance scores; applying the dynamic attention weights to perform weighted pooling on the deep feature sequence to generate a context feature vector; and processing the context feature vector using a linear classifier to obtain a classification score vector.

[0039] A convolutional neural network (CNN) is a shallow feature extractor consisting of two lightweight one-dimensional convolutional layers, batch normalization, and ELU (Exponential Linear Unit) activation. It is used to automatically learn local spatiotemporal filters on MI signals and output low-dimensional feature maps. A state-space model (SSM-Mamba) is a deep sequence encoder using the Mamba-S6 core. Through input-dependent parameterization and parallel scanning algorithms, it maps the feature sequence output by the CNN to a high-dimensional hidden state sequence, capturing long-distance temporal relationships. A routing network is an attention weight generator consisting of two MLP (Multilayer Perceptron) layers. It takes the features of each frame of the deep feature sequence as input and outputs a scalar of the discriminative contribution of the corresponding frame. A linear classifier is a decision module consisting of fully connected layers and Softmax activation, mapping the weighted pooled context vector to a class logit vector. Shallow spatiotemporal features refer to the tensors output by the CNN, whose dimensionality has been reduced and each element contains coupled information of local temporal patterns and spatial channels, but has not yet encoded global temporal sequences. The deep feature sequence refers to the high-dimensional hidden state sequence output by Mamba-S6, where each frame incorporates all contextual information from the start time to the current time. The importance score is a scalar value calculated by the routing network for each time step in the sequence; a larger value indicates that the time step is more critical for classification. The dynamic attention weights are softmax-normalized importance score vectors whose distribution adjusts in real-time according to changes in the input signal, used for weighted aggregation of the sequence. The context feature vector is a fixed-length vector obtained by weighting and summing the deep feature sequence using the dynamic attention weights, concentrating the most discriminative temporal information before feeding it into the classifier.

[0040] First, the brain-controlled rehabilitation system inputs the MI signal into two layers of 1D-CNN: the first layer uses... The convolutional kernel slides along the time dimension and filters each channel independently. After batch normalization and ELU activation, it is downsampled by a factor of 2. The second layer uses... Convolution further compresses the spatial-temporal dimensions, preserving local motion rhythms while reducing sequence length and alleviating subsequent computational burden. Then, the shallow features output by the CNN are fed into Mamba-S6: first linearly expanded to high dimension and split into two parts. One part is discretized by input-dependent parameters Δ, B, and C and then scanned in parallel to update the hidden state. The other part is gated with the state output using SiLU, and after stacking four Mamba Blocks, a deep feature sequence is obtained. This linear complexity global modeling captures the long-range dependencies between early bursts and late recovery in motion imagery, compensating for the insufficient receptive field of CNNs. Next, the routing network maps each frame of the deep feature sequence to a scalar importance score through two MLP layers, generates dynamic attention weights after Softmax, and compresses the entire sequence into a 128-dimensional context vector using weighted summation, amplifying brief but discriminative time points and suppressing noise segments. Finally, the linear classifier performs a fully connected mapping on these 128-dimensional vectors to the number of classes, outputting a logit vector that can be directly used by the fusion module. This design ensures end-to-end trainability and fixed feature dimensions, which facilitates subsequent confidence weighting.

[0041] As an example, the construction steps of the Mamba dynamic routing spatiotemporal network model include: constructing a shallow spatiotemporal feature extraction module based on a first convolutional block and a second convolutional block, wherein the kernel size of the first convolutional block is larger than that of the second convolutional block, and the number of output channels of the first convolutional block is smaller than that of the second convolutional block; constructing a deep temporal context encoding module based on a linear layer, a state-space model, and a gating function; constructing a dynamic routing pooling module based on a routing network; constructing a linear classifier based on the number of preset action categories; and constructing the Mamba dynamic routing spatiotemporal network model based on the shallow spatiotemporal feature extraction module, the deep temporal context encoding module, the dynamic routing pooling module, and the linear classifier.

[0042] Mamba-STA-Net consists of three core modules that work together to perform dynamic modeling and robust classification of MI signals: (1) Shallow spatiotemporal feature extraction module (a front-end network consisting of a first convolutional block and a second convolutional block connected in series, used to extract low-level spatiotemporal features from the raw EEG signal): To avoid the optimization challenges and overfitting risks caused by directly inputting raw high-dimensional EEG signals (characterized by low signal-to-noise ratio and strong non-stationarity) into subsequent complex long-sequence models (such as Mamba), this application designs a dedicated, trainable shallow spatiotemporal feature extraction module at the network front end. The core design philosophy of this module is to learn the basic spatiotemporal structure of the EEG signal in a layered and decoupled manner. Instead of using a fixed filter bank, this application designs a lightweight 1D-CNN as the feature extraction head, enabling it to adaptively learn the optimal feature filters from the data, thereby providing high-quality, information-intensive input for subsequent deep temporal modeling.

[0043] Let the input EEG tensor be ,in This refers to the batch size. This refers to the number of channels. This refers to the number of time points. This module first transforms the input into a format that uses dimensionality compression. The input is then processed through two cascaded convolutional blocks (ConvBlock).

[0044] ① First convolutional block (Temporal Feature Block): This module consists of an initial feature extraction unit comprised of temporal convolution, batch normalization, ELU activation, and 2x downsampling. Its kernel size is larger than that of the second convolutional block, but its output channel count is smaller. The kernel size refers to the length of the one-dimensional convolutional kernel in the temporal dimension. and The number of output channels refers to the number of feature maps output by the convolutional layer. and ).

[0045] The core responsibility of this module is to act as a time-characteristic filter, designed to capture the dynamic changes and local timing patterns of the signal from each individual EEG channel. Its detailed calculation process is as follows: 1) Temporal Convolution: One-dimensional convolution operation is used, and its convolution kernel... Along the time dimension For input Perform sliding calculations. For the first [number] [unit] in the batch... The sample, the first The output channel, the first The first input channel, the first The time point, the first Each time step within a convolutional kernel, its convolutional output Defined as: in, It refers to the dimension as ( , , Input feature tensor; This refers to the trainable weights (convolutional kernels) of the first convolutional layer, with a dimension of ; This refers to the kernel size of the first convolutional layer; This refers to the trainable bias vector of the first convolutional layer, with dimension . ; This refers to the number of input channels; This refers to the number of output channels of the first convolutional layer.

[0046] 2) Batch Normalization (BN): To address the internal covariate offset issue and stabilize training, batch normalization is performed on the convolution output. For the th... Each channel has a normalized output. for: in, This refers to the input tensor of the batch normalization layer (i.e., the output of the previous convolution step). A single element of the BN layer input refers to the batch-normalized input tensor. In the batch (i.e., the output of the previous convolution step), ,aisle Time point The corresponding original activation value; , These refer to the current batch of data in the channel. The mean and variance of the vector; , These refer to the learnable affine transformation parameters (scaling and translation); The numerical stability minimum is set to prevent division by zero.

[0047] 3) Nonlinear activation (ELU Activation): The exponential linear unit (ELU) activation function is used to enhance the model's nonlinear expressive power. in, This refers to the input value of the activation function; This refers to the hyperparameter of the ELU function, which controls the saturation region of the negative value part, and is usually set to 1.0.

[0048] 4) Temporal downsampling and regularization: The final output of this convolutional block It can be summarized as follows: in, For batch size, This refers to the number of output channels of the first convolutional layer. This refers to the length of the output sequence of the first convolutional block. This refers to the output tensor of the first convolution module.

[0049] ② Second convolutional block (Spatio-temporal Fusion Block): This module refers to the refined feature extraction unit that follows the first convolutional block and is also composed of temporal convolution, batch normalization, and ELU activation. Its convolutional kernel size is smaller than that of the first convolutional block, and its number of output channels is greater than that of the first convolutional block.

[0050] Output of the previous stage Based on this, the module aims to use a convolutional kernel with a smaller kernel size. Further refine the features to learn more detailed and abstract combinations of local feature patterns. Among these, This refers to the number of output channels of the second convolutional block (e.g., 32). This refers to the kernel size of the second convolutional block (e.g., 3). This refers to the trainable weights (convolutional kernels) of the second convolutional layer.

[0051] The calculation process is exactly the same as that of the first convolutional block, and can be summarized by the following formula: The final output of this module This forms the final shallow feature sequence. ,in, This refers to the length of the output sequence of the second convolutional block. This refers to the output tensor of the second convolutional module (convolutional module and convolutional block have the same meaning).

[0052] This sequence is not only lower in dimensionality and easier to process, but each of its feature vectors also contains discriminative local spatiotemporal information learned end-to-end from the original signal. This high-quality feature sequence will serve as the input to the core MambaEncoder module of this application, laying a solid foundation for its successful deep temporal context modeling.

[0053] (2) Deep Temporal Context Encoding Module (MambaEncoder, an encoder consisting of a linear layer, a Mamba-S6 state-space model, and a gated function concatenated together, used for long-range temporal modeling of shallow feature sequences): The core innovation of this application lies in introducing the state-of-the-art state-space model Mamba as the backbone of the network. Its linear computational complexity and powerful context dependency modeling capabilities, particularly in handling long sequences, enable deep processing of feature sequences output by the CNNHead. Traditional recurrent neural networks (RNNs) are limited by the vanishing / exploding gradient problem, making it difficult to capture long-range dependencies. While the Transformer model addresses this issue through its self-attention mechanism, its quadratic computational and memory complexity makes it impractical for processing extremely long sequences like EEG.

[0054] Mamba fundamentally revolutionizes the traditional State-Space Model (SSM) architecture through an innovative Selective State Space Mechanism (S6). It not only solves the problem of LTI (Linear Time-Invariant) SSMs being unable to adapt to dynamic inputs, but also overcomes the sequential computation bottleneck of RNNs through a hardware-aware parallel scanning algorithm. This enables the Mamba Encoder to efficiently learn complex, long-range temporal dependencies in EEG signals end-to-end.

[0055] MambaEncoder receives the feature sequence from the first stage. The sequence is first transposed to ,in, This refers to the length of the transposed sequence, which is equal to the number of time points output by the second convolutional block. ; This refers to the feature dimension after transposition, which is equal to the number of output channels of the second convolutional block. ; This refers to The sequence obtained after the transpose operation (permute(0,2,1)) is adapted to the (sequence length, feature dimension) format commonly used in the Mamba model.

[0056] The encoder is composed of multiple stacked MambaBlocks. Each block performs a complete selective state-space transformation on the input sequence. Its workflow can be broken down into the following steps: ① Input linear projection and gating mechanism: MambaBlock first processes the input through a linear layer. Projected onto a higher dimension, it splits into two parts. and This is used for subsequent gating calculations and state updates. The linear layer refers to the weight matrix W and biases that map the input features to the hidden state. , It means Projected to a higher dimension through the internal linear layer of MambaBlock The tensor after; This refers to the hidden feature dimension (e.g., 128) used internally by MambaEncoder. It determines the expressive power and computational complexity of the model. Inside MambaBlock, all core computations, including the State Space Model (SSM) and gating mechanisms, are performed in this high-dimensional feature space. It refers to the gated input tensor, which is the upper half feature sequence obtained after linear projection and splitting. This tensor is used to calculate the gate vector, which is responsible for controlling the information flow output by the S6 module and determining which context information should be retained. It refers to the input tensor of the S6 module, which is the lower half feature sequence obtained after linear projection and splitting, used to calculate and drive the dynamic state transition parameters.

[0057] Here, `split()` refers to the data splitting function. It refers to a linear transformation function.

[0058] ② The core of the selective state-space model (S6): This is the core computing unit of Mamba, which... The core of this processing lies in the parameterization of its input-dependent parameters, i.e., its key parameters. , , It is input At each time step Dynamically generated: in, At time step State transition matrix It describes the change of the system state over time; Indicates at time step Control input matrix It describes the effect of external inputs on the system state; Indicates at time step The output matrix It describes how the system state is mapped to the output; Indicates at time step input vector It is the input data after preprocessing or transformation.

[0059] The state matrix It is a trainable but fixed parameter. This design allows the model to dynamically adjust its state updates and output readings based on the current input.

[0060] ③ Discretization and State Update: To implement this on digital computing devices, the continuous state-space equations are discretized using the Zero-Order Hold (ZOH) method, yielding update rules for discrete states. The discretized state matrix... and input matrix According to the selectivity parameter The calculation shows that: in, This represents the matrix exponential function, which maps the continuous-time system dynamics to the discrete-time dynamics. This represents the state transition matrix of a continuous-time system. Represents the identity matrix, and Same dimension.

[0061] Note: In actual implementation, First-order Taylor expansion is typically used for efficient approximation.

[0062] The hidden state of the system The update follows the recursive formula: in, These represent the hidden states of the previous time step and the current time step, respectively. This refers to the time step The input feature vector; , This refers to the time step Dynamically generated, discretized state and input matrices; For the state dimension.

[0063] ④ Selective scanning and convolutional representation: The above recursive relation is mathematically equivalent to a time-varying convolution operation. Using a parallel scan algorithm optimized for modern hardware (GPUs), the output of the entire sequence can be efficiently computed in one go, avoiding the sequential computation bottleneck of traditional RNNs. Its convolutional form can be expressed as: in, Indicates time The system output vector; Indicates at time step The output matrix It describes how the system state is mapped to the output; Indicates at time step input vector ; Indicates from time 1 to During this period, all discretized state matrices The product of two products; In time step A dynamically generated, discretized input matrix. The output of the entire sequence is the output for all time steps. A set of.

[0064] This reveals the essence of Mamba: a deep convolutional model with an infinite receptive field (theoretically able to see all information from the beginning to the present) and a kernel function that changes dynamically over time.

[0065] ⑤ Output and Gating Activation: Finally, the output of MambaBlock It is the output of the S6 module Compared with the previously separated The result is obtained by element-wise multiplication using a SiLU (Swish) gate function (which employs SiLU-activated element-wise multiplication to control information flow). in, This refers to the output sequence of the S6 module; It refers to the gated vector sequence separated after linear projection of the input (that is, the gated input tensor mentioned earlier, which refers to the upper half feature sequence obtained after linear projection and splitting). It represents the element-wise product (Hadamard Product). This refers to the output sequence after passing through a MambaBlock. It is the final result of the MambaBlock processing the original input and will be used as the input of the next MambaBlock or as the final output of the MambaEncoder.

[0066] This gating mechanism further enhances the model's nonlinearity and selectivity. After processing through all MambaBlocks in the MambaEncoder, the final output feature sequence is... At each time point, complete, dynamically selected global contextual information based on content is integrated, providing a high-quality feature foundation for subsequent attention focusing.

[0067] (3) Dynamic Routing Pooling Module (a pooling unit consisting of a routing network, Softmax normalization, and weighted summation, used to compress deep feature sequences into fixed-length context vectors based on attention weights): After MambaEncoder completes deep temporal context encoding, an efficient pooling mechanism is needed to convert the variable-length feature sequences. The features are aggregated into a fixed-length vector for use by the subsequent classifier. While traditional Global Average Pooling (GAP) is simple and efficient, its core drawback is that it treats features across all time steps equally, assigning them the same weights. This mechanism "dilutes" transient features that are short-lived throughout the signal but crucial for classification decisions (e.g., specific neural bursts at the onset of motor imagery), thus limiting the model's final performance.

[0068] To address this issue, this application designs an adaptive and learnable DynamicRoutingPooling module. The core design philosophy of this module continues the original STA (Significant Static Allocation) concept of this application, allowing the model to autonomously and dynamically assign different importance weights to each part of the sequence based on the different input data. It is no longer a static averaging operation, but a dynamic, weighted summation process focused on key information.

[0069] At the core of this module is a trainable routing network (routing_net), a small multilayer perceptron (MLP) designed to serve as an attention score generator. Its detailed workflow is as follows: ① Calculation of attention score (“importance”): The output sequence of MambaEncoder It is fed into routing_net. This network is a non-linear transform that it applies to each time step. high-dimensional feature vectors Mapped to a scalar, unnormalized "importance score". .

[0070] in, This refers to the time step The input feature vector has a dimension of ; , This refers to the trainable weight matrix and bias vector of the first hidden layer in routing_net; It refers to the hyperbolic tangent activation function, used to introduce nonlinearity; , This refers to the trainable weight vectors and bias scalars of the output layer of routing_net; It refers to the final calculated time step. Raw scores for importance.

[0071] After this step, the entire sequence It is converted into a fractional vector .

[0072] ② Attention weight normalization: To transform the original scores into an effective probability distribution, this application uses the Softmax function to transform the score vector. In the dimension of time Normalize to generate a set of attention weights that sum to 1. .

[0073] in, This refers to the time step Normalized attention weights. The higher the score, the greater the corresponding weight.

[0074] ③Weighted Pooling: Ultimately, this application utilizes these attention weights For the entire Mamba output sequence Weighted summation is performed to "condense" the long sequence into a single context vector that focuses on information from all key time points. .

[0075] in, This refers to the final output, fixed-length context vector. Indicates time The system output vector (MambaEncoder final output sequence) In time eigenvectors).

[0076] The DynamicRoutingPooling module enables the model to dynamically and intelligently determine which time segments to focus on based on the characteristics of each EEG sample, while ignoring parts that are irrelevant to the task or full of noise. This preserves the discriminative information in the sequence to the greatest extent, greatly improving the model's final classification accuracy and robustness.

[0077] (4) Classifier Output: The DynamicRoutingPooling module outputs highly condensed and information-rich context vectors. Subsequently, this application requires a final decision layer to map the high-dimensional feature space to a task-specific class probability space. To this end, this application employs a standard linear classifier followed by a Softmax activation function to complete the final classification decision.

[0078] ① Linear Transformation: Context vector First, a linear transformation is performed through a fully connected layer (nn.Linear) to calculate the raw, unnormalized score for each category, which this application refers to as logits. ,in, This refers to the number of preset action categories, which is the total number of action categories that the rehabilitation equipment needs to perform (e.g., grasping, flexion-extension, point grip, etc.) as predefined by the system (also known as the number of target frequency categories, which is the number of different SSVEP target frequencies that the system needs to identify, with different frequencies corresponding to different action categories). This value determines the classifier output. The feature dimensions.

[0079] in, This refers to the context vector from the DynamicRoutingPooling module; This refers to the trainable weight matrix of the classifier; This refers to the logit score vector for each category; It refers to the trainable bias vector of the classifier.

[0080] ② Softmax probability output: To convert the logit score into a standard posterior probability distribution, this application applies the Softmax function. For the ... Each category has a final predicted probability. The calculation is as follows: in, This refers to the first step in MI path determination. The final probability of each category; , It refers to the logit vector.

[0081] The module ultimately outputs a probability distribution vector. The final decoding result of the MI path (second decoding path) will be sent to the fusion decision mechanism unit and intelligently fused with the output of the SSVEP path.

[0082] Step S40: Perform posterior probability distribution transformation and weighted fusion on the correlation score vector and the classification score vector to obtain the fused probability distribution.

[0083] It should be noted that the fusion probability distribution refers to the final probability vector obtained by linearly weighting the relevance score vector and the classification score vector according to preset weights after the system converts them into posterior probabilities respectively. Each dimension of the fusion probability distribution represents the comprehensive confidence level of the corresponding action category.

[0084] As an example, the step of performing posterior probability distribution transformation and weighted fusion on the correlation score vector and the classification score vector to obtain a fused probability distribution includes: performing posterior probability distribution transformation on the correlation score vector and the classification score vector through a normalization function to obtain a correlation probability distribution vector and a classification probability distribution vector; and performing weighted fusion on the correlation probability distribution vector and the classification probability distribution vector according to preset steady-state visual evoked potential weight coefficients and preset motion imagery weight coefficients to obtain a fused probability distribution.

[0085] The normalization function, specifically the Softmax function, maps the original score vector to a probability distribution with a sum of 1. The correlation probability distribution vector, after Softmax normalization, represents the posterior probability of the SSVEP branch for each action category. The classification probability distribution vector, after Softmax normalization, represents the posterior probability of the MI branch for each action category. The preset steady-state visual evoked potential weight coefficients are fixed fusion weights pre-assigned to the SSVEP branch to increase the proportion of decisions based on high signal-to-noise ratio signals. The preset motor imagery weight coefficients are fixed fusion weights pre-assigned to the MI branch to preserve rehabilitation information from the MI signal during fusion.

[0086] First, the system feeds the relevance score vector and the classification score vector into Softmax, normalizing each component to a probability value of 0–1, thus obtaining the relevance probability distribution vector and the classification probability distribution vector. Then, the relevance probability distribution vector is weighted by 0.7 and the classification probability distribution vector is weighted by 0.3, and the vectors are linearly added dimension by dimension to output the fused probability distribution.

[0087] To overcome the inherent limitations of traditional rule-based hierarchical arbitration logic, such as low information utilization, rigid decision boundaries, and poor adaptability to dynamic changes in signal quality, this application proposes a novel decision-making mechanism that fuses information in the probability domain. This mechanism no longer relies on a single highest-confidence category (“Winner-takes-all”), but instead fully utilizes the complete posterior probability distribution output by two parallel decoding paths. A mathematically more rigorous adaptive weighted fusion model is used to generate the final system control commands. This aims to elevate the fusion decision-making process from an “expert system” based on manually set prior rules to a more scientific and robust data-driven “intelligent arbitration system.”

[0088] (1) Probabilistic Vector Generation: The input to this unit is the raw output from two parallel decoding paths, which is then uniformly converted into a standardized posterior probability vector to ensure scale consistency in subsequent fusion operations.

[0089] SSVEP path (first decoding path) output: The raw output from the FBTRCA algorithm is a relevance score vector. To convert it into a probability distribution, this application uses the Softmax function for normalization: in, This refers to the first step in SSVEP path determination. The posterior probability of each action category; For the signal to be measured and the first The relevance score of the frequency template corresponding to each action category; This refers to the relationship between the signal to be measured and the first... The relevance score of the frequency template corresponding to each action category; This refers to the temperature hyperparameter, used to adjust the entropy (i.e., smoothness) of the probability distribution. Smaller... This will make the probability distribution more "sharp," tending towards a hard maximum; a larger This will make the distribution more "smooth".

[0090] MI path output: The output from Mamba-STA-Net has already passed through its end Softmax classifier and is itself a standard posterior probability distribution vector. .

[0091] (2) Probabilistic Weighted Fusion: In obtaining the probability vectors of the two branches and Then, the system calculates the final fusion probability vector using a linear weighted sum model. : in, , : These are the prior weight coefficients assigned to the SSVEP and MI paths, respectively. Their values ​​are preset by the system based on the inherent signal-to-noise ratio (SNR) characteristics of the two modes, and satisfy the following conditions: In this embodiment, considering the high stability and high signal-to-noise ratio of the SSVEP signal, It is assigned a high value (e.g., 0.7) to ensure the baseline reliability of system control, while It is then assigned an auxiliary weight (e.g., 0.3).

[0092] (3) Final Decision Making: Final system control commands From the fusion probability vector The category with the highest probability value is determined: This fusion mechanism effectively integrates all the information from the two decoding paths by performing a smooth weighted average in the probability domain, significantly improving the robustness of decision-making and the accuracy in complex signal scenarios. Compared with traditional hard voting or rule-based arbitration, it has obvious performance advantages.

[0093] (4) Output smoothing: Temporal window consistency check: To further improve the stability of control commands and prevent equipment jitter or erroneous actions caused by single misjudgments, this application introduces an output smoothing mechanism. This mechanism is effective for continuous... A time window (e.g.) =5) Output within The results are determined by a unanimous vote.

[0094] Only when in In a series of consecutive decision results, only when more than M (e.g., M=3) results belong to the same category is that category confirmed as the final, stable system output command and sent to the rehabilitation device for execution. Otherwise, the system will maintain the stable state of the previous moment and will not generate new control commands. This approach, by introducing consistency constraints in the time dimension, effectively filters out instantaneous noise and accidental misjudgments, ensuring high reliability of the final output command and smooth operation.

[0095] Step S50: Generate a target control strategy based on the action category corresponding to the highest probability value in the fusion probability distribution.

[0096] It should be noted that the target control strategy refers to directly converting the preset hand movement command corresponding to the movement category with the highest probability in the fusion probability distribution into a sequence of drive parameters that the rehabilitation device can execute, so that the rehabilitation device can complete the specified movement according to the command. The preset hand movement command refers to the specific hand movement command directly executed by the rehabilitation device, such as "right hand grasp", "right hand single finger flexion and extension", or "right hand tapping", etc., which are used to drive the pneumatic rehabilitation glove to complete the corresponding rehabilitation training movement.

[0097] Understandably, the system first performs an argmax operation on the fused probability distribution to obtain the index with the highest probability. Then query the category-action mapping table to... The data is translated into action labels such as "right-hand grasping." These labels are then encapsulated into structured control strategy objects containing category IDs, durations, and intensity levels, and immediately stored in the strategy cache, thus completing the generation of the target control strategy. This target control strategy is used to instantly convert the user's EEG intentions into action commands that the rehabilitation glove can recognize, enabling personalized, low-latency human-computer interaction.

[0098] The brain-controlled rehabilitation system of this application achieves precise driving of hand movements through wearable devices based on a target control strategy, taking into account both the initiative of rehabilitation training and the high efficiency of intention recognition, and is suitable for intelligent auxiliary treatment of nerve injury rehabilitation.

[0099] The brain-controlled rehabilitation system of this application includes an EEG acquisition module, a visual guidance module, an algorithm module for executing the aforementioned control strategy generation method, a pneumatic rehabilitation glove device, and a rehabilitation device control module. The EEG acquisition module is used to acquire the user's SSVEP and MI EEG signals in real time; the visual guidance module is used to display a synchronous fusion induction interface to guide the user in generating the aforementioned EEG signals; the algorithm module is used to acquire the aforementioned EEG signals, perform fusion recognition analysis, generate a target control strategy based on the recognition results, and drive the pneumatic rehabilitation glove device to complete the corresponding actions.

[0100] Please refer to Figure 3 , Figure 3 This is a schematic diagram of the brain-controlled rehabilitation system provided in Embodiment 1 of the control strategy generation method for brain-controlled rehabilitation equipment according to this application. The user generates EEG signals through a stimulation-guided interface. These signals are acquired in real time by the EEG acquisition module and transmitted to the algorithm module via the LSL stream protocol. The algorithm module uses a parallel dual-path decoding architecture to decode the SSVEP and MI signals, generating control commands. The control commands are sent to the pneumatic rehabilitation glove via serial communication to execute corresponding rehabilitation actions. The stimulation-guided interface plays a motor imagery training video while superimposing multiple visual stimulation blocks flashing at different frequencies on the edge area of ​​the interface to induce SSVEP signals. The user focuses on the flashing area at a specified frequency, enabling active input of commands for system mode, action category, or state switching. Through this fusion decoding method, the entire system achieves precise driving of hand movements, balancing the initiative of rehabilitation training with the efficiency of intention recognition, and is suitable for intelligent auxiliary treatment of nerve injury rehabilitation.

[0101] Please refer to Figure 4 , Figure 4 This is a flowchart illustrating the brain-controlled rehabilitation system provided in Embodiment 1 of the control strategy generation method for brain-controlled rehabilitation equipment according to this application. The brain-generated electroencephalogram (EEG) signals are acquired by an EEG acquisition module and processed by a host computer. The algorithm module inside the host computer employs a parallel dual-path decoding architecture to decode the acquired SSVEP and MI signals and generate control commands. These control commands are then sent to a pneumatic rehabilitation glove via serial communication to drive the glove to perform corresponding rehabilitation movements. Simultaneously, a display screen presents visual stimuli or guidance to the user via HDMI to induce specific EEG signals. Through this closed-loop rehabilitation path, the entire system achieves precise control of hand movements while balancing the initiative of rehabilitation training with the high efficiency of intention recognition, making it suitable for intelligent auxiliary treatment in the rehabilitation of nerve injuries.

[0102] In this embodiment, the EEG acquisition module includes a DSI-24 multi-channel EEG acquisition system, which includes a dry electrode EEG cap, a signal amplifier, and an acquisition terminal. The multi-channel EEG acquisition system transmits the acquired multi-channel EEG signals to the algorithm module in the host computer for processing in real time via the Lab Streaming Layer (LSL) protocol.

[0103] Specifically, please refer to Figure 5 , Figure 5 This is a schematic diagram of the fusion visual guidance process provided in Embodiment 1 of the control strategy generation method for brain-controlled rehabilitation devices according to this application. The brain generates SSVEP signals when viewing visual stimuli flashing at different frequencies, and simultaneously performs motor imagery to generate corresponding EEG signals. After unified preprocessing, these signals are decoded through two decoding paths: one path specifically decodes the SSVEP signals, and the other path decodes the MI signals. During decoding, the SSVEP signals undergo high-precision frequency identification through task-related component analysis using a filter group, while the MI signals are decoded through a Mamba-STA-Net network architecture that fuses a state-space model and dynamic routing attention. The decoding results from the two paths are intelligently weighted and fused through an adaptive weighted fusion unit based on the probability domain, ultimately generating control commands to drive rehabilitation gloves or other rehabilitation devices to perform diverse hand movements, achieving precise and proactive rehabilitation training.

[0104] The visual guidance interface module adopts a fusion-style induction interface design. While playing the motion imagery training video, multiple visual stimulus blocks flashing at different frequencies are superimposed on the edge area of ​​the interface to induce SSVEP signals. While watching the video, users can actively input commands to switch system modes, action categories, or states by staring at the flashing area at a specified frequency, without having to switch interfaces, thus ensuring the continuity and focus of the interaction process.

[0105] The algorithm module, located in the host of the brain-controlled rehabilitation system, is used for preprocessing, feature extraction, fusion classification, and action determination of the acquired EEG signals. The preprocessing stage includes filtering, downsampling, and artifact removal to eliminate interference factors such as electrooculography (EOG). The core classification algorithm uses a self-designed fusion decoding network—BIM-FusionNet. This algorithm has the ability to process SSVEP and MI signals in parallel, extracting their frequency and spatiotemporal features. It then performs fusion determination through confidence weighting and time window consistency mechanisms, outputting a target control strategy. Based on this strategy, corresponding system control commands can be generated. The system control commands are transmitted from the algorithm module to the pneumatic rehabilitation glove device via serial communication. The pneumatic rehabilitation glove device, through multi-motor drive and a flexible transmission structure, enables various rehabilitation training modes, including right-hand grasping, right-hand progressive flexion-flexion, right-hand point gripping, and right-hand single-finger sequential flexion-extension, meeting the user's personalized rehabilitation needs.

[0106] This embodiment provides a control strategy generation method for brain-controlled rehabilitation devices. First, the brain-controlled rehabilitation system acquires and preprocesses multi-channel EEG signals to obtain a target EEG signal containing SSVEP and MI signals, ensuring the purity and independence of the two signals and providing high-quality input for subsequent accurate decoding. Next, the system decodes the SSVEP signal using task-related component analysis with a filter group through a first decoding path, obtaining a correlation score vector. Using sub-band filtering and spatial projection templates, high-precision frequency identification of the SSVEP signal is achieved, providing a stable and reliable control signal for the system. Subsequently, the system decodes the MI signal using a Mamba dynamic routing spatiotemporal network model through a second decoding path, obtaining a classification score vector. This model effectively improves the robustness and accuracy of MI signal decoding. Then, the system performs posterior probability distribution transformation and weighted fusion on the correlation score vector and classification score vector to obtain a fused probability distribution. Finally, the system generates a target control strategy based on the action category corresponding to the highest probability value in the fused probability distribution, driving the rehabilitation device to complete the corresponding action. This achieves a balance between the control precision of EEG signal decoding and the neural activation effect, providing an efficient and intelligent solution for neurological injury rehabilitation.

[0107] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 6 , Figure 6 This is a flowchart illustrating the second embodiment of the control strategy generation method for brain-controlled rehabilitation devices according to this application. The first decoding path employs task-related component analysis using a filter group. Step S20 of the control strategy generation method for brain-controlled rehabilitation devices includes steps S21 to S26: Step S21: Construct a bandpass filter bank containing multiple sub-bands, and divide the steady-state visual evoked potential signal into sub-bands using the bandpass filter bank to obtain multiple sub-band signals.

[0108] It should be noted that a subband refers to a narrow frequency band signal retained after bandpass filtering. A bandpass filter bank is a collection of multiple bandpass filters with different center frequencies and non-overlapping passbands. Multiple subband signals refer to the frequency band signals output by the bandpass filter bank, with each subband signal corresponding to a specific frequency range.

[0109] Understandably, the system is designed with four second-order Butterworth bandpass filters centered on the commonly used SSVEP frequencies of 8, 10, 12, and 15 Hz, each with a passband width of 2 Hz, connected end to end to form a filter bank; then the bank is applied in parallel to the SSVEP signal to output four non-overlapping sub-band signals at once, each containing only the component of the corresponding center frequency ±1 Hz.

[0110] Subband filter group construction: The acquired raw EEG signal ,in For the number of channels, This refers to the time duration. To enhance the ability to distinguish between different frequency responses, the system is pre-configured. Bandpass filter banks of different frequency bands The original signal is divided into subbands: in, Indicates the first The signal is filtered by the individual bands.

[0111] Step S22: For each preset target frequency, construct a corresponding neural response waveform reference template based on the training dataset.

[0112] It should be noted that the preset target frequencies refer to a set of fixed flicker frequency values ​​pre-set by the system to induce SSVEP. The training dataset refers to the multi-channel EEG data collected during the experimental phase, with labels corresponding to the aforementioned preset target frequencies. The neural response waveform reference template refers to a time-series template representing the typical EEG response at each preset target frequency, obtained by averaging and spatial filtering on the training dataset, and is used for subsequent correlation matching.

[0113] Understandably, the system first selects all trials with the same preset target frequency from the training dataset by label. For each trial, it extracts 0-1000 ms of occipital channel data after the stimulus onset, removes bad segments, and retains several signals. A moving average is used to remove DC, ensuring waveform alignment and baseline consistency. Second, the system averages the retained signals across trials to obtain a 1×1000-point mean waveform. Task-related component analysis is then performed on this mean waveform, using generalized eigenvalue decomposition to find the top three spatial filters that maximize intra-class correlation. The filtered signals are averaged again to obtain a 1×1000 waveform, which serves as the reference template for that frequency. Finally, the system stores the four templates in frequency order into a dictionary and serializes them to a file. These are then directly loaded during subsequent online decoding, avoiding redundant calculations and improving real-time performance.

[0114] Step S23: Extract the spatial filter of the preset target frequency using a task-related component analysis algorithm.

[0115] It should be noted that Task Related Component Analysis (TRCA) involves, during the training phase, calculating the intra-class covariance matrix (the cross-correlation between trials at the same frequency) and the overall covariance matrix (the total energy of all trials) for all trial data at the same preset target frequency. Then, it solves the generalized eigenvalue problem to obtain a set of eigenvectors arranged in descending order of eigenvalues; these vectors are the spatial filters. The spatial filter has the same dimension as the number of channels. By performing an inner product with the multi-channel EEG signal, the original high-dimensional data can be projected into a one-dimensional time series, maximizing inter-trial consistency while preserving the target frequency energy and suppressing non-target activities and noise.

[0116] As an example, the step of extracting the spatial filter of the preset target frequency using the task-related component analysis algorithm includes: obtaining a training dataset corresponding to the preset target frequency, the training dataset including EEG signals from multiple trials; calculating the intra-class covariance matrix and the global covariance matrix based on the EEG signals; and solving the intra-class covariance matrix and the global covariance matrix using the generalized eigenvalue decomposition algorithm to obtain the spatial filter.

[0117] Multiple trials of EEG signals refer to several segments of multi-channel SSVEP recordings acquired by the system at the same preset target frequency, with each segment corresponding to one visual stimulus trial. The intra-class covariance matrix is ​​a matrix calculated from these trial data, describing the channel cross-correlation structure among trials within the same frequency. The global covariance matrix is ​​a matrix calculated by combining all trial data, describing the overall channel energy and correlation. The generalized eigenvalue decomposition algorithm is a numerical method for solving the generalized eigenvalue problem, used to obtain the optimal spatial filter vector from the intra-class covariance matrix and the global covariance matrix.

[0118] First, the system reads all trial EEG signals of the same target frequency from the training library according to labels. For each trial, the system extracts the stimulus 1 second after each trial, arranges the channels in sequence into a three-dimensional tensor, and then flattens it into a matrix for later use. This ensures that all samples are time-aligned and have consistent channel order. Second, the system calculates the within-class covariance matrix of the flattened matrix: first, it calculates the channel cross-correlation between every two trials and takes the average, obtaining... Matrix; then calculate the overall covariance matrix: after concatenating all trials into an overall matrix, calculate the channel covariance to obtain The matrices, representing class consistency and overall energy respectively, facilitate subsequent maximization of intra-class similarity. Finally, the system solves the generalized eigenvalue equation, takes the eigenvector corresponding to the largest eigenvalue and normalizes it to obtain the spatial filter; this vector retains the most intra-class energy and suppresses global noise, and can be directly used for online projection.

[0119] For each target frequency Construct a reference template And use the TRCA algorithm to extract its spatial filter. To enhance the stable signal characteristics between different trials at the target frequency, among which, For the number of channels, This refers to the number of time points. The specific steps are as follows: (1) Construct the within-class covariance matrix Let the training set be the first... The data from this experiment are There are a total of This experiment. The within-class covariance matrix is ​​then defined as: in, It refers to the first EEG signal data from this experiment It refers to the first EEG signal data from this experiment This represents the transpose of a matrix.

[0120] (2) Construct the overall covariance matrix : (3) Optimize the spatial filter, the goal of which is to solve the spatial filter. This maximizes the within-class covariance and normalizes the overall energy. in, Indicates the optimal spatial filter; This represents the weight vector to be optimized. The formula means that among all possible weight vectors... In this problem, we seek a weight vector that maximizes the ratio of the objective function to constraints or penalty terms. The optimal solution to this optimization problem can be obtained using the generalized eigenvalue decomposition method, which aims to maximize the objective function while satisfying the constraints.

[0121] Step S24: Project the sub-band signal onto the spatial filter of the corresponding frequency to obtain the projected signal.

[0122] It should be noted that the projection signal refers to the signal that projects the first... The one-dimensional time series obtained by linearly projecting the sub-band signal corresponding to a preset frequency onto a dedicated spatial filter of that frequency retains only the frequency component and suppresses other frequency bands and noise.

[0123] Understandably, the system puts the first A subband signal corresponding to a preset target frequency (e.g., 10Hz) is multiplied by a dedicated spatial filter for that frequency, resulting in a one-dimensional time series, i.e., a projected signal. Its amplitude is greatly enhanced at the 10Hz frequency, while other frequencies and background noise are significantly attenuated.

[0124] Step S25: Calculate the Pearson correlation coefficient between the projection signal and the neural response waveform reference template.

[0125] It should be noted that the Pearson correlation coefficient is a scalar measure of the linear similarity in the time domain between the projected signal and the neural response waveform reference template, and its value ranges from [value missing]. The closer the value is to 1, the more consistent the waveforms of the two waveforms are.

[0126] Understandably, after aligning the projected signal with the neural response waveform reference template by time point, the system first calculates the mean of both signals separately, then calculates the sum of the products of the differences between corresponding points in the two sequences and divides the product of their respective standard deviations to obtain the Pearson correlation coefficient.

[0127] Step S26: Weighted fusion of the Pearson correlation coefficients of all subbands to obtain the correlation score vector.

[0128] Understandably, the system calculates the Pearson correlation coefficients for each subband and sums them by weighting them with preset decreasing weights (e.g., 1, 0.9, 0.8, 0.7) to generate a correlation score vector.

[0129] EEG signal to be tested ,in, For the number of channels, To determine the number of time points, first obtain the number of time points using the same sub-band filter bank at the [number of time points]. Size with weight: in, It refers to the electroencephalogram (EEG) signal to be measured.

[0130] Then project it onto the first Sub-band, for the first Target frequency trained spatial filter superior: Next, the projected signal is calculated. Reference template corresponding to this frequency Pearson correlation coefficient between : To integrate information from all subbands, for each target frequency Final response score By examining all The correlation coefficients of each subband are weighted and fused to obtain: in, For the first The weighting factor for subbands is usually set as a decreasing weight based on bandwidth or experience; It is an index used to adjust the contribution level of each item.

[0131] Final output: Correlation Score Vector.

[0132] Thus, this application is Different target frequencies (i.e.) A final response score is calculated for each category. Unlike the traditional "winner-takes-all" strategy, the first decoding path of this application does not make a final decision at this stage.

[0133] Instead, it will all The response scores for each category are combined to form a relevance score vector. And this is taken as the final output of the decoding path: This score vector fully preserves the correlation strength information between the tested signal and each possible category. It serves as "evidence" for the SSVEP branch and is fed into the fusion decision mechanism unit, where it is transformed into a standard posterior probability distribution and finally intelligently fused with the output of the MI path. This design ensures lossless information transfer during the final decision-making stage, which is crucial for achieving high-precision fusion.

[0134] This embodiment first constructs a bandpass filter bank containing multiple sub-bands, and then uses this filter bank to divide the SSVEP signal into multiple sub-band signals. This helps to separate different frequency components and improve the accuracy of signal processing. Next, for each preset target frequency, a corresponding neural response waveform reference template is constructed based on the training dataset, providing a standard for subsequent matching and enhancing recognition reliability. Then, a spatial filter for the preset target frequency is extracted using a task-related component analysis algorithm to enhance the target signal and suppress noise. Subsequently, the sub-band signal is projected onto the spatial filter of the corresponding frequency to obtain a projected signal that highlights the target frequency characteristics. Next, the Pearson correlation coefficient between the projected signal and the neural response waveform reference template is calculated to quantify the similarity between the signal and the template. Finally, the Pearson correlation coefficients of all sub-bands are weighted and fused to obtain a correlation score vector that comprehensively reflects the degree of signal matching, improving the accuracy and robustness of signal recognition, providing a basis for the generation of target control strategies, and balancing the control accuracy and neural activation effect of EEG signal decoding.

[0135] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the control strategy generation method for brain-controlled rehabilitation devices. Any simple modifications based on this technical concept are within the protection scope of this application.

[0136] This application also provides a control strategy generation device for brain-controlled rehabilitation equipment. Please refer to [link / reference]. Figure 7 The control strategy generation device for brain-controlled rehabilitation equipment includes: The preprocessing module 10 is used to acquire multi-channel EEG signals and preprocess the multi-channel EEG signals to obtain target EEG signals. The target EEG signals correspond to preset action categories and include steady-state visual evoked potential signals and motor imagery signals. The first decoding module 20 is used to decode the steady-state visual evoked potential signal through a first decoding path to obtain a correlation score vector. The first decoding path adopts task correlation component analysis of the filter group. The second decoding module 30 is used to decode the motion image signal through the second decoding path to obtain a classification score vector. The second decoding path adopts the Mamba dynamic routing spatiotemporal network model. The weighted fusion module 40 is used to perform posterior probability distribution transformation and weighted fusion on the relevance score vector and the classification score vector to obtain a fused probability distribution. The strategy generation module 50 is used to generate a target control strategy based on the action category corresponding to the highest probability value in the fusion probability distribution.

[0137] This application provides a control strategy generation device for brain-controlled rehabilitation devices. The control strategy generation device for brain-controlled rehabilitation devices includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the control strategy generation method for brain-controlled rehabilitation devices in the above embodiment 1.

[0138] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the control strategy generation method for brain-controlled rehabilitation devices in the above embodiments.

[0139] The control strategy generation device, equipment, and storage medium for brain-controlled rehabilitation devices provided in this application, employing the control strategy generation method for brain-controlled rehabilitation devices described in the above embodiments, can solve the technical problem of how to obtain a control strategy that balances intention recognition accuracy and rehabilitation neural activation effect. Compared with the prior art, the beneficial effects of this device, equipment, and storage medium are the same as those of the control strategy generation method for brain-controlled rehabilitation devices provided in the above embodiments, and the disclosed features are the same, which will not be repeated here.

[0140] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for generating control strategies for brain-controlled rehabilitation devices, characterized in that, The method includes: Multi-channel EEG signals are acquired and preprocessed to obtain target EEG signals, which correspond to preset action categories. The target EEG signals include steady-state visual evoked potential signals and motor imagery signals. The steady-state visual evoked potential signal is decoded through the first decoding path to obtain a correlation score vector. The first decoding path uses task correlation component analysis of the filter group. The motion image signal is decoded through a second decoding path to obtain a classification score vector. The second decoding path adopts the Mamba dynamic routing spatiotemporal network model. The relevance score vector and the classification score vector are transformed by posterior probability distribution and weighted fusion to obtain a fused probability distribution. A target control strategy is generated based on the action category corresponding to the highest probability value in the fusion probability distribution.

2. The method as described in claim 1, characterized in that, The first decoding path employs task-related component analysis of the filter group; The step of decoding the steady-state visual evoked potential signal through the first decoding path to obtain the correlation score vector includes: A bandpass filter bank containing multiple sub-bands is constructed, and the steady-state visual evoked potential signal is divided into sub-bands by the bandpass filter bank to obtain multiple sub-band signals; For each preset target frequency, a corresponding neural response waveform reference template is constructed based on the training dataset; The spatial filter for the preset target frequency is extracted using a task-related component analysis algorithm; The sub-band signal is projected onto the spatial filter at the corresponding frequency to obtain the projected signal; Calculate the Pearson correlation coefficient between the projected signal and the neural response waveform reference template; The Pearson correlation coefficients of all subbands are weighted and fused to obtain a correlation score vector.

3. The method as described in claim 2, characterized in that, The step of extracting the spatial filter of the preset target frequency using the task-related component analysis algorithm includes: Obtain the training dataset corresponding to the preset target frequency, wherein the training dataset includes EEG signals from multiple trials; Based on the EEG signals, calculate the within-class covariance matrix and the global covariance matrix; The spatial filter is obtained by solving the intra-class covariance matrix and the global covariance matrix using the generalized eigenvalue decomposition algorithm.

4. The method as described in claim 1, characterized in that, The second decoding path adopts the Mamba dynamic routing spatiotemporal network model, which includes a convolutional neural network, a state space model, a routing network, and a linear classifier. The step of decoding the motion image signal through the second decoding path to obtain the classification score vector includes: The shallow spatiotemporal features of the motion image signal are extracted using a convolutional neural network; The shallow spatiotemporal features are modeled using a state-space model to perform long-range temporal dependency modeling, resulting in a deep feature sequence. The importance score of each time point in the deep feature sequence is calculated using a routing network; Dynamic attention weights are generated based on the importance scores; The dynamic attention weights are applied to the deep feature sequence for weighted pooling to generate a context feature vector; The context feature vector is processed by a linear classifier to obtain a classification score vector.

5. The method as described in claim 1, characterized in that, The construction steps of the Mamba dynamic routing spatiotemporal network model include: A shallow spatiotemporal feature extraction module is constructed based on the first convolutional block and the second convolutional block. The kernel size of the first convolutional block is larger than that of the second convolutional block, and the number of output channels of the first convolutional block is smaller than that of the second convolutional block. A deep temporal context coding module is constructed based on linear layers, state-space models, and gating functions; Build a dynamic routing pooling module based on the routing network; Construct a linear classifier based on the number of preset action categories; The Mamba dynamic routing spatiotemporal network model is constructed based on the shallow spatiotemporal feature extraction module, the deep temporal context encoding module, the dynamic routing pooling module, and the linear classifier.

6. The method as described in claim 1, characterized in that, The step of performing posterior probability distribution transformation and weighted fusion on the relevance score vector and the classification score vector to obtain the fused probability distribution includes: By performing a posterior probability distribution transformation on the relevance score vector and the classification score vector using a normalization function, the relevance probability distribution vector and the classification probability distribution vector are obtained. The correlation probability distribution vector and the classification probability distribution vector are weighted and fused according to the preset steady-state visual evoked potential weight coefficient and the preset motion imagination weight coefficient to obtain the fused probability distribution.

7. The method according to any one of claims 1 to 6, characterized in that, The target EEG signals include steady-state visual evoked potential signals and motor imagery signals; The step of preprocessing the multi-channel EEG signals to obtain the target EEG signal includes: The multi-channel EEG signal is subjected to band-pass filter bank for frequency band separation filtering to obtain a band-separated signal; Based on the functional localization of brain regions, the frequency band separation signal is divided into channel subsets to obtain steady-state visual pathway signals and motor imagery pathway signals; The motion imagery pathway signal is identified and removed by independent component analysis or template matching-based electrooculography artifact detection algorithm to obtain a pure motion imagery signal. The steady-state visual pathway signal and the pure motion imagery signal are subjected to amplitude normalization processing to obtain the steady-state visual evoked potential signal and the motion imagery signal.

8. A control strategy generation device for brain-controlled rehabilitation equipment, characterized in that, The device includes: A preprocessing module is used to acquire multi-channel EEG signals and preprocess the multi-channel EEG signals to obtain target EEG signals. The target EEG signals correspond to preset action categories and include steady-state visual evoked potential signals and motor imagery signals. The first decoding module is used to decode the steady-state visual evoked potential signal through a first decoding path to obtain a correlation score vector. The first decoding path adopts task correlation component analysis of the filter group. The second decoding module is used to decode the motion image signal through the second decoding path to obtain a classification score vector. The second decoding path adopts the Mamba dynamic routing spatiotemporal network model. The weighted fusion module is used to perform posterior probability distribution transformation and weighted fusion on the relevance score vector and the classification score vector to obtain a fused probability distribution. The strategy generation module is used to generate a target control strategy based on the action category corresponding to the highest probability value in the fusion probability distribution.

9. A control strategy generation device for brain-controlled rehabilitation equipment, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the control strategy generation method for a brain-controlled rehabilitation device as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the control strategy generation method for brain-controlled rehabilitation devices as described in any one of claims 1 to 7.

Images