Control strategy generation method, device and equipment for brain-controlled rehabilitation device and storage medium

Abstract

The application discloses a control strategy generation method and device for brain-controlled rehabilitation equipment, an equipment and a storage medium, relates to the technical field of signal processing, and comprises the following steps: acquiring multi-channel electroencephalogram signals and performing preprocessing to obtain target electroencephalogram signals comprising steady-state visual evoked potential signals and motor imagery signals, wherein the target electroencephalogram signals correspond to preset action categories; decoding the steady-state visual evoked potential signals by using a filter group task-related component analysis to obtain a correlation score vector; decoding the motor imagery signals by using a Mamba dynamic routing spatiotemporal network model to obtain a classification score vector; performing posterior probability distribution conversion and weighted fusion on the correlation score vector and the classification score vector to obtain a fusion probability distribution; and generating a target control strategy according to an action category corresponding to a highest probability value in the fusion probability distribution. The control strategy obtained by the application can take into account both intention recognition accuracy and rehabilitation neural activation effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of signal processing, and particularly relates to a control strategy generation method and device for brain-controlled rehabilitation equipment, a device and a storage medium. BACKGROUND

[0002] Upper limb movement disorder (such as hand function loss after stroke and spinal cord injury) is one of the most difficult problems in neural rehabilitation. Traditional treatment relies on manual methods, robot assistance or electrical stimulation, but the patient's active participation is low, the training process is rigid, and there is a lack of personalized feedback, which makes it difficult to meet the continuous, efficient and home-based rehabilitation needs. Brain-computer interface (BCI) technology through the "brain-device-feedback" closed loop is expected to rebuild neural plasticity; among them, steady-state visual evoked potential (SSVEP) has the advantages of high recognition rate, fast response and short training period, and motor imagery (MI) can directly activate the motor cortex and has a significant neural activation effect. If the high control precision of SSVEP and the neural activation effect of MI can be used at the same time, the rehabilitation effect will be greatly improved.

[0003] At present, most brain-controlled rehabilitation equipment adopts a single paradigm: (1) SSVEP single paradigm system: under visual stimulation, the occipital region electroencephalogram is collected, and filter bank task-related component analysis (FBTRCA) and other algorithms are used for decoding to output high-precision control instructions to drive the exoskeleton or rehabilitation gloves to complete the action; (2) MI single paradigm system: under the motor imagery task, the sensorimotor cortex electroencephalogram is collected, and convolutional neural network (CNN), recurrent neural network (RNN) or Transformer network is used for decoding to drive the rehabilitation device; (3) simple cascade or hard decision fusion: a few studies try to decode SSVEP and then decode MI, or use voting, gating and other rule-based fusion to realize dual-mode control.

[0004] The current problems with brain-controlled rehabilitation devices are: (1) Inability to balance accuracy and neural activation effect: The SSVEP system has precise control but lacks active activation of the motor cortex; the MI system has neural activation effect, but recognition is unstable due to low signal-to-noise ratio, nonlinearity and individual differences. (2) Bottleneck in MI signal decoding: Traditional CNN / RNN / Transformer cannot simultaneously capture the long-term temporal dependence and spatial features of electroencephalogram (EEG) signals, and has high computational complexity and poor individual generalization. (3) The fusion strategy is too simple: The existing cascade or hard decision mechanism has low information utilization rate, poor robustness in the face of signal conflict, noise or quality fluctuation, and cannot dynamically balance the confidence of the two paths. Therefore, how to obtain a control strategy that balances the accuracy of intention recognition and the effect of rehabilitation neural activation has become an urgent problem to be solved.

[0005] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The purpose of this application is to provide a method, device, equipment and storage medium for generating control strategies for brain-controlled rehabilitation devices, aiming to solve the technical problem of how to obtain control strategies that take into account both the accuracy of intention recognition and the effect of rehabilitation neural activation.

[0007] To achieve the above objectives, this application proposes a method for generating control strategies for brain-controlled rehabilitation devices, the method comprising:

[0008] Multi-channel EEG signals are acquired and preprocessed to obtain target EEG signals, which correspond to preset action categories. The target EEG signals include steady-state visual evoked potential signals and motor imagery signals.

[0009] The steady-state visual evoked potential signal is decoded through the first decoding path to obtain a correlation score vector. The first decoding path uses task correlation component analysis of the filter group.

[0010] The motion image signal is decoded through a second decoding path to obtain a classification score vector. The second decoding path adopts the Mamba dynamic routing spatiotemporal network model.

[0011] The relevance score vector and the classification score vector are transformed by posterior probability distribution and weighted fusion to obtain a fused probability distribution.

[0012] A target control strategy is generated based on the action category corresponding to the highest probability value in the fusion probability distribution.

[0013] In addition, to achieve the above object, the application further provides a control strategy generation device for a brain-controlled rehabilitation device.

[0014] The preprocessing module is configured to acquire a multi-channel electroencephalogram signal, and preprocess the multi-channel electroencephalogram signal to obtain a target electroencephalogram signal corresponding to a preset action category, the target electroencephalogram signal including a steady-state visual evoked potential signal and a motor imagery signal.

[0015] The first decoding module is configured to decode the steady-state visual evoked potential signal through a first decoding path to obtain a correlation score vector, the first decoding path adopting a filter set task-related component analysis.

[0016] The second decoding module is configured to decode the motor imagery signal through a second decoding path to obtain a classification score vector, the second decoding path adopting a Mamba dynamic routing spatio-temporal network model.

[0017] The weighted fusion module is configured to perform posterior probability distribution conversion and weighted fusion on the correlation score vector and the classification score vector to obtain a fusion probability distribution.

[0018] The strategy generation module is configured to generate a target control strategy according to an action category corresponding to a highest probability value in the fusion probability distribution.

[0019] In addition, to achieve the above object, the application further provides a control strategy generation device for a brain-controlled rehabilitation device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the control strategy generation method for the brain-controlled rehabilitation device.

[0020] In addition, to achieve the above object, the application further provides a storage medium, which is a computer-readable storage medium, and the storage medium stores a computer program, the computer program being executable by a processor to implement the steps of the control strategy generation method for the brain-controlled rehabilitation device.

[0021] The one or more technical solutions provided by the application have at least the following technical effects:

[0022] Firstly, the brain-controlled rehabilitation system acquires multi-channel electroencephalogram signals and pre-processes them to obtain target electroencephalogram signals containing steady-state visual evoked potential signals and motor imagery signals, ensuring the purity and independence of the two signals and providing high-quality input for subsequent accurate decoding. Then, the system decodes the steady-state visual evoked potential signals through the first decoding path using filter set task-related component analysis to obtain a correlation score vector, and uses sub-band filtering and spatial projection template construction to achieve high-precision frequency identification of the steady-state visual evoked potential signals, providing stable and reliable control signals for the system. Subsequently, the system decodes the motor imagery signals through the second decoding path using the Mamba dynamic routing spatio-temporal network model to obtain a classification score vector, which effectively improves the decoding robustness and accuracy of the motor imagery signals. Then, the system converts and weights the correlation score vector and the classification score vector to obtain a fused probability distribution. Finally, the system generates a target control strategy according to the action category corresponding to the highest probability value in the fused probability distribution to drive the rehabilitation device to complete the corresponding action, achieving a balance between control accuracy of electroencephalogram signal decoding and neural activation effect, and providing an efficient and intelligent solution for neural injury rehabilitation. BRIEF DESCRIPTION OF DRAWINGS

[0023] The accompanying drawings, which are incorporated into and form a part of the specification, illustrate an embodiment consistent with the present application and, together with the description, serve to explain the principles of the application.

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0025] Figure 1 A flowchart provided for the control strategy generation method for brain-controlled rehabilitation equipment embodiment one of the present application;

[0026] Figure 2 A preprocessing flowchart provided for the control strategy generation method for brain-controlled rehabilitation equipment embodiment one of the present application;

[0027] Figure 3 A structure diagram of the brain-controlled rehabilitation system provided for the control strategy generation method for brain-controlled rehabilitation equipment embodiment one of the present application;

[0028] Figure 4 A flowchart of the brain-controlled rehabilitation system provided for the control strategy generation method for brain-controlled rehabilitation equipment embodiment one of the present application;

[0029] Figure 5A fusion type visual guidance process schematic diagram provided by the control strategy generation method for the brain control rehabilitation device in Embodiment 1 of the application;

[0030] Figure 6 A process schematic diagram provided by the control strategy generation method for the brain control rehabilitation device in Embodiment 2 of the application;

[0031] Figure 7 A module structure schematic diagram of the control strategy generation device for the brain control rehabilitation device in the embodiment of the application.

[0032] The object implementation, functional features and advantages of the application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0033] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the application, and are not used to limit the application. In order to better understand the technical solutions of the application, the specific embodiments will be described in detail below with reference to the drawings and specific embodiments.

[0034] It should be noted that the execution subject of the embodiment of the application can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a brain control rehabilitation system, etc. capable of realizing the above functions. The brain control rehabilitation system is taken as an example to describe the embodiment and the following embodiments.

[0035] Based on this, the embodiment of the application provides a control strategy generation method for a brain control rehabilitation device, which is described in detail with reference to Figure 1 , Figure 1 A process schematic diagram of the control strategy generation method for the brain control rehabilitation device in the first embodiment of the application.

[0036] In the embodiment, the control strategy generation method for the brain control rehabilitation device includes steps S10-S50:

[0037] Step S10, acquiring a multi-channel electroencephalogram signal, and preprocessing the multi-channel electroencephalogram signal to obtain a target electroencephalogram signal, the target electroencephalogram signal corresponding to a preset action category, the target electroencephalogram signal including a steady-state visual evoked potential signal and a motor imagery signal.

[0038] It should be noted that the multi-channel electroencephalogram signal refers to the electroencephalogram activity data collected from different positions on the scalp at the same time through multiple electrodes, reflecting the synchronous discharge information of neurons in multiple functional areas of the brain. The target electroencephalogram signal refers to the part of the electroencephalogram data retained after preprocessing and denoising, which contains both SSVEP and MI components, and can reliably correspond to the specific intention of the user.

[0039] The preset action category refers to a number of hand rehabilitation actions (such as grasping, single finger flexion, point flexion, etc.) defined by the system in advance, which is used to establish a one-to-one correspondence with the target brain electrical signal, and realize the mapping of intention to action. The SSVEP signal refers to the periodic brain electrical response generated in the occipital cortex and synchronized with the stimulation frequency when the brain continuously watches the fixed frequency flickering visual stimulus. The MI signal refers to the brain electrical activity with event-related synchronization / desynchronization characteristics that appears in the motor-sensory cortex when the subject only mentally simulates a certain limb action without actually performing it.

[0040] It can be understood that different target brain electrical signals correspond to different action categories, the flickering frequency of the SSVEP signal is different, that is, it corresponds to different action categories, and the spatiotemporal characteristics of the MI signal generated are different, that is, it corresponds to different action categories.

[0041] In the brain-controlled rehabilitation system, each hand rehabilitation action category is bound to the brain intention through two parallel brain electrical "fingerprints": (1) Frequency fingerprint - SSVEP signal: the visual guidance interface is represented by small blocks flickering at different fixed frequencies (such as 8 Hz, 10 Hz, 12 Hz, etc.), and different action categories; when the user gazes at a certain frequency block, the SSVEP energy peak value generated in the occipital lobe is exactly the frequency, and the system locks the frequency as the identifier of the corresponding action category. (2) Feature fingerprint - MI signal: motor imagination tasks (imagining right hand grasping, imagining right hand single finger flexion, etc.) will induce specific space-time-frequency patterns of rhythmic event-related desynchronization / resynchronization (ERD / ERS) in the motor-sensory cortex. These patterns will be mapped into a unique feature vector, and each vector corresponds to a preset action category. Therefore, as long as one target brain electrical signal presents a certain specific SSVEP frequency or a certain specific MI feature, the system will uniquely classify it as the action category bound to it; if both appear at the same time, the system will integrate the probabilities of the two, and finally still point to a clear hand rehabilitation action with the highest confidence.

[0042] Therefore, as long as one target brain electrical signal presents a certain specific SSVEP frequency or a certain specific MI feature, the system will uniquely classify it as the action category bound to it; if both appear at the same time, the system will integrate the probabilities of the two, and finally still point to a clear hand rehabilitation action with the highest confidence.

[0043] As an example, the target EEG signal includes a steady-state visual evoked potential signal and a motor imagery signal; the step of preprocessing the multi-channel EEG signal to obtain the target EEG signal includes: performing band separation filtering on the multi-channel EEG signal using a bandpass filter bank to obtain a band-separated signal; dividing the band-separated signal into channel subsets according to brain region functional localization to obtain a steady-state visual pathway signal and a motor imagery pathway signal; performing artifact recognition and removal on the motor imagery pathway signal using independent component analysis or an electrooculography artifact recognition algorithm based on template matching to obtain a clean motor imagery signal; and performing amplitude normalization processing on the steady-state visual pathway signal and the clean motor imagery signal to obtain the steady-state visual evoked potential signal and the motor imagery signal.

[0044] A bandpass filter bank is a collection of bandpass filters with different center frequencies and non-overlapping passbands, used to divide the raw EEG signal into several sub-bands according to frequency bands. For SSVEP signals, based on a preset stimulation frequency... Construct a set of narrowband bandpass filters to extract frequency bands from the signal:

[0045]

[0046] in, Indicates the first The SSVEP signal corresponding to a preset target frequency in time The value of is the sub-band signal obtained after bandpass filtering; BandPass indicates the bandpass filtering operation, which is used to extract components within a specific frequency range from the original signal; This indicates the original multichannel EEG signals in time. The value; It is the first The center frequency of the bandpass filter range of a preset target frequency.

[0047] For MI signals, an 8–30Hz bandpass filter is uniformly applied to extract the relevant data. and Rhythmic components:

[0048]

[0049] in, This indicates the time of the MI signal after bandpass filtering. The value of .

[0050] Band-separated signals refer to a set of sub-band signals distributed in different frequency bands after being processed by a band-pass filter set. Brain region function localization is to divide the multi-channel electroencephalogram electrode into a spatial correspondence of visual-related regions and motor-related regions according to the anatomical and functional distribution of the visual cortex and motor cortex of the brain. SSVEP-pathway signals are electroencephalogram signals extracted from occipital region electrodes and used for further decoding of SSVEP after band separation. MI-pathway signals are electroencephalogram signals extracted from motor cortex region electrodes and used for further decoding of motor imagery features after band separation. Independent component analysis (ICA) is a blind source separation method that identifies and removes eye movement artifacts by calculating statistically independent spatial components. Electrooculogram template-matching artifact removal is a technique that uses pre-established electrooculogram artifact templates to detect the correlation between the signal segment and the artifact, thereby locating and removing the electrooculogram artifact. Pure motor imagery signals refer to the neural source electroencephalogram activity related to motor imagery after removing eye movement, electromyography and other artifacts. Motor imagery branch channels (C3, C4, Cz) are susceptible to eye movement (EOG) and electromyography (EMG) interference.

[0051] Please refer to Figure 2 , Figure 2 The pre-processing flowchart provided for the control strategy generation method embodiment one of the brain-controlled rehabilitation device of the present application first uniformly pre-processes the original electroencephalogram signal, including steps such as filtering, channel selection and spatial division, artifact removal and signal standardization, to construct a high-quality input basis required for decoding. Subsequently, the SSVEP signal branch and the MI signal branch are further processed: after frequency filtering and channel selection, the SSVEP signal branch is subjected to channel and spatial division, and finally outputs the pre-processed SSVEP signal; while the MI signal branch, after completing frequency filtering and channel selection, still needs to be subjected to artifact removal, such as ICA or electrooculogram artifact recognition algorithm based on template matching, to remove non-neural source artifact signals such as eye movement, and finally subjected to signal standardization, to output the pre-processed MI signal. This process ensures the decoupling of the two signals in spatial distribution, helps to improve the discriminability of subsequent model feature extraction, and at the same time guarantees the pathway independence and functional complementarity between SSVEP signals and MI signals, laying a solid foundation for constructing a high-robustness dual-mode brain-controlled signal basis.

[0052] Firstly, the brain-controlled rehabilitation system calls a set of preset band-pass filter banks to filter the original multi-channel electroencephalogram signal channel by channel in 8-30 Hz, 30-45 Hz and other non-overlapping narrow bands, so as to split the aliasing wide frequency signal into multiple frequency band signals, and lay a clear foundation for subsequent extraction of SSVEP and MI required frequency bands. Secondly, the system classifies the channel data of the occipital region (O1, O2, Oz) into the steady-state visual pathway and the channel data of the sensory motor region (C3, C4, Cz) into the motor imagery pathway according to the pre-labeled visual cortex and motor cortex spatial coordinates, so as to ensure that the two pathways do not interfere with each other in space and improve the feature purity. Then, the system centralizes and whitens the motor imagery pathway signal, calls FastICA (independent component analysis algorithm) to extract 20 independent components with symmetric approximation and tanh non-linear function; for each component, the system calculates the Pearson correlation coefficient between the component and the pre-stored eye template (average waveform of blinking) with a 100 ms sliding window, and if the correlation coefficient is greater than 0.35, the mixed matrix vector corresponding to the component is set to zero as a whole, and the remaining components remain unchanged. Finally, the remaining components are reconstructed in the time domain through the inverse mixed matrix to complete the eye blink artifact removal and avoid misjudgment of the intention. Finally, the system calculates the mean and standard deviation of the signals obtained from the two pathways channel by channel, performs z-score normalization to unify the signal amplitude in the range of zero mean and unit variance, eliminates individual differences and channel gain differences, and ensures the consistency and robustness of the input of the subsequent SSVEP and MI decoding model.

[0053] Step S20, decoding the steady-state visual evoked potential signal through a first decoding path to obtain a correlation score vector, the first decoding path adopting filter bank task-related component analysis.

[0054] It should be noted that the first decoding path refers to an algorithm pipeline in the system that specifically processes SSVEP signals, and its input is the SSVEP data of the occipital channel, and its output is used for subsequent fusion decision. The correlation score vector is a multi-dimensional vector, each dimension giving the correlation coefficient between the signal to be tested and the corresponding frequency template, and fully retaining its matching strength with each candidate category. Filter bank task-related component analysis (Filter Bank Task-Related Component Analysis, FBTRCA) is a multi-subband spatial filtering method that first extracts the spatial projection that maximizes the intra-class consistency of each subband signal, and then fuses the subband correlation coefficients to achieve high-precision identification of SSVEP frequencies.

[0055] It can be understood that the FBTRCA can effectively deal with the feature differences of the SSVEP in different frequency bands, enhance the sensitivity to each frequency component through sub-band filtering, and then use the task-related component analysis (TRCA) to extract a spatial filter highly related to the target frequency, maximize the consistency of the intra-class signal, and thus accurately identify the target frequency under complex background noise, provide a high-precision and high-stability control signal for the system, and ensure the reliability of the subsequent fusion decision.

[0056] In step S30, the motor imagery signal is decoded through a second decoding path to obtain a classification score vector, and the second decoding path adopts a Mamba-based spatio-temporal attention network model.

[0057] It should be noted that the second decoding path refers to an algorithm channel in the system specially used for processing the MI signal, which takes the motor cortex channel electroencephalogram as input and outputs a confidence vector for the fusion decision. The second decoding path aims to solve the technical pain points of the MI signal, such as low signal-to-noise ratio, complex long-range temporal dependence, and significant individual differences.

[0058] The classification score vector is a vector with a dimension equal to the number of action categories, and each element represents the original logit value (logit value) of the MI signal determined as the corresponding category, which can be converted into a posterior probability through Softmax (Softmax function).

[0059] The Mamba-based spatio-temporal attention network model (Mamba-STA-Net) is an end-to-end network that extracts shallow spatio-temporal features by a one-dimensional convolutional neural network (1D-CNN), captures long-range temporal dependence by a Mamba state space module, focuses on key time points by dynamic routing pooling, and finally outputs classification scores through a linear layer. Through a "Hierarchical Feature Extraction" strategy, the network model combines the local perception ability of the convolutional neural network, the long-range temporal modeling ability of the Mamba state space model, and the dynamic routing pooling mechanism originally invented by the present application, achieving high robustness and high precision decoding of the MI signal.

[0060] As an example, the second decoding path adopts a Mamba dynamic routing spatio-temporal network model, the Mamba dynamic routing spatio-temporal network model comprising a convolutional neural network, a state space model, a routing network and a linear classifier; the step of decoding the motor imagery signal through the second decoding path to obtain a classification score vector comprises: extracting shallow spatio-temporal features of the motor imagery signal through the convolutional neural network; modeling long-range temporal dependence of the shallow spatio-temporal features by using the state space model to obtain a deep feature sequence; calculating an importance score of each time point in the deep feature sequence through the routing network; generating a dynamic attention weight based on the importance score; applying the dynamic attention weight to weighted pooling of the deep feature sequence to generate a context feature vector; processing the context feature vector through the linear classifier to obtain a classification score vector.

[0061] The convolutional neural network refers to a shallow feature extractor composed of two layers of lightweight one-dimensional convolutional layers, batch normalization and ELU (Exponential Linear Unit) activation, which is used to automatically learn local spatio-temporal filters on the MI signal and output low-dimensional feature maps. The state space model (SSM-Mamba) refers to a deep sequence encoder with Mamba-S6 core, which maps the feature sequence output by the CNN into a high-dimensional hidden state sequence through parameterized input dependence and parallel scanning algorithm, capturing long-distance temporal relationship. The routing network refers to an attention weight generator composed of a two-layer MLP (Multilayer Perceptron), which inputs each frame of deep feature sequence and outputs a scalar of discriminative contribution degree of the corresponding frame. The linear classifier refers to a decision module composed of a fully connected layer and a Softmax activation, which maps the context vector after weighted pooling into a class logit vector. The shallow spatio-temporal feature refers to the tensor output by the CNN, whose dimension has been reduced and each element contains the coupling information of local time pattern and spatial channel, but the global time sequence has not been encoded. The deep feature sequence refers to the high-dimensional hidden state sequence output by Mamba-S6, which integrates all context information from the starting time to the current time in each frame. The importance score refers to the scalar value calculated by the routing network for each time step in the sequence, and the larger the value, the more critical the time step to classification. The dynamic attention weight refers to the importance score vector normalized by Softmax, whose distribution is adjusted in real time with the input signal, and is used to aggregate the sequence with weight. The context feature vector refers to the fixed length vector obtained by weighted summing the deep feature sequence with the dynamic attention weight, which concentrates the most discriminative time sequence information to send to the classifier.

[0062] First, the brain-controlled rehabilitation system inputs the MI signal into two layers of 1D-CNN: the first layer uses The convolution kernel slides along the time dimension and filters each channel independently, and after batch normalization and ELU activation, it is down-sampled by 2. The second layer uses Convolution further compresses the space-time dimension, which not only preserves the local motion rhythm but also reduces the sequence length, reducing the subsequent computational burden. Then, the shallow features output by the CNN are sent to Mamba-S6: first linearly expanded to high dimension and split into two parts, one part is updated by parallel scanning after discretization of input-dependent parameters Δ, B, C, and the other part is gated with the state output by SiLU, and after stacking 4 MambaBlocks, a deep feature sequence is obtained; This linear complexity global modeling can capture the long-range dependence of early burst and late recovery of motor imagery, making up for the lack of CNN receptive field. Next, the routing network maps each frame of the deep feature sequence to a scalar importance score through two layers of MLP, and after Softmax, it generates dynamic attention weights. The weighted sum compresses the entire sequence into a 128-dimensional context vector, amplifying the short but discriminative time points and suppressing noise segments. Finally, the linear classifier maps the 128-dimensional vector to the class number through full connection, outputting a logit vector, which is directly used by the fusion module; This design ensures end-to-end training, and the fixed feature dimension is convenient for subsequent confidence weighting.

[0063] As an example, the construction steps of the Mamba dynamic routing spatio-temporal network model include: constructing a shallow spatio-temporal feature extraction module according to a first convolutional block and a second convolutional block, the convolution kernel size of the first convolutional block is larger than the second convolutional block, and the output channel number of the first convolutional block is smaller than the second convolutional block; constructing a deep time series context encoding module according to a linear layer, a state space model and a gating function; constructing a dynamic routing pooling module according to a routing network; constructing a linear classifier according to the number of pre-set action categories; and constructing a Mamba dynamic routing spatio-temporal network model according to the shallow spatio-temporal feature extraction module, the deep time series context encoding module, the dynamic routing pooling module and the linear classifier.

[0064] Mamba-STA-Net includes three core modules, which work together to complete the dynamic modeling and high-robustness classification of MI signals:

[0065] (1) Shallow spatio-temporal feature extraction module (a front-end network composed of a first convolutional block and a second convolutional block, used to extract low-level spatio-temporal features from original EEG signals):

[0066] To avoid the optimization challenges and overfitting risks caused by directly inputting raw high-dimensional EEG signals (characterized by low signal-to-noise ratio and strong non-stationarity) into subsequent complex long-sequence models (such as Mamba), this application designs a dedicated, trainable shallow spatiotemporal feature extraction module at the network front end. The core design philosophy of this module is to learn the basic spatiotemporal structure of the EEG signal in a layered and decoupled manner. Instead of using a fixed filter bank, this application designs a lightweight 1D-CNN as the feature extraction head, enabling it to adaptively learn the optimal feature filters from the data, thereby providing high-quality, information-intensive input for subsequent deep temporal modeling.

[0067] Let the input EEG tensor be ,in This refers to the batch size. This refers to the number of channels. This refers to the number of time points. This module first transforms the input into a format that uses dimensionality compression. The input is then processed through two cascaded convolutional blocks (ConvBlock).

[0068] ① First convolutional block (Temporal Feature Block):

[0069] This module consists of an initial feature extraction unit comprised of temporal convolution, batch normalization, ELU activation, and 2x downsampling. Its kernel size is larger than that of the second convolutional block, but its output channel count is smaller. The kernel size refers to the length of the one-dimensional convolutional kernel in the temporal dimension. and The number of output channels refers to the number of feature maps output by the convolutional layer. and ).

[0070] The core responsibility of this module is to act as a time-characteristic filter, designed to capture the dynamic changes and local timing patterns of the signal from each individual EEG channel. Its detailed calculation process is as follows:

[0071] 1) Temporal Convolution:

[0072] One-dimensional convolution operation is used, and its convolution kernel... Along the time dimension For input Perform sliding calculations. For the first [number] [unit] in the batch... The sample, the first The output channel, the first The first input channel, the first The time point, the first the number of internal time steps, the convolution output is defined as:

[0073]

[0074] where, denotes the input feature tensor of dimension , , ); denotes the trainable weights (convolution kernels) of the first convolutional layer of dimension ; denotes the kernel size of the first convolutional layer; denotes the trainable bias vector of the first convolutional layer of dimension ; denotes the number of input channels; denotes the number of output channels of the first convolutional layer.

[0075] 2) Batch Normalization (BN):

[0076] To solve the internal covariate shift problem and stabilize training, the convolution output is batch normalized. For the th channel, its normalized output is:

[0077]

[0078] where, denotes the input tensor of the batch normalization layer (i.e., the convolution output of the previous step); is a single element of the BN layer input, which denotes the original activation value in the input tensor of the batch normalization (i.e., the convolution output of the previous step) corresponding to the batch , channel , and time point ; , respectively denote the mean and variance of the current batch data on the channel ; , respectively denote the learnable affine transformation parameters (scaling and translation); is a numerically stable small value set to prevent division by zero.

[0079] 3) ELU Activation:

[0080] An exponential linear unit (ELU) activation function is used to enhance the non-linear expression ability of the model:

[0081]

[0082] where, is the input value of the activation function; is the hyperparameter of the ELU function, controlling the saturation region of the negative part, usually set to 1.0.

[0083] 4) Temporal downsampling and regularization:

[0084] The final output of this convolution block can be summarized as:

[0085]

[0086] where, is the batch size, is the number of output channels of the first convolution layer, is the output sequence length of the first convolution block, is the output tensor of the first convolution module.

[0087] ② Second convolution block (Spatio-temporal Fusion Block):

[0088] This module refers to the detailed feature extraction unit that follows the first convolution block and is also composed of temporal convolution, batch normalization, and ELU activation. Its convolution kernel size is smaller than that of the first convolution block, and the number of output channels is greater than that of the first convolution block.

[0089] Based on the output of the previous stage , this module aims to further refine features and learn more fine and abstract local feature pattern combinations through a smaller convolution kernel . Among them, is the number of output channels of the second convolution block (e.g., 32), is the convolution kernel size of the second convolution block (e.g., 3), is the trainable weight of the second convolution layer (convolution kernel).

[0090] The calculation process is exactly the same as the first convolution block and can be summarized by the following formula:

[0091]

[0092] The final output of this module forms the final shallow feature sequence , where, is the output sequence length of the second convolution block, is the output tensor of the second convolution module (the meaning of convolution module and convolution block is the same).

[0093] The sequence not only has lower dimensionality and is easier to process, but also each feature vector contains discriminative local spatio-temporal information learned from the original signal in an end-to-end manner. This high-quality feature sequence will serve as the input of the core MambaEncoder module of the present application, laying a solid foundation for its successful deep temporal context modeling.

[0094] (2) Deep temporal context encoding module (MambaEncoder, an encoder composed of a linear layer, a Mamba-S6 state space model, and a gating function, used for long-range temporal modeling of the shallow feature sequence):

[0095] The core innovation of the present application is to introduce the latest state space model Mamba as the backbone of the network, to deeply process the feature sequence output by the CNNHead, with its linear computational complexity in processing long sequences and strong context dependency modeling capability. Traditional recurrent neural networks (RNNs) are limited by the gradient vanishing / explosion problem and are difficult to capture long-range dependencies; while the Transformer model solves this problem through a self-attention mechanism, but its quadratic computational and memory complexity makes it impractical for processing extremely long sequences such as EEG.

[0096] Mamba achieves a fundamental innovation in architecture through an innovative selective state space mechanism (Selective State Space Mechanism, S6). It not only solves the problem of LTI (Linear Time-Invariant) SSM not being able to adapt to dynamic inputs, but also overcomes the sequential computation bottleneck of RNNs through a hardware-aware parallel scanning algorithm. This enables the MambaEncoder to efficiently and end-to-end learn the complex, long-range temporal dependencies in EEG signals.

[0097] The MambaEncoder receives the feature sequence from the first stage , which is first transposed to , where is the length of the transposed sequence, equal to the number of time points output by the second convolutional block ; is the feature dimension of the transposed sequence, equal to the output channel number of the second convolutional block ; is the sequence obtained after transposing permute(0,2,1), which is to adapt to the (sequence length, feature dimension) format commonly used by the Mamba model.

[0098] The encoder is internally stacked by multiple MambaBlocks, each of which performs a complete selective state space transformation on the input sequence. The workflow can be decomposed into the following steps:

[0099] ① Input linear projection and gating mechanism:

[0100] The MambaBlock first projects the input to a higher dimension through a linear layer and splits it into two parts, and , which are used for subsequent gating computation and state update. The linear layer refers to the weight matrix W and bias , that map the input features to the hidden state. After the MambaBlock internal linear layer projection to a higher dimension , the tensor is obtained; The hidden feature dimension used inside the MambaEncoder (e.g., 128) determines the model's expressive power and computational complexity. All core calculations, including the state space model (SSM) and gating mechanism, are performed in this high-dimensional feature space. The gating input tensor is the upper half of the feature sequence obtained after linear projection and splitting. This tensor is used to calculate the gating vector, which controls the information flow of the S6 module output and determines which context information should be retained. The input tensor of the S6 module is the lower half of the feature sequence obtained after linear projection and splitting, which is used to calculate and drive the dynamic state transition parameters.

[0101]

[0102] where split() is the data splitting function, is the linear transformation function.

[0103] ② Selective state space model (S6) core:

[0104] This is the core computing unit of Mamba, which processes . Its core lies in its input-dependent parametrization, i.e., its key parameters , , are dynamically generated by the input at each time step :

[0105]

[0106]

[0107]

[0108] where, at time step state transition matrix which describes how the system state changes over time; denotes the input vector control input matrix which describes the influence of external inputs on the system state; at time step output matrix which describes how the system state maps to the output; denotes the input vector at time step which is the pre-processed or transformed input data.

[0109] While the state matrix is a trainable but fixed parameter. This design allows the model to dynamically adjust its state update and output reading based on the content of the current input.

[0110] ③ Discretization and State Update:

[0111] For implementation on digital computing devices, the continuous state space equation is discretized through the "Zero-Order Hold (ZOH)" method to obtain the update rule in discrete state. Among them, the discretized state matrix and input matrix are calculated according to the selective parameters :

[0112]

[0113]

[0114] where, denotes the matrix exponential function, which maps the continuous-time system dynamics to discrete-time dynamics; denotes the state transition matrix of the continuous-time system; denotes the identity matrix, which is the same dimension as .

[0115] Note: In actual implementation, a first-order Taylor expansion is usually used for efficient approximation.

[0116] Hidden state of the system The update of the hidden state

[0117]

[0118] where, and denote the hidden state at the previous time step and the current time step, respectively; is the input feature vector at time step , is the dynamically generated, discretized state and input matrix at time step . is the state dimension.

[0119] (4) Selective scanning and convolutional representation:

[0120] The above recursive relation is mathematically equivalent to a time-varying convolution operation. Through a parallel scanning algorithm optimized for modern hardware (GPU), the output of the entire sequence can be efficiently calculated at once, avoiding the sequential computation bottleneck of traditional RNNs. Its convolutional form can be represented as:

[0121]

[0122] where, denotes the system output vector at time ; denotes the output matrix at time step , which describes how the system state maps to the output; denotes the input vector at time step ; denotes the concatenation of all discretized state matrices from time 1 to ; is the dynamically generated, discretized input matrix at time step . The output of the entire sequence is the set of all time steps .

[0123] This reveals the essence of Mamba: a deep convolutional model with an infinite receptive field (theoretically, it can see all information from the beginning to the current time) and a dynamically changing kernel function over time.

[0124] (5) Output and gating activation:

[0125] Finally, the output of the MambaBlock is the output of S6 module is the separated gate vector sequence is the output of element-wise multiplication by a SiLU gate function

[0126]

[0127] where, is the output sequence of S6 module; is the separated gate vector sequence after linear projection of input (the gate input tensor in the previous section, is the upper half of the feature sequence after linear projection and split) is the element-wise multiplication (Hadamard Product), is the output sequence of MambaBlock, which is the final result of MambaBlock processing the original input, and will be used as the input of the next MambaBlock or as the final output of MambaEncoder.

[0128] This gating mechanism further enhances the nonlinearity and selectivity of the model. After the hierarchical processing of all MambaBlocks in MambaEncoder, the final feature sequence integrates the complete and dynamically selected global context information at each time point, providing a high-quality feature basis for subsequent attention focusing.

[0129] (3) Dynamic Routing Pooling (a pooling unit composed of routing network, Softmax normalization and weighted summation, used to compress the depth feature sequence into a fixed-length context vector according to attention weight):

[0130] After MambaEncoder completes the depth temporal context encoding, an efficient pooling mechanism is needed to aggregate the variable-length feature sequence into a fixed-length vector for subsequent classifier use. Although traditional Global Average Pooling (GAP) is simple and efficient, its core defect is that it treats all time steps equally and assigns the same weight . This mechanism "dilutes" the transient features that have a very short duration in the entire signal but are crucial for classification decisions (e.g., specific neural bursts in the initial stage of motor imagination), thus limiting the final performance of the model.

[0131] To solve this problem, the present application designs an adaptive and learnable DynamicRoutingPooling module. The core design philosophy of this module is to continue the original STA idea of the present application, that is, to let the model autonomously and dynamically assign different importance weights to each part of the sequence according to the different input data. It is no longer a static average operation, but a dynamic, weighted summation process focusing on key information.

[0132] The core of this module is a trainable routing network (routing_net), which is a small multi-layer perceptron (MLP) designed to serve as an attention score generator. Its detailed workflow is as follows:

[0133] ① Attention score (“importance”) calculation:

[0134] The output sequence of MambaEncoder is fed into routing_net. This network is a nonlinear transformation that maps the high-dimensional feature vector at each time step to a scalar, unnormalized “importance score” .

[0135]

[0136] where is the input feature vector at time step with dimension ; , are the trainable weight matrix and bias vector of the first hidden layer of routing_net; is the hyperbolic tangent activation function used to introduce nonlinearity; , are the trainable weight vector and bias scalar of the output layer of routing_net; is the final calculated raw score representing the importance of time step .

[0137] After this step, the entire sequence is converted into a score vector .

[0138] ② Attention weight normalization:

[0139] To convert the raw scores into an effective probability distribution, the present application uses the Softmax function on the score vector over the time dimension Normalization is performed to generate a set of attention weights that sum to 1 .

[0140]

[0141] where, is the attention weight at time step The normalized attention weights. The higher the score, the greater the weight corresponding to the point.

[0142] ③ Weighted Pooling:

[0143] Finally, the application uses these attention weights to perform a weighted summation on the entire Mamba output sequence , thereby "condensing" the long sequence into a single context vector that focuses all the key time point information .

[0144]

[0145] where, is the final output fixed-length context vector, represents the system output vector at time (MambaEncoder final output sequence at time feature vector).

[0146] The DynamicRoutingPooling module learns end-to-end, allowing the model to dynamically and intelligently determine which time segments should be paid more attention to based on the characteristics of each EEG sample, while ignoring those irrelevant to the task or full of noise, thereby retaining the discriminative information in the sequence to the greatest extent and greatly improving the final classification accuracy and robustness of the model.

[0147] (4) Classifier Output:

[0148] After the DynamicRoutingPooling module outputs a highly condensed and information-rich context vector , the application needs a final decision layer to map it from a high-dimensional feature space to a task-specific class probability space. To this end, the application uses a standard linear classifier followed by a Softmax activation function to complete the final classification decision.

[0149] ① Linear Transformation:

[0150] Context vector First, a linear transformation is performed by a fully connected layer (nn.Linear) to calculate the original, non-normalized scores for each class, which are referred to as logits in this application wherein, is the number of preset action categories, which refers to the total number of action categories (for example, gripping, flexing, point gripping, etc.) that the rehabilitation device needs to perform, which is also referred to as the number of target frequency categories, which is the number of different SSVEP target frequencies that the system needs to identify, and different frequencies correspond to different action categories, and this value determines the feature dimension of the classifier output .

[0151]

[0152] wherein, is the context vector from the DynamicRoutingPooling module; is the trainable weight matrix of the classifier; is the logit score vector for each class; is the trainable bias vector of the classifier.

[0153] ② Softmax probability output:

[0154] To convert the logit scores into a standard posterior probability distribution, this application applies the Softmax function. For the i-th class, the final prediction probability is calculated as follows:

[0155]

[0156] wherein, is the final probability of the i-th class determined by the MI path; , is the logit vector.

[0157] The probability distribution vector output by this module will be the final decoding result of the MI path (the second decoding path) and will be sent to the fusion decision mechanism unit for intelligent fusion with the output of the SSVEP path.

[0158] In step S40, the correlation score vector and the classification score vector are converted into posterior probability distributions and weighted fused to obtain a fused probability distribution.

[0159] ​​It should be noted that the fusion probability distribution refers to a final probability vector obtained by linearly weighting the relevance score vector and the classification score vector according to preset weights after the relevance score vector and the classification score vector are respectively converted into posterior probability, and each dimension of the final probability vector represents the comprehensive confidence of the corresponding action category.

[0160] As an example, the step of converting the relevance score vector and the classification score vector into posterior probability distribution and fusing by weighting to obtain the fusion probability distribution includes: converting the relevance score vector and the classification score vector into relevance probability distribution vector and classification probability distribution vector through a normalization function; and fusing the relevance probability distribution vector and the classification probability distribution vector by weighting according to a preset steady-state visual evoked potential weight coefficient and a preset motor imagery weight coefficient to obtain the fusion probability distribution.

[0161] The normalization function refers to a Softmax function, which is used to map the original score vector into a probability distribution with a total sum of 1. The relevance probability distribution vector refers to the relevance score vector after Softmax normalization, which represents the posterior probability of the SSVEP branch for each action category. The classification probability distribution vector refers to the classification score vector after Softmax normalization, which represents the posterior probability of the MI branch for each action category. The preset steady-state visual evoked potential weight coefficient refers to a fixed fusion weight pre-allocated by the system to the SSVEP branch, which is used to improve the decision proportion of high signal-to-noise ratio signals. The preset motor imagery weight coefficient refers to a fixed fusion weight pre-allocated by the system to the MI branch, which is used to retain the rehabilitation information of the MI signal in the fusion.

[0162] Firstly, the system sends the relevance score vector and the classification score vector into Softmax respectively, normalizes each component to a probability value of 0-1, and obtains the relevance probability distribution vector and the classification probability distribution vector; then, the relevance probability distribution vector is weighted by 0.7 and the classification probability distribution vector is weighted by 0.3, and the linear addition is performed dimension by dimension, and the fusion probability distribution is output.

[0163] To overcome the inherent limitations of traditional rule-based hierarchical arbitration logic, such as low information utilization rate, rigid decision boundary, poor adaptability to dynamic changes of signal quality, etc., a new decision mechanism for information fusion in the probability domain is proposed. This mechanism no longer relies on a single highest confidence category (“Winner-takes-all”), but makes full use of the complete posterior probability distribution output by the two parallel decoding paths. Through an adaptive weighted fusion model that is more rigorous in mathematics, the final system control instruction is generated. This aims to elevate the fusion decision process from an “expert system” based on artificial prior rules to a more scientific and robust “intelligent arbitration system” driven by data.

[0164] (1) Probabilistic Vector Generation:

[0165] The input of this unit is the raw output from two parallel decoding paths, and it converts them into normalized posterior probability vectors to ensure the scale consistency of the subsequent fusion operation.

[0166] SSVEP path (first decoding path) output: the raw output from the FBTRCA algorithm is a correlation score vector . To convert it into a probability distribution, the present application applies the Softmax function for normalization:

[0167]

[0168] where, is the posterior probability of the th action class determined by the SSVEP path; is the correlation score of the test signal with the frequency template corresponding to the th action class; is the correlation score of the test signal with the frequency template corresponding to the th action class; is the temperature parameter, which is used to adjust the entropy (i.e., the smoothness) of the probability distribution. A smaller will make the probability distribution more "sharp" and tend to hard maximum; a larger will make the distribution more "smooth".

[0169] MI path output: the output from Mamba-STA-Net has already passed through the Softmax classifier at its end, and itself is a standard posterior probability distribution vector .

[0170]

[0171] (2) Probabilistic Weighted Fusion:

[0172] After obtaining the probability vectors and from the two branches, the system calculates the final fusion probability vector through a linear weighted sum model:

[0173]

[0174] where, 、 : are the prior weight coefficients assigned to SSVEP and MI paths respectively, whose values are preset by the system according to the inherent signal-to-noise ratio (SNR) characteristics of the two modalities, and satisfy In this embodiment, considering the high stability and high signal-to-noise ratio of SSVEP signals, is given a higher value (for example, 0.7) to ensure the reliability of the reference of system control, and is given an auxiliary weight (for example, 0.3).

[0175] (3) Final Decision Making:

[0176] The final system control instruction is determined by the class with the highest probability value in the fusion probability vector :

[0177]

[0178] This fusion mechanism can effectively integrate all the information of the two decoding paths through a smooth weighted average in the probability domain, significantly improving the robustness and accuracy of the decision in complex signal scenarios, and has obvious performance advantages compared with traditional hard voting or rule-based arbitration.

[0179] (4) Output Smoothing: Temporal Window Consistency Check:

[0180] To further improve the stability of the control instruction and prevent device jitter or false actions caused by single misjudgment, the present application introduces an output smoothing mechanism. This mechanism performs a consistency vote on the output results in the last time windows (for example = 5).

[0181] Only when more than M (for example, M = 3) results are the same class in the last continuous decision results, the class is confirmed as the final, stable system output instruction and sent to the rehabilitation device for execution. Otherwise, the system will maintain the stable state of the last time and not produce a new control instruction. This introduces a consistency constraint in the time dimension, effectively filtering out transient noise and accidental misjudgments, ensuring the high reliability of the final output instruction and the smoothness of the operation.

[0182] Step S50, generating a target control strategy according to the action class corresponding to the highest probability value in the fusion probability distribution. ​

[0183] It should be noted that the target control strategy refers to converting the preset hand action instruction corresponding to the action category with the highest probability in the fusion probability distribution into a driving parameter sequence executable by the rehabilitation device, so that the rehabilitation device can complete the specified action according to the instruction. The preset hand action instruction refers to a specific hand action command directly executed by the rehabilitation device, such as "right hand grip", "right hand single finger flexion", or "right hand point grip", etc., which is used to drive the pneumatic rehabilitation glove to complete the corresponding rehabilitation training action.

[0184] It can be understood that first, the system obtains the highest probability index by argmax on the fusion probability distribution , and then queries the category-action mapping table to convert into action labels such as "right hand grip"; then the labels are packaged into a structured control strategy object containing category ID, duration, and force level, and immediately stored in the strategy cache to complete the generation of the target control strategy. The target control strategy is used to instantly convert the user's brain electrical intention into an action instruction recognizable by the rehabilitation glove, realizing personalized and low-latency human-computer interaction.

[0185] The brain-controlled rehabilitation system of the present application realizes precise driving of hand actions through wearable devices according to the target control strategy, taking into account the initiative of rehabilitation training and the efficiency of intention recognition, and is suitable for intelligent auxiliary treatment of neural injury rehabilitation.

[0186] The brain-controlled rehabilitation system of the present application includes an electroencephalogram acquisition module, a visual guidance module, an algorithm module for executing the above control strategy generation method, a pneumatic rehabilitation glove device, and a rehabilitation device control module. Among them, the electroencephalogram acquisition module is used to acquire the SSVEP electroencephalogram signal and the MI electroencephalogram signal of the user in real time; the visual guidance module is used to display a synchronous fusion induction interface to guide the user to generate the above-mentioned electroencephalogram signals; the algorithm module is used to acquire the above-mentioned electroencephalogram signals, perform fusion recognition analysis, generate a target control strategy based on the recognition result, and drive the pneumatic rehabilitation glove device to complete the corresponding action.

[0187] Please refer to Figure 3 , Figure 3The structure schematic diagram of the brain control rehabilitation system provided in the embodiment one of the control strategy generation method for the brain control rehabilitation device is provided, in which a user generates electroencephalogram signals through a stimulation guide interface, the signals are collected in real time by an electroencephalogram collection module and transmitted to an algorithm module through an LSL stream protocol. The algorithm module adopts a parallel double-path decoding architecture to decode SSVEP signals and MI signals, and generates control instructions. The control instructions are sent to a pneumatic rehabilitation glove through serial communication, and corresponding rehabilitation actions are performed. The stimulation guide interface is responsible for playing a motor imagery training video while superimposing a plurality of visual stimulation blocks flashing at different frequencies on the edge area of the interface, for inducing SSVEP signals. The user gazes at the flashing area of the specified frequency to realize active instruction input of system mode, action category or state switching. The whole system realizes precise driving of hand actions through this fusion decoding method, taking into account the initiative of rehabilitation training and the efficiency of intention recognition, and is suitable for intelligent auxiliary treatment of neural injury rehabilitation.

[0188] Please refer to Figure 4 , Figure 4 The flowchart of the brain control rehabilitation system provided in the embodiment one of the control strategy generation method for the brain control rehabilitation device is provided, in which electroencephalogram signals generated by the brain are collected by an electroencephalogram collection module and processed by a host computer. The algorithm module in the host computer adopts a parallel double-path decoding architecture to decode the collected SSVEP signals and MI signals, and generates control instructions. These control instructions are then sent to a pneumatic rehabilitation glove through serial communication to drive the glove to complete corresponding rehabilitation actions. At the same time, a display screen presents visual stimulation or guidance to the user through HDMI display to induce specific electroencephalogram signals. The whole system realizes precise driving of hand actions through this closed-loop rehabilitation path, while taking into account the initiative of rehabilitation training and the efficiency of intention recognition, and is suitable for intelligent auxiliary treatment of neural injury rehabilitation.

[0189] In this embodiment, the electroencephalogram collection module includes a DSI-24 multi-channel electroencephalogram collection system, including a dry electrode electroencephalogram cap, a signal amplifier and a collection terminal. The multi-channel electroencephalogram collection system transmits the collected multi-channel electroencephalogram signals in real time to the algorithm module in the host computer through the laboratory streaming layer (LSL) protocol for processing.

[0190] Specifically, please refer to Figure 5 , Figure 5The fusion type visual guidance flowchart provided in the embodiment one of the control strategy generation method for the brain control rehabilitation device is shown in the figure, in which the brain generates SSVEP signals when watching visual stimuli of different frequencies, and at the same time, the brain generates corresponding electroencephalogram signals by imagining motion. After unified preprocessing, these signals are decoded through two decoding paths respectively: one path decodes SSVEP signals, and the other path decodes MI signals. In the decoding process, the SSVEP signals are subjected to high-precision frequency identification through filter group task-related component analysis, and the MI signals are decoded through the Mamba-STA-Net network architecture which fuses the state space model and dynamic routing attention. The decoding results of the two paths are intelligently weighted and fused through a probability domain-based adaptive weighting fusion unit, and finally the control instructions are generated to drive the rehabilitation gloves or other rehabilitation devices to complete diversified hand movements, and realize precise and active rehabilitation training.

[0191] The visual guidance interface module adopts a fusion type induction interface design, that is, a plurality of visual stimulation blocks with different frequencies are superimposed in the edge area of the interface while playing the motion imagination training video, for inducing SSVEP signals. The user can realize active instruction input of system mode, action category or state switching by gazing at the flickering area of the specified frequency while watching the video, without interface switching, so as to ensure the continuity and concentration of the interaction process.

[0192] The algorithm module is arranged in the host of the brain control rehabilitation system, and is used for preprocessing, feature extraction, fusion classification and action decision of the collected electroencephalogram signals. The preprocessing stage includes filtering, downsampling and artifact rejection processing, and interference factors such as electrooculogram are eliminated. The core classification algorithm adopts a fusion decoding network BIM-FusionNet designed independently, which has the ability to process SSVEP and MI signals in parallel, can extract the frequency features and space-time features of the signals, and can output target control strategies through confidence weighting and time window consistency mechanism. The system control instructions are transmitted to the pneumatic rehabilitation glove device through serial communication by the algorithm module, and the pneumatic rehabilitation glove device realizes a plurality of rehabilitation training modes including right hand grasping, right hand gradual finger flexion, right hand finger flexion and right hand single finger sequential flexion through multi-motor driving and flexible transmission structure, so as to meet the individual rehabilitation needs of the user.

[0193] This embodiment provides a control strategy generation method for brain-controlled rehabilitation devices. First, the brain-controlled rehabilitation system acquires and preprocesses multi-channel EEG signals to obtain a target EEG signal containing SSVEP and MI signals, ensuring the purity and independence of the two signals and providing high-quality input for subsequent accurate decoding. Next, the system decodes the SSVEP signal using task-related component analysis with a filter group through a first decoding path, obtaining a correlation score vector. Using sub-band filtering and spatial projection templates, high-precision frequency identification of the SSVEP signal is achieved, providing a stable and reliable control signal for the system. Subsequently, the system decodes the MI signal using a Mamba dynamic routing spatiotemporal network model through a second decoding path, obtaining a classification score vector. This model effectively improves the robustness and accuracy of MI signal decoding. Then, the system performs posterior probability distribution transformation and weighted fusion on the correlation score vector and classification score vector to obtain a fused probability distribution. Finally, the system generates a target control strategy based on the action category corresponding to the highest probability value in the fused probability distribution, driving the rehabilitation device to complete the corresponding action. This achieves a balance between the control precision of EEG signal decoding and the neural activation effect, providing an efficient and intelligent solution for neurological injury rehabilitation.

[0194] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 6 , Figure 6 This is a flowchart illustrating the second embodiment of the control strategy generation method for brain-controlled rehabilitation devices according to this application. The first decoding path employs task-related component analysis using a filter group. Step S20 of the control strategy generation method for brain-controlled rehabilitation devices includes steps S21 to S26:

[0195] Step S21: Construct a bandpass filter bank containing multiple sub-bands, and divide the steady-state visual evoked potential signal into sub-bands using the bandpass filter bank to obtain multiple sub-band signals.

[0196] It should be noted that a subband refers to a narrow frequency band signal retained after bandpass filtering. A bandpass filter bank is a collection of multiple bandpass filters with different center frequencies and non-overlapping passbands. Multiple subband signals refer to the frequency band signals output by the bandpass filter bank, with each subband signal corresponding to a specific frequency range.

[0197] Understandably, the system is designed with four second-order Butterworth bandpass filters centered on the commonly used SSVEP frequencies of 8, 10, 12, and 15 Hz, each with a passband width of 2 Hz, connected end to end to form a filter bank; then the bank is applied in parallel to the SSVEP signal to output four non-overlapping sub-band signals at once, each containing only the component of the corresponding center frequency ±1 Hz.

[0198] Sub-band filter set construction: the original EEG signal collected , wherein is the number of channels, is the length of time. In order to enhance the ability to distinguish different frequency responses, the system pre-constructs a band-pass filter set of different frequency bands , and divides the original signal into sub-bands:

[0199]

[0200] , wherein represents the filtered signal of the th sub-band.

[0201] Step S22, for each preset target frequency, a corresponding neural response waveform reference template is constructed according to the training data set.

[0202] It should be noted that the preset target frequency refers to a set of fixed flicker frequency values pre-set by the system for evoking SSVEP. The training data set refers to the multi-channel EEG data collected in the experimental stage, and the label corresponds to each preset target frequency. The neural response waveform reference template refers to the time sequence template representing the typical EEG response of each preset target frequency after averaging and spatial filtering on the training data set, which is used for subsequent correlation matching.

[0203] It can be understood that first, the system picks out all the trials of the same preset target frequency from the training data set according to the label, and extracts the occipital channel data of 0-1000 ms after the start of stimulation for each trial, removes the bad segments, and retains a number of signals, removes the direct current with sliding average, ensures waveform alignment and consistent baseline. Second, the system averages the retained signals to obtain a 1x1000 point mean waveform, and then performs task-related component analysis on the mean waveform to obtain the first three spatial filters that maximize the within-class correlation, and then averages the filtered signals to obtain a 1x1000 waveform as the reference template of the frequency. Finally, the system stores the four templates in the dictionary according to the frequency order and serializes them to the file, which is directly loaded in the subsequent online decoding to avoid repeated calculation and improve real-time performance.

[0204] Step S23, extracting the spatial filter of the preset target frequency through the task-related component analysis algorithm.

[0205] It should be noted that the task-related component analysis algorithm (TRCA) refers to, in the training stage, all the test data of the same preset target frequency, first calculating the within-class covariance matrix (interrelation between each test of the same frequency) and the total covariance matrix (total energy of all tests), then solving the generalized eigenvalue problem, obtaining a set of eigenvectors arranged in descending order of eigenvalue; these vectors are spatial filters. The dimension of the spatial filter is the same as the number of channels, and the inner product of the spatial filter and the multi-channel electroencephalogram signal can project the original high-dimensional data into a one-dimensional time sequence, while retaining the energy of the target frequency, maximizing the consistency between tests and suppressing non-target activities and noise.

[0206] As an example, the step of extracting the spatial filter of the preset target frequency by the task-related component analysis algorithm includes: obtaining a training data set corresponding to the preset target frequency, the training data set including electroencephalogram signals of multiple tests; calculating the within-class covariance matrix and the total covariance matrix according to the electroencephalogram signals; solving the within-class covariance matrix and the total covariance matrix by a generalized eigenvalue decomposition algorithm to obtain a spatial filter.

[0207] The electroencephalogram signals of multiple tests refer to several multi-channel SSVEP records collected by the system under the same preset target frequency, each corresponding to a visual stimulation test. The within-class covariance matrix refers to a matrix calculated from these test data, describing the channel intercorrelation structure between each test of the same frequency. The total covariance matrix refers to a matrix calculated from the combined data of all tests, describing the overall channel energy and correlation. The generalized eigenvalue decomposition algorithm refers to a numerical method for solving the generalized eigenvalue problem, used to obtain the optimal spatial filter vector from the within-class covariance matrix and the total covariance matrix.

[0208] Firstly, the system reads all the electroencephalogram signals of the same target frequency from the training library according to the label, extracts 1s after stimulation for each test, and arranges the channels in sequence into a three-dimensional tensor, which is then flattened into a matrix for standby. This ensures that all samples are time-aligned and have consistent channel order. Secondly, the system calculates the within-class covariance matrix for the flattened matrix: first, calculate the channel intercorrelation between each two tests and take the average to obtain the matrix; then calculate the total covariance matrix: concatenate all the tests into a whole matrix and calculate the channel covariance to obtain the matrix, which respectively reflect the same consistency and total energy, facilitating subsequent maximization of within-class similarity. Finally, the system solves the generalized eigenvalue equation, takes the eigenvector corresponding to the largest eigenvalue and normalizes it to obtain the spatial filter; this vector retains the most within-class energy and suppresses global noise, and can be directly used for online projection.

[0209] For each target frequency , a reference template And use the TRCA algorithm to extract its spatial filter. To enhance the stable signal characteristics between different trials at the target frequency, among which, For the number of channels, This refers to the number of time points. The specific steps are as follows:

[0210] (1) Construct the within-class covariance matrix Let the training set be the first... The data from this experiment are There are a total of This experiment. The within-class covariance matrix is ​​then defined as:

[0211]

[0212] in, It refers to the first EEG signal data from this experiment It refers to the first EEG signal data from this experiment This represents the transpose of a matrix.

[0213] (2) Construct the overall covariance matrix :

[0214]

[0215] (3) Optimize the spatial filter, the goal of which is to solve the spatial filter. This maximizes the within-class covariance and normalizes the overall energy.

[0216]

[0217] in, Indicates the optimal spatial filter; This represents the weight vector to be optimized. The formula means that among all possible weight vectors... In this problem, we seek a weight vector that maximizes the ratio of the objective function to constraints or penalty terms. The optimal solution to this optimization problem can be obtained using the generalized eigenvalue decomposition method, which aims to maximize the objective function while satisfying the constraints.

[0218] Step S24: Project the sub-band signal onto the spatial filter of the corresponding frequency to obtain the projected signal.

[0219] It should be noted that the projection signal refers to the signal that projects the first... The one-dimensional time series obtained by linearly projecting the sub-band signal corresponding to a preset frequency onto a dedicated spatial filter of that frequency retains only the frequency component and suppresses other frequency bands and noise.

[0220] Understandably, the system puts the first The sub-band signal corresponding to a preset target frequency (such as 10 Hz) is multiplied by a spatial filter dedicated to the frequency, and a one-dimensional time sequence, i.e., a projection signal, is obtained, in which the amplitude is greatly enhanced at the 10 Hz frequency, and other frequencies and background noise are significantly attenuated.

[0221] In step S25, the Pearson correlation coefficient of the projection signal and the neural response waveform reference template is calculated.

[0222] It should be noted that the Pearson correlation coefficient refers to a scalar that measures the degree of linear similarity between the projection signal and the neural response waveform reference template in the time domain, and the value range is to 1, and the closer the value is to 1, the more consistent the waveforms are.

[0223] It can be understood that after the system aligns the projection signal and the neural response waveform reference template by time points, the mean values of the two are calculated respectively, and then the sum of the products of the difference values of the corresponding points is divided by the product of the standard deviations to obtain the Pearson correlation coefficient.

[0224] In step S26, the Pearson correlation coefficients of all sub-bands are weighted and fused to obtain a correlation score vector.

[0225] It can be understood that the system weights and sums the Pearson correlation coefficients calculated for each sub-band according to a preset decreasing weight (for example, 1, 0.9, 0.8, 0.7) to generate a correlation score vector.

[0226] The measured electroencephalogram signal , wherein is the number of channels, is the number of time points, and first, the same sub-band filter set is used to obtain the sub-band component:

[0227]

[0228] , wherein is the measured electroencephalogram signal.

[0229] Then, it is projected to the sub-band, which is the target frequency trained spatial filter:

[0230]

[0231] Next, the Pearson correlation coefficient between the projected signal and the reference template corresponding to the frequency is calculated:​

[0232]

[0233] For the integration of information from all sub-bands, the final response score for each target frequency is obtained by weighted fusion of the correlation coefficients of all sub-bands:

[0234]

[0235] where is the weighting factor for the th sub-band, usually set as a decreasing weight according to bandwidth or experience; is an exponent to adjust the contribution of each term.

[0236] Final output: Correlation Score Vector.

[0237] So far, the application has calculated a final response score for each of the different target frequencies (i.e. categories). Unlike the traditional "winner-takes-all" strategy, the first decoding path of the application does not make a final decision at this stage.

[0238] Instead, it combines the response scores of all categories into a Correlation Score Vector and takes it as the final output of the decoding path:

[0239]

[0240] This score vector completely retains the association strength information of the test signal with each possible category. It will serve as the "evidence" of the SSVEP branch and be sent to the fusion decision mechanism unit, where it will be converted into a standard posterior probability distribution and finally fused with the output of the MI path. This design ensures the lossless transmission of information at the final decision stage and is the key to achieving high-precision fusion.

[0241] ​The embodiment first constructs a band-pass filter bank containing multiple sub-bands, and performs sub-band division on the SSVEP signal through the filter bank to obtain multiple sub-band signals, which helps to separate different frequency components and improve the accuracy of signal processing. Then, for each preset target frequency, a corresponding neural response waveform reference template is constructed according to the training data set to provide a standard for subsequent matching and enhance the recognition reliability. Then, the spatial filter of the preset target frequency is extracted through the task-related component analysis algorithm to enhance the target signal and suppress noise. Subsequently, the sub-band signals are projected to the spatial filter of the corresponding frequency to obtain the projection signals highlighting the characteristics of the target frequency. Then, the Pearson correlation coefficient of the projection signals and the neural response waveform reference template is calculated to quantify the similarity of the signals and the template. Finally, the Pearson correlation coefficients of all sub-bands are weighted and fused to obtain a correlation score vector that comprehensively reflects the matching degree of the signals, improving the accuracy and robustness of signal recognition, providing a basis for the generation of the target control strategy, and being able to balance the control accuracy of electroencephalogram decoding and the neural activation effect.

[0242] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the control strategy generation method for brain-controlled rehabilitation equipment based on the technical concept. More simple transformations in the form are within the protection scope of the present application.

[0243] The present application also provides a control strategy generation device for brain-controlled rehabilitation equipment, please refer to Figure 7 , the control strategy generation device for brain-controlled rehabilitation equipment comprises:

[0244] The preprocessing module 10 is configured to acquire a multi-channel electroencephalogram signal, and pre-process the multi-channel electroencephalogram signal to obtain a target electroencephalogram signal, wherein the target electroencephalogram signal corresponds to a preset action category, and the target electroencephalogram signal comprises a steady-state visual evoked potential signal and a motor imagery signal.

[0245] The first decoding module 20 is configured to decode the steady-state visual evoked potential signal through a first decoding path to obtain a correlation score vector, wherein the first decoding path adopts a filter group task-related component analysis.

[0246] The second decoding module 30 is configured to decode the motor imagery signal through a second decoding path to obtain a classification score vector, wherein the second decoding path adopts a Mamba dynamic routing spatio-temporal network model.

[0247] The weighted fusion module 40 is configured to perform posterior probability distribution conversion and weighted fusion on the correlation score vector and the classification score vector to obtain a fusion probability distribution.

[0248] The strategy generation module 50 is configured to generate a target control strategy according to an action category corresponding to a highest probability value in the fusion probability distribution.

[0249] The application provides a control strategy generation device for a brain-controlled rehabilitation device, which comprises at least one processor and a memory in communication connection with the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the control strategy generation method for the brain-controlled rehabilitation device in the first embodiment.

[0250] The application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to perform the control strategy generation method for the brain-controlled rehabilitation device in the above embodiments.

[0251] The control strategy generation device, the device and the storage medium provided by the application adopt the control strategy generation method for the brain-controlled rehabilitation device in the above embodiments, and can solve the technical problem of how to obtain a control strategy that takes into account the intention recognition accuracy and the rehabilitation neural activation effect. Compared with the prior art, the device, the equipment and the storage medium have the same beneficial effects as the control strategy generation method for the brain-controlled rehabilitation device provided in the above embodiments, and have the same disclosed features, which will not be described here.

[0252] The above only describes some embodiments of the application, and does not limit the patent scope of the application. Any equivalent structural transformation, direct / indirect application in other related technical fields based on the technical concept of the application, and the content of the specification and drawings are all included in the patent protection scope of the application.

Claims

1. A method for generating control strategies for brain-controlled rehabilitation devices, characterized in that, The method includes: Multi-channel EEG signals are acquired and preprocessed to obtain target EEG signals, which correspond to preset action categories. The target EEG signals include steady-state visual evoked potential signals and motor imagery signals. The steady-state visual evoked potential signal is decoded through the first decoding path to obtain a correlation score vector. The first decoding path uses task correlation component analysis of the filter group. The motion image signal is decoded through a second decoding path to obtain a classification score vector. The second decoding path adopts the Mamba dynamic routing spatiotemporal network model. The relevance score vector and the classification score vector are transformed by posterior probability distribution and weighted fusion to obtain a fused probability distribution. A target control strategy is generated based on the action category corresponding to the highest probability value in the fusion probability distribution. The Mamba dynamic routing spatiotemporal network model includes a convolutional neural network, a state-space model, a routing network, and a linear classifier. The step of decoding the motion image signal through the second decoding path to obtain the classification score vector includes: The shallow spatiotemporal features of the motion image signal are extracted using a convolutional neural network; The shallow spatiotemporal features are modeled using a state-space model to perform long-range temporal dependency modeling, resulting in a deep feature sequence. The importance score of each time point in the deep feature sequence is calculated using a routing network; Dynamic attention weights are generated based on the importance scores; The dynamic attention weights are applied to the deep feature sequence for weighted pooling to generate a context feature vector; The context feature vector is processed by a linear classifier to obtain a classification score vector; The construction steps of the Mamba dynamic routing spatiotemporal network model include: A shallow spatiotemporal feature extraction module is constructed based on the first convolutional block and the second convolutional block. The kernel size of the first convolutional block is larger than that of the second convolutional block, and the number of output channels of the first convolutional block is smaller than that of the second convolutional block. A deep temporal context coding module is constructed based on linear layers, state-space models, and gating functions; Build a dynamic routing pooling module based on the routing network; Construct a linear classifier based on the number of preset action categories; The Mamba dynamic routing spatiotemporal network model is constructed based on the shallow spatiotemporal feature extraction module, the deep temporal context encoding module, the dynamic routing pooling module, and the linear classifier.

2. The method as described in claim 1, characterized in that, The first decoding path employs task-related component analysis of the filter group; The step of decoding the steady-state visual evoked potential signal through the first decoding path to obtain the correlation score vector includes: A bandpass filter bank containing multiple sub-bands is constructed, and the steady-state visual evoked potential signal is divided into sub-bands by the bandpass filter bank to obtain multiple sub-band signals; For each preset target frequency, a corresponding neural response waveform reference template is constructed based on the training dataset; The spatial filter for the preset target frequency is extracted using a task-related component analysis algorithm; The sub-band signal is projected onto the spatial filter at the corresponding frequency to obtain the projected signal; Calculate the Pearson correlation coefficient between the projected signal and the neural response waveform reference template; The Pearson correlation coefficients of all subbands are weighted and fused to obtain a correlation score vector.

3. The method as described in claim 2, characterized in that, The step of extracting the spatial filter of the preset target frequency using the task-related component analysis algorithm includes: Obtain the training dataset corresponding to the preset target frequency, wherein the training dataset includes EEG signals from multiple trials; Based on the EEG signals, calculate the within-class covariance matrix and the global covariance matrix; The spatial filter is obtained by solving the intra-class covariance matrix and the global covariance matrix using the generalized eigenvalue decomposition algorithm.

4. The method as described in claim 1, characterized in that, The step of performing posterior probability distribution transformation and weighted fusion on the relevance score vector and the classification score vector to obtain the fused probability distribution includes: By performing a posterior probability distribution transformation on the relevance score vector and the classification score vector using a normalization function, the relevance probability distribution vector and the classification probability distribution vector are obtained. The correlation probability distribution vector and the classification probability distribution vector are weighted and fused according to the preset steady-state visual evoked potential weight coefficient and the preset motion imagination weight coefficient to obtain the fused probability distribution.

5. The method according to any one of claims 1 to 4, characterized in that, The target EEG signals include steady-state visual evoked potential signals and motor imagery signals; The step of preprocessing the multi-channel EEG signals to obtain the target EEG signal includes: The multi-channel EEG signal is subjected to band-pass filter bank for frequency band separation filtering to obtain a band-separated signal; Based on the functional localization of brain regions, the frequency band separation signal is divided into channel subsets to obtain steady-state visual pathway signals and motor imagery pathway signals; The motion imagery pathway signal is identified and removed by independent component analysis or template matching-based electrooculography artifact detection algorithm to obtain a pure motion imagery signal. The steady-state visual pathway signal and the pure motion imagery signal are subjected to amplitude normalization processing to obtain the steady-state visual evoked potential signal and the motion imagery signal.

6. A control strategy generation device for brain-controlled rehabilitation equipment, characterized in that, The device includes: A preprocessing module is used to acquire multi-channel EEG signals and preprocess the multi-channel EEG signals to obtain target EEG signals. The target EEG signals correspond to preset action categories and include steady-state visual evoked potential signals and motor imagery signals. The first decoding module is used to decode the steady-state visual evoked potential signal through a first decoding path to obtain a correlation score vector. The first decoding path adopts task correlation component analysis of the filter group. The second decoding module is used to decode the motion imagery signal through a second decoding path to obtain a classification score vector. The second decoding path employs a Mamba dynamic routing spatiotemporal network model. The Mamba dynamic routing spatiotemporal network model includes a convolutional neural network, a state-space model, a routing network, and a linear classifier. The step of decoding the motion imagery signal through the second decoding path to obtain a classification score vector includes: extracting shallow spatiotemporal features of the motion imagery signal through a convolutional neural network; performing long-range temporal dependency modeling on the shallow spatiotemporal features using a state-space model to obtain a deep feature sequence; calculating the importance score of each time point in the deep feature sequence through a routing network; generating dynamic attention weights based on the importance scores; and applying the dynamic attention weights to the deep feature sequence. The process involves: weighted pooling to generate a context feature vector; processing the context feature vector using a linear classifier to obtain a classification score vector; and constructing the Mamba dynamic routing spatiotemporal network model, which includes: constructing a shallow spatiotemporal feature extraction module based on a first convolutional block and a second convolutional block, wherein the kernel size of the first convolutional block is larger than that of the second convolutional block, and the number of output channels of the first convolutional block is smaller than that of the second convolutional block; constructing a deep temporal context encoding module based on a linear layer, a state-space model, and a gating function; constructing a dynamic routing pooling module based on the routing network; constructing a linear classifier based on the number of preset action categories; and constructing the Mamba dynamic routing spatiotemporal network model based on the shallow spatiotemporal feature extraction module, the deep temporal context encoding module, the dynamic routing pooling module, and the linear classifier. The weighted fusion module is used to perform posterior probability distribution transformation and weighted fusion on the relevance score vector and the classification score vector to obtain a fused probability distribution. The strategy generation module is used to generate a target control strategy based on the action category corresponding to the highest probability value in the fusion probability distribution.

7. A control strategy generation device for brain-controlled rehabilitation equipment, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the control strategy generation method for a brain-controlled rehabilitation device as described in any one of claims 1 to 5.

8. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the control strategy generation method for brain-controlled rehabilitation devices as described in any one of claims 1 to 5.

Images