Decoding identification method for motion imagery electroencephalogram signals

Abstract

The invention discloses a decoding identification method for a motion imagination electroencephalogram signal, and relates to the technical field of motion imagination decoding. Comprising the following steps: acquiring a motion imagination electroencephalogram signal of a to-be-detected user; the motion imagination electroencephalogram signals are input into a pre-trained multi-branch differential attention time network, and the pre-trained multi-branch differential attention time network comprises a multi-branch feature fusion module, a sliding window layer, a multi-head differential attention mechanism and an adaptive residual separable time convolution network which are connected in sequence; and obtaining a motion intention classification result of the to-be-detected user. According to the method, data features can be enhanced more stably and accurately, and the application reliability of a BCI system is remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of motor imagery decoding technology, and in particular to a method for decoding and recognizing motor imagery EEG signals. Background Technology

[0002] Brain-computer interfaces (BCIs), a cutting-edge technology in neuroengineering and rehabilitation engineering, aim to establish a direct communication pathway between the brain and external devices for patients with severe motor dysfunction (such as amyotrophic lateral sclerosis, spinal cord injury, etc.), thereby restoring their ability to communicate and control their bodies. BCI technology shows great promise in biomedical engineering fields such as medical rehabilitation, functional compensation and enhancement, and is an important technological means to improve patients' quality of life and promote the development of rehabilitation medicine.

[0003] Among numerous non-invasive neurophysiological signal acquisition technologies, electroencephalography (EEG) has become the most widely used signal source in BCI systems due to its advantages such as portable equipment, low cost, non-invasive safety, and extremely high temporal resolution. Motor imagery (MI) is an endogenous BCI paradigm that does not rely on external sensory stimuli and is voluntarily controlled by the user. It achieves intention recognition by decoding specific electrical activity in the sensorimotor cortex of the brain when the user imagines limb movements. This characteristic of MI-BCI gives it unique advantages in rehabilitation training and assistive device control. However, the inherent low signal-to-noise ratio, non-stationarity, and significant individual variability of EEG signals pose a significant challenge to high-precision and robust motor intention decoding, which is also the core technical bottleneck restricting the clinical application of BCI systems.

[0004] For decoding MI-EEG signals, research methods have evolved from traditional biosignal processing methods that rely on manual feature extraction based on prior knowledge, such as cospatial patterns, to end-to-end intelligent decoding models represented by deep learning. Deep learning models, such as convolutional neural networks (CNNs), can automatically learn deep spatiotemporal features in EEG signals, surpassing traditional methods in decoding accuracy. Typical examples include DeepConvNet and EEGNet.

[0005] However, to achieve reliable application of BCI systems in complex clinical and daily environments, current deep learning models still face several key limitations in their design. Among them, ATCNet, as the most representative hybrid architecture model in recent years, has made significant progress in motion imagery decoding tasks, but still has two core problems that urgently need to be solved.

[0006] First, while ATCNet innovatively combines multi-head self-attention mechanisms with temporal convolutional networks, it has inherent limitations in extracting spatiotemporal features from EEG signals. The first stage of the model utilizes self-attention to highlight key features in motor imagery data, followed by TCN extraction of higher-order temporal features. However, this design fails to adequately address the inherent low signal-to-noise ratio (SNR) problem of EEG signals. In practical applications, general self-attention mechanisms are highly susceptible to artifacts from electromyography (EMG), electrooculography (EOG), and environmental noise, leading to misallocation of "attention" and an inability to stably focus on task-related neural activity patterns. This sensitivity to noise severely restricts the model's robustness in complex environments.

[0007] Secondly, ATCNet's TCN component suffers from structural bottlenecks in long-term dynamic modeling. While TCN theoretically expands the receptive field through layer stacking, in practice, ATCNet's receptive field growth strictly depends on a linear increase in network depth. Capturing ultra-long-range dependencies spanning the entire motor imagery process (typically lasting several seconds) requires constructing extremely deep network structures. This significantly increases computational complexity and memory consumption, hindering deployment on resource-constrained devices, and makes gradient propagation difficult, making it hard for the network to effectively learn long-range associations between key cognitive intentions. Furthermore, deep stacking exacerbates the risk of overfitting, especially given the limited availability of EEG data samples.

[0008] The two aforementioned technical shortcomings overlap: the multi-head self-attention mechanism provides incorrect initial features under environmental interference, while the TCN component cannot robustly extract key long-range patterns spanning several seconds from this noisy data. This inherent unreliability, combined with insufficient explicit modeling capabilities, leads to unstable model performance in real-world complex scenarios, severely limiting the reliability of BCI systems in practical applications. Summary of the Invention

[0009] Therefore, it is necessary to provide a method for decoding and recognizing motor imagery EEG signals to address the aforementioned technical problems.

[0010] This invention provides a method for decoding and recognizing motor imagery EEG signals, comprising: Acquire the EEG signals of the user's motor imagery; The motion-imagining EEG signals are input into a pre-trained multi-branch differential attention temporal network, which includes: a multi-branch feature fusion module, a sliding window layer, a multi-head differential attention mechanism, and an adaptive residual separable temporal convolutional network connected in sequence. The spatiotemporal features of motion-imagination EEG signals are extracted in parallel by using convolution operations with convolution kernels of different sizes through a multi-branch feature fusion module, in order to capture the diverse neural activity patterns in motion-imagination EEG signals and obtain a comprehensive spatiotemporal feature map. By using a sliding window layer, the comprehensive spatiotemporal feature map is divided into multiple consecutive short sequence window features in the time dimension. Each short sequence window feature represents a neural activity pattern of motor imagination EEG signal. For each short sequence window feature, a linear transformation projection is performed on the short sequence window feature through a multi-head differential attention mechanism module to obtain two sets of independent attention maps; the differential attention weight is determined by the difference in attention scores between the two sets of independent attention maps, and the short sequence window feature is weighted by the differential attention weight to obtain the enhanced window feature corresponding to the short sequence window feature; all enhanced window features are concatenated according to the division order to obtain the motion imagination time series; By performing multiple one-dimensional separable convolution operations on the motion-imagined time series through an adaptive residual separable temporal convolutional network, long-distance temporal dependencies in the motion-imagined time series are captured, and the final temporal features are obtained. The softmax function is used to predict the classification probability of the final temporal features to obtain the classification result of the user's motion intention.

[0011] Optionally, the multi-branch feature fusion module includes: a temporal convolutional layer and multiple parallel feature extraction branches, wherein the feature extraction branches are composed of depthwise separable convolutional layers and standard or separable convolutional layers of different kernel sizes connected in series.

[0012] Optionally, the spatiotemporal features of the motion-imagination EEG signal are extracted in parallel using convolution operations with convolution kernels of different sizes through a multi-branch feature fusion module, in order to capture the diverse neural activity patterns in the motion-imagination EEG signal and obtain a comprehensive spatiotemporal feature map, specifically including... Based on the following formula, local temporal patterns of motion-imagination EEG signals are extracted through temporal convolutional layers to obtain an initial feature map: ; in, Imagine brain signals for movement. For learnable weights of temporal convolutional layers, For convolution operations, For batch normalization processing, This is the initial feature map; For each feature extraction branch, spatial feature filtering is performed on the initial feature map using a depthwise separable convolutional layer based on the following formula: ; in, For depthwise separable convolution, For the weights of depthwise separable convolutions, It is the activation function of the exponential linear unit. This is the initial feature map after spatial feature filtering; The initial feature map after spatial feature filtering is extracted by standard convolutional layers and separable convolutional layers with different kernel sizes in this feature extraction branch to extract the dynamic changes of features at different time scales, and the output feature map is obtained. All output feature maps are fused using an element-wise addition method to obtain a comprehensive spatiotemporal feature map.

[0013] Optionally, the output feature map is obtained by dynamically analyzing the initial feature map after spatial feature filtering through standard convolutional layers and separable convolutional layers with different kernel sizes of the feature extraction branch at different time scales, based on the following formula: ; in, This is the initial feature map after spatial feature filtering. For the first Convolution operations for each feature extraction branch For the first The convolution kernel weights corresponding to the convolution operations of each feature extraction branch. For average pooling operation, It is the activation function of the exponential linear unit. For the first Output feature maps of each feature extraction branch; Based on the following formula, all output feature maps are fused using an element-wise addition method to obtain a comprehensive spatiotemporal feature map: ; in, The output feature maps for each feature extraction branch, To initially integrate spatiotemporal feature maps, To synthesize the spatiotemporal feature map, This is a random deactivation operation.

[0014] Optionally, the integrated spatiotemporal feature map can be divided into multiple consecutive short-sequence window features in the time dimension using a sliding window based on the following formula: ; in, For the first A short sequence window feature, while For the first The starting index of a short sequence window feature in the time dimension. For the first The end index of a short sequence window feature in the time dimension This is a comprehensive spatiotemporal feature map.

[0015] Optionally, the short sequence window features can be linearly transformed and projected using a multi-head differential attention mechanism module based on the following formula: ; ; ; in, Features of short sequence windows For query, As key, For value, , and It is a learnable projection matrix; The query and key are evenly divided into two independent attention maps. The differential attention weight is determined based on the difference in attention scores between the two independent attention maps using the following formula: ; in, For the dimension of attention head, For learnable scalars, The attention score for the first set of independent attention maps. The attention score for the second set of independent attention maps; The enhanced window features corresponding to the short sequence window features are obtained by weighting the short sequence window features using the following formula through differential attention weights: ; in, To output the projection matrix; Based on the following formula, all enhanced window features are concatenated according to the division order to obtain the motion imagination time series: ; in, The total number of windows. Imagine a time series for motion.

[0016] Optionally, the adaptive residual separable temporal convolutional network includes: multiple residual blocks connected in sequence; each residual block includes: a one-dimensional separable convolutional layer, a batch normalization layer, an activation function layer and a random dropout layer connected in sequence, and the one-dimensional separable convolutional layer of each residual block is associated with the one-dimensional separable convolutional layers of other residual blocks through residual connections.

[0017] Optionally, multiple one-dimensional separable convolution operations are performed on the motion imagery time series using an adaptive residual separable temporal convolutional network to capture long-range temporal dependencies in the motion imagery time series, obtaining the final temporal features, specifically including: For each residual block, the input features are subjected to depthwise convolution and pointwise convolution through a one-dimensional separable convolutional layer to obtain the convolutional output features. The convolutional output features are normalized by batch normalization layers to obtain normalized features; the normalized features are then nonlinearly transformed by activation function layers to obtain activated features. The activation features are regularized by a random dropout layer to obtain the output features of the residual block; the output features of each residual block are added to the corresponding input features through residual connections to obtain the residual fusion features. The residual fusion feature output from the last residual block is used as the final temporal feature of the motion imagination time series.

[0018] The decoding and recognition method for motor imagery EEG signals provided in this embodiment of the invention has the following advantages compared with the prior art: This invention constructs a progressive technical path of "multi-branch feature fusion - differential enhancement - adaptive residual separable temporal convolution". In the differential enhancement stage, the sliding window divides the long sequence into continuous short sequence window features, which can limit the interference range of irrelevant context. The differential attention mechanism weights the short sequence window features by calculating the score difference between two sets of independent attention maps, amplifies the attention to key neural activity patterns, effectively eliminates common mode noise, and can achieve more stable and accurate enhancement of data features, significantly improving the application reliability of the BCI system.

[0019] In the multi-branch feature fusion stage, four parallel but structurally different branches are used to adaptively extract diverse spatiotemporal features from EEG signals at different scales. This effectively solves the problem of incomplete feature extraction in traditional single-scale models while maintaining low model complexity through an ingenious structure.

[0020] In the adaptive residual separable temporal convolution stage, multi-scale dynamics are captured in parallel through adaptive multi-branch dilated separable convolutions, and long-range residual connections are used to repeatedly inject the original input features into the deep network. This structure efficiently integrates multi-scale deep features with key initial contextual information, thereby achieving accurate classification of time series. Attached Figure Description

[0021] Figure 1 The MDAT-Net network structure diagram is shown in one embodiment of a method for decoding and recognizing motor imagery EEG signals. Figure 2A multi-head differential attention module diagram of a method for decoding and recognizing motor imagery EEG signals provided in one embodiment; Figure 3 An adaptive residual separable temporal convolutional network diagram for a method of decoding and recognizing motion-imagined EEG signals provided in one embodiment; Figure 4 This is a performance comparison chart of MDAT-Net and benchmark methods for decoding and recognizing motion-imagination EEG signals provided in one embodiment on the HGD dataset. Figure 5 This is a schematic diagram illustrating the accuracy of MDAT-Net on the HGD dataset for various subjects using a decoding and recognition method for motor imagery EEG signals provided in one embodiment. Figure 6 The confusion matrix of MDAT-Net for all subjects on the BCI-IV-2a dataset, which is a decoding and recognition method for motor imagery EEG signals provided in one embodiment. Figure 7 A visualization of the two-dimensional feature space of MDAT-Net on the BCI-IV-2a dataset, which is a decoding and recognition method for motor imagination EEG signals provided in one embodiment. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0023] For high-precision decoding of motor imagery EEG signals, this invention proposes a decoding and recognition method for motor imagery EEG signals, aiming to improve the decoding performance and robustness of the MI-BCI system. The main contributions are as follows:

[0024] ① A multi-branch feature fusion module was designed, which adaptively extracts diverse spatiotemporal features from EEG signals at different scales through four parallel but structurally distinct branches. This design effectively solves the problem of incomplete feature extraction in traditional single-scale models while maintaining low model complexity through its ingenious structure.

[0025] ② To ensure the stability of the attention mechanism when focusing on long-distance sequences and to address the problem of traditional models being easily interfered with by irrelevant context, a module combining a sliding window and a differential attention mechanism was designed. This module efficiently processes sequences through a sliding window and innovatively utilizes a differential method to eliminate common-mode noise in attention calculations, thereby amplifying the effective focus on key information and achieving more stable and accurate enhancement of data features.

[0026] ③ To efficiently capture long-range temporal dependencies, an adaptive residual separable temporal convolutional network was designed. This network captures multi-scale dynamics in parallel through adaptive multi-branch dilated separable convolutions and repeatedly injects the original input features into the deep network using long-range residual connections. This structure efficiently integrates multi-scale deep features with key initial contextual information, thereby achieving accurate classification of time series.

[0027] ④ The classification results of the proposed algorithm were systematically compared with other state-of-the-art classification algorithms on three publicly available motion imagery datasets. The experimental results fully demonstrate that the proposed method has superior effectiveness and generalization ability on the MI classification task.

[0028] This invention provides a method for decoding and recognizing motor imagery EEG signals, the method comprising: Acquire the motor imagery EEG signals of the user to be tested. Input the motor imagery EEG signals into a pre-trained multi-branch differential attention temporal network. The pre-trained multi-branch differential attention temporal network includes: a multi-branch feature fusion module, a sliding window layer, a multi-head differential attention mechanism, and an adaptive residual separable temporal convolutional network connected in sequence.

[0029] By using a multi-branch feature fusion module, convolution operations with convolution kernels of different sizes are employed to extract the spatiotemporal features of motor imagery EEG signals in parallel, in order to capture the diverse neural activity patterns in motor imagery EEG signals and obtain a comprehensive spatiotemporal feature map.

[0030] By using a sliding window layer, the comprehensive spatiotemporal feature map is divided into multiple consecutive short sequence window features in the time dimension. Each short sequence window feature represents a neural activity pattern of motor imagination EEG signal. For each short sequence window feature, a multi-head differential attention mechanism is used to perform a linear transformation projection on the feature, resulting in two independent attention maps. The differential attention weight is determined by the difference in attention scores between the two independent attention maps, and this weight is then used to weight the short sequence window feature, yielding the enhanced window feature corresponding to that short sequence window feature. All enhanced window features are then concatenated according to the partitioning order to obtain the motion imagery time series.

[0031] By performing multiple one-dimensional separable convolution operations on the motion-imagined time series using an adaptive residual separable temporal convolutional network, the long-distance temporal dependencies of the motion-imagined time series are captured, resulting in the final temporal features.

[0032] The softmax function is used to predict the classification probability of the final temporal features to obtain the classification result of the user's motion intention.

[0033] A specific embodiment of the present invention is provided: 1. Implementation content.

[0034] To achieve high-precision motion image decoding, the proposed multi-branch differential attention temporal network (MDAT-Net) constructs an end-to-end serial architecture, as shown below. Figure 1 As shown, the existing challenges are addressed in a progressively deeper manner through three core modules.

[0035] First, to overcome the limitation of fixed receptive fields in single-scale convolution, the model employs a multi-branch feature fusion module (MFM) to extract spatiotemporal features in parallel. Different branches utilize convolutional kernels of different sizes to capture diverse neural activity patterns, ultimately fusing them into a more informative and robust comprehensive feature set.

[0036] Next, the integrated features are fed into the sliding window (SW) and multi-head differential attention (MDA) modules. This module innovatively uses the temporal differences of features to guide attention allocation, which can not only effectively suppress common-mode noise in EEG signals, but also enable the model to stably focus on the dynamic changes of signals most relevant to the task, thus solving the problem of easy attention distraction.

[0037] Finally, the attention-enhanced feature sequences are processed by an adaptive residual separable temporal convolutional network (ARSTCN) to efficiently capture long-range temporal dependencies and output the final classification result. The entire process is interconnected and aims to achieve accurate recognition of the intention to imagine motion.

[0038] 1.1 Multi-branch feature fusion module.

[0039] To effectively address the complexity of neural electrical activity spanning multiple frequency bands and brain regions in motor imagery tasks, a Multi-Scale Motion Model (MFM) was designed to adaptively capture and integrate multi-scale spatiotemporal representations from raw EEG signals. The design of this module strictly adheres to the principles of enhancing model robustness and reproducibility. Firstly, a shared temporal convolutional layer (with a kernel size of [missing information]) is used... The input signal undergoes preliminary feature extraction to capture basic local temporal patterns. This process can be summarized as follows:

[0040] (1) in, Represents the raw EEG signal data input. These are the learnable weights of the temporal convolutional layer. This represents the convolution operation, while This represents batch normalization. To prevent overfitting, all convolutional layers employ L2 weight regularization (with a decay coefficient). ) and maximum norm constraint ( This reflects the consideration of generalization ability in model design.

[0041] After initial temporal feature extraction, to learn the co-activation patterns between different EEG channels, the module feeds the feature maps into four depthwise separable convolutional layers (depth multiplier of 2) for spatial feature filtering. This operation can be represented as:

[0042] (2) in, This represents a depthwise separable convolution with weights of . It is specifically designed for spatial filtering of the channel dimension of feature maps. The activation function represents the exponential linear unit (ELU), chosen to alleviate the vanishing gradient problem and introduce nonlinear expressive power. This step is designed to efficiently separate the spatial distribution characteristics of the signal, laying the foundation for subsequent multi-scale analysis.

[0043] Next is the core design of this module—four parallel feature extraction branches. Their purpose is to capture the dynamic changes of features at different time scales, which also provides a reasonable explanation for the method selection. These four branches receive the same spatially filtered feature map, but use convolutional kernels of different sizes (respectively...). , , and The first branch uses a standard two-dimensional convolutional layer (kernel size...). Its larger receptive field is designed to capture macroscopic feature patterns over a longer time span. The other three branches employ computationally more efficient separable convolutional layers (by decomposing standard convolution into depthwise convolutions and pointwise convolutions), configured with layers of different sizes. , and The convolutional kernels are optimized to capture feature details at medium and fine time scales. The feature extraction process for each branch can be summarized as follows:

[0044] (3) in, It is after the size is Spatial feature maps after random deactivation processing with an average pooling and a deactivation rate of 0.25. Representing the The convolution operations used in each branch (standard or separable). It is the corresponding convolutional kernel weight, and This represents the average pooling operation, used to reduce feature dimensionality and enhance feature translation invariance. Through this multi-scale design, the model can simultaneously focus on neural rhythm changes at different durations during motor imagery.

[0045] To effectively integrate heterogeneous features extracted from different scales, the module ultimately fuses the output feature maps from the four branches. An element-wise addition approach is used instead of simple concatenation. This is because it encourages different branches to learn complementary features without significantly increasing the number of parameters, thereby enhancing the robustness and information richness of the final feature representation. The fusion process is as follows:

[0046] (4) in, These are the output feature maps of the four parallel branches. Finally, a random deactivation layer with a deactivation rate of 0.3 is applied to the fused features to further improve the model's generalization ability.

[0047] (5) in, This module outputs a feature map that is then passed to subsequent network layers for in-depth processing. Through this carefully designed multi-branch architecture, and by providing detailed parameters including regularization to ensure reproducibility, this module effectively extracts a comprehensive set of multi-scale spatiotemporal features from the raw EEG signal, providing a solid foundation for achieving high-precision motion imagery decoding.

[0048] 1.2 Sliding window and differential attention module.

[0049] Following multi-scale feature fusion, to further enhance focus on key time segments while suppressing inherent noise interference in EEG signals, the model introduces SW and MDA modules. Traditional attention mechanisms are prone to performance instability when processing long sequences due to low signal-to-noise ratios. This module, by combining efficient sequence processing methods with advanced MDA mechanisms, aims to amplify the effective information in the signal and eliminate interference from irrelevant context.

[0050] First, the module uses the SW strategy to process the fused feature map. Perform segmentation processing, where It refers to the batch size. It is the number of feature channels. This refers to the number of time points. By setting a fixed number of windows (5), the long sequence is divided into multiple overlapping, more manageable shorter sequences. This method not only improves computational efficiency but also provides a localized context for the stable operation of the attention mechanism. The process of generating a window can be represented as follows:

[0051] (6) in, Representing the The feature sequence of each window, and and These are the start and end indices of the window in the time dimension.

[0052] Subsequently, each feature window It is sent to the core MDA for processing. The structure of this mechanism is as follows: Figure 2 As shown, its design is inspired by differential amplifiers. Its core idea is to eliminate common-mode noise by calculating the difference between two independent attention maps, thereby highlighting the signal components most relevant to the task. Specifically, the input feature window... First, the query is obtained through linear transformation projection. ), key ) and value ):

[0053] (7) (8) (9) in, , and It is a learnable projection matrix. Subsequently, and The queries were evenly divided into two independent groups. ) and key ( ),and It is then used as a whole for subsequent weighted summation.

[0054] Next, by calculating two sets Q - K Attention scores are calculated and subtracted to obtain the final differential attention weights. To enhance generalization ability, Dropout is applied to the differential weights. The weighted calculation process for the single-head differential attention weights is as follows:

[0055] (10) in, It is the dimension of attention head. It is a learnable scalar used to dynamically balance the contributions of two attention maps, with its initial value... It is set to 0.8. This subtraction operation can effectively cancel out the noise patterns that coexist in the two attention maps, thus making the model's focus sparser and more concentrated.

[0056] To capture information from different representation subspaces, this mechanism is extended to a multi-head form. Each head performs the aforementioned differential attention computation independently, and the outputs of all heads are concatenated and then subjected to group normalization. This normalization strategy effectively addresses the problem of increased differences in statistical features among heads caused by attention sparsity. To further balance the differential operations, the normalized tensor is multiplied by a scaling factor. Finally, the scaled result is passed through a final linear projection to generate the module's output. Let... Given the output after concatenating all headers, the output of MDA can be expressed as:

[0057] (11) in, This is the output projection matrix. After processing all the projected features (SWs), the outputs of each window are concatenated to restore the enhanced feature sequence of the same length as the input.

[0058] (12) in, It is the total number of windows. This is the final output feature sequence, which has undergone noise suppression and key information enhancement. Through the ingenious combination of SW and MDA, this module can stably and accurately enhance data features, providing high-quality input for subsequent time-dependent modeling.

[0059] 1.3 Adaptive residual separable temporal convolutional network.

[0060] After the model undergoes precise feature enhancement via a sliding window differential attention module, the final sequence is fed into ARSTCN for deep dynamic temporal modeling and final classification. Its structure is as follows: Figure 3 As shown. The core objective of this network module is to efficiently capture long-range dependencies throughout the entire motion imagination time series, thereby overcoming the limitations of traditional models in terms of receptive field and achieving deep and refined decoding of temporal information.

[0061] To build a deep and effectively trainable network, ARSTCN consists of multiple carefully designed residual blocks stacked together. The core of each residual block is a one-dimensional separable convolution, which decomposes standard convolution into depthwise convolution and pointwise convolution, while maintaining powerful feature extraction performance.

[0062] In a single convolutional unit, the input features first pass through a one-dimensional separable convolutional layer to obtain one-dimensional dilated separable convolutional features (multi-scale temporal patterns extracted through adaptive dilated convolution). Subsequently, batch normalization is applied to accelerate convergence and stabilize the training process, resulting in batch-normalized features (standardized features with zero mean and unit variance). The ReLU activation function is then used to introduce necessary nonlinearity, enhancing the model's ability to fit complex temporal patterns, resulting in nonlinear activation features. Finally, Dropout is applied to improve the model's generalization ability, yielding regularized output features. H .

[0063] This process can be summarized by the following formula: (13) in, This represents the input feature sequence of the convolutional unit. W The weight parameters are for the separable convolution kernel. D It is the expansion coefficient that controls the size of the receptive field.

[0064] To ensure effective gradient propagation in deep networks, the model employs residual connections across the board. For the first residual block of the network, its output is the sum of the results processed by the two internal convolutional units and the initial input. Specifically, before performing the addition operation, the model checks whether the number of channels in the input and output are consistent. If they are inconsistent, a connection is established. Convolution transforms the original input The input is projected to the same channel dimension as the output; otherwise, if the number of channels matches, it is directly concatenated. This adapted input is defined as the "residual source". It is used not only for the first residual block, but also as the reference source for all subsequent deep residual connections. The output of the first residual block can be expressed as:

[0065] (14) in, This represents the output of the convolution stack within the first residual block. This is the raw input after channel adaptation (and the application of specific Dropout if necessary).

[0066] To further enhance the model's temporal modeling capabilities, ARSTCN's deep blocks introduce an adaptive dilated convolution mechanism and employ a unique dense residual structure. Specifically, for the deepest blocks of the network... Each block, whose convolutional layers will grow from an exponentially increasing set of inflation coefficients ( The optimal dilation rate is dynamically selected within the convolutional layers. This design allows the network to flexibly adjust its receptive field to capture dependencies at different time scales. Furthermore, these deep blocks employ a more complex residual connection mechanism: after each convolutional layer within a block, its output is connected to the aforementioned "residual source" originating from the network's initial state. The inputs are added together. This fast connection from start to finish greatly facilitates the backpropagation of gradients and ensures that the original input information can continuously influence the deep feature learning of the network, effectively avoiding performance degradation.

[0067] After passing through a depth of depth After processing by the ARSTCN network, the model outputs a feature sequence. Since the features at the top layer of the network have already incorporated the historical context information of the entire sequence through a large receptive field, the model only extracts the output feature vector of the last time point of the sequence. Used for subsequent classification decisions:

[0068] (15) in, This is the final high-dimensional feature representation that contains the dynamics of the entire sequence over time. Finally, since the model front-end uses a sliding window mechanism, it is necessary to effectively fuse the classification information generated by all windows. This model adopts an average fusion strategy, that is, the feature vector extracted from each window... Each feature is passed through a fully connected layer, and the arithmetic mean of all results is taken. This fused feature is then passed through a softmax activation function to obtain the motion intent classification result of the user under test (i.e., the final predicted probability of each motion imagination category).

[0069] 2. Experiment.

[0070] 2.1 Dataset.

[0071] To evaluate the performance of the proposed method, experiments were conducted on multiple datasets, including two publicly available benchmark datasets (BCI-IV-2a and BCI-IV-2b) provided by BCI CompetitionIV and a larger dataset (HGD) collected in-house.

[0072] BCI-IV-2a: This dataset contains EEG data from 9 subjects. Each subject performed four MI tasks: left hand, right hand, both feet, and tongue. Data was collected from two independent sessions, one for classifier training and the other for testing. Each session consisted of 6 runs, with each run containing 48 trials. Signals were recorded at a sampling rate of 250 Hz, covering 22 EEG channels and 3 EEG channels (25 channels in total), and bandpass filtered from 0.5 to 100 Hz.

[0073] BCI-IV-2b: This dataset also contains EEG data from 9 subjects, covering two types of MI tasks: left-handed and right-handed. For each subject, a total of 5 sessions were recorded: the first two were training sessions without feedback, and the last three were sessions with online smile feedback. The training sessions without feedback consisted of 6 runs, each with 20 trials (10 trials per class), for a total of 120 trials per session. The online feedback sessions consisted of 4 runs, each with 40 trials (20 trials per class). Data were recorded through 3 bipolar EEG channels (C3, Cz, C4) and 3 unipolar EOG channels (6 channels in total), at a sampling rate of 250Hz, with bandpass filtering from 0.5-100Hz and notch filtering applied at 50Hz.

[0074] High Gamma Dataset (HGD): This dataset contains data from 14 healthy subjects. Each subject performed four motor imagery tasks: imagining movement of the left hand, right hand, both feet, and remaining still. The training set contains approximately 880 trials, while the test set contains 160 trials. Signals were initially recorded using 128 electrodes, later reduced to 44 electrodes covering the motor cortex, with the sampling rate adjusted to 250 Hz.

[0075] 2.2 Data Preprocessing.

[0076] During the data preprocessing stage, all datasets underwent a series of standardized procedures to adapt them to the model input. For the large HGD dataset, specific preprocessing was performed first. This process included artifact removal, where any trials with an absolute amplitude exceeding 800mV within a time window of 0 to 4000ms were identified and removed from the dataset. Following this, 44 specific channels covering the motor cortex were selected from the electrodes of the original recordings to focus on signals from brain regions most relevant to the task.

[0077] After completing these initial steps for specific datasets, all datasets (including HGD, BCI-IV-2a, and BCI-IV-2b) were processed using a unified workflow. Signals were resampled to 250Hz, and a high-pass filter was applied as needed to remove low-frequency components below 4Hz. To improve model convergence, the data was normalized using exponential shift normalization. Finally, a time window from 500ms before the motion visualization cue to 4000ms after the cue was extracted from each trial and used as input data.

[0078] In the final preparation phase, to ensure the reproducibility of the experiment, a fixed random seed was used to shuffle the training and test sets. The processed data was reshaped into a four-dimensional tensor format adapted to the model input, i.e., (number of trials, 1, number of channels, number of time points), and the class labels were also converted to one-hot encoded format accordingly.

[0079] 2.3 Training environment setup.

[0080] The experiment was conducted on a server equipped with two A6000 GPUs and 128GB of system memory, running Windows 10. The code was implemented based on the Tensorflow framework and relied on CUDA 11.3. Model training used the Adam optimizer with a learning rate of 0.0009. The training process consisted of 500 epochs, and an early stopping mechanism was introduced to optimize the training process. Specifically, the accuracy on the validation set was monitored; if the validation accuracy did not improve within 100 consecutive epochs, training was terminated early, and the model weights from the optimal validation set performance were restored.

[0081] 2.4 Evaluation Indicators.

[0082] To objectively measure and compare the classification performance of the above models, classification accuracy and the kappa score (K-score) were used as the core evaluation metrics. Classification accuracy directly reflects the model's ability to correctly classify all test samples and is the most commonly used benchmark for evaluating the overall performance of the model. Its calculation formula is as follows:

[0083] (16) in, This represents the total number of samples correctly classified by the model. This represents the total number of samples in the test set. This metric measures the degree of consistency between the model's predictions and the true labels.

[0084] However, considering that in multi-class classification tasks, chance may lead to some samples being correctly classified, accuracy alone may not fully reflect the model's true discriminative ability. Therefore, the K-score is introduced as a supplementary evaluation metric. It measures the consistency between the model's predictions and random guesses, thus providing a more robust assessment of classification performance. Its calculation formula is:

[0085] (17) in, This represents the actual observational accuracy of the model (i.e., the accuracy defined above), while This represents the probability of accidental consistency of the expected outcome. The K-score is calculated by summing the products of the number of samples in each category predicted by the model and the number of samples in each category in the true labels, and then dividing by the square of the total number of samples. The K-score value is usually between -1 and 1. The closer the value is to 1, the better the model's classification performance and the greater the difference from random guessing.

[0086] 3. Results and Analysis.

[0087] 3.1 Ablation experiment.

[0088] To systematically verify the effectiveness and necessity of each core module in MDAT-Net, ablation experiments were conducted on the BCI-IV-2a dataset. Using the complete model as a baseline, the contribution of each module to performance was quantitatively evaluated by removing the MFM, SW, MDA, and ARSTCN components one by one. The experimental results (Table 1) show that there are indispensable synergistic relationships among the components. The specific experimental configuration is as follows:

[0089] Table 1 Comparison of model decoding performance under different component removals After removing the ARSTCN module, the model consists of MFM, SW, MDA, and a classification layer. The feature sequences are directly input into the global average pooling and classification layers without temporal modeling. The average accuracy decreased from 85.73% to 62.38%, and the K-score decreased from 0.8097 to 0.4985, indicating that this module plays a key role in capturing long-term dependencies.

[0090] When evaluating the MFM module, the model retained only the standard convolutional branch for spatiotemporal feature extraction, followed by single-branch SW, MDA, and ARSTCN. The average accuracy decreased to 70.02%, and the K-score was 0.6003, demonstrating that multi-scale feature extraction is an important prerequisite for high-precision decoding.

[0091] In the experiment where MDA was removed, the model structure consisted of MFM, SW, and ARSTCN, with the features processed by SW directly input into ARSTCN. The average accuracy decreased to 83.91%, and the K-score was 0.7855, indicating that the differential attention mechanism can effectively suppress noise and focus on key information.

[0092] After removing the SW module, the model includes MFM, MDA, and ARSTCN, and global attention is directly calculated for the complete feature sequence. The average accuracy decreased to 82.02%, and the K-score was 0.7603, indicating that segmented processing is the foundation for the reliable operation of the attention mechanism.

[0093] In summary, the ablation experiments demonstrate that the MDAT-Net architecture possesses high integrity and rationality. The functional modules, such as multi-scale feature extraction, attention focusing in noisy environments, and deep temporal modeling, are all indispensable; the absence of any component leads to a significant performance degradation. Together, they form a complementary and organic whole, ensuring the model's superior performance.

[0094] 3.2 Performance comparison.

[0095] To comprehensively evaluate the performance of MDAT-Net, the model was compared with seven state-of-the-art methods in EEG motor intention classification, including EEGNet, EEG-TCNet, TCNet-Fusion, FBCNet, EEG Conformer, ADFCNN, TMSA-Net, CLTNet, HCANN, and SMANet. To ensure the stability and reliability of the comparison, each comparison model was retrained and evaluated on three datasets with its original configuration.

[0096] On the four-class BCI-IV-2a dataset (Table 2), MDAT-Net demonstrates comprehensive performance advantages, with both average accuracy (85.73%) and K-score (0.8097) significantly higher than all compared methods. Specifically, compared to the second-best performing CLTNet (83.03%) and TMSA-Net (83.02%), this model improves average accuracy by more than 2.7 percentage points, representing a significant advancement in EEG decoding. Further analysis shows that traditional or relatively simple models such as EEGNet (74.85%) and FBCNet (76.47%) have a significant performance gap compared to MDAT-Net, indicating that simple convolutional structures are insufficient to fully capture the complex spatiotemporal features in motor imagery tasks. Compared to EEG-TCNet (77.97%) and TCNet-Fusion (79.90%), which also employ the TCN architecture, MDAT-Net's significant advantage reflects the crucial role of the MFM and MDA modules in enhancing feature representation and suppressing noise. Furthermore, compared to EEG Conformer (78.78%) and ADFCNN (82.87%), which incorporate Transformer or self-attention mechanisms, this model demonstrates stronger robustness in low signal-to-noise ratio EEG signals thanks to its differential attention mechanism, effectively mitigating the problem of attention distraction. Simultaneously, its standard deviation (7.81) is the lowest among all models, proving its superior cross-subject stability.

[0097] Table 2 Performance comparison of MDAT-Net and benchmark methods on the BCI-IV-2a dataset This advantage is evident in both binary classification tasks (BCI-IV-2b dataset, Table 3) and quadruple classification tasks (HGD dataset, Table 3). Figure 4 All values ​​were maintained. It is noteworthy that although BCI-IV-2b is a binary classification task, theoretically simpler than the four-class HGD task, MDAT-Net achieved a higher average accuracy on the latter (96.30% vs 90.04%). Analysis suggests this phenomenon stems from differences in the characteristics of the datasets themselves.

[0098] Table 3 Performance comparison of MDAT-Net and benchmark methods on the BCI-IV-2b dataset First, there is a significant difference in data quality and information richness. The BCI-IV-2b dataset contains only 3 EEG channels, with limited spatial resolution, which restricts the model's ability to capture fine spatial patterns to distinguish between left- and right-handed imagination tasks with highly similar neural activities. In contrast, the HGD dataset uses 44 channels covering the motor cortex, providing richer spatial information and helping to extract more discriminative features.

[0099] Secondly, the scale of training data is a key factor. The HGD dataset has more subjects (14) and more training trials per subject, resulting in a significantly larger overall dataset than the BCI-IV-2b dataset. Larger datasets help deep learning models learn the inherent patterns of the task more fully, improving generalization ability and robustness.

[0100] Finally, there are differences in the inherent separability of the tasks. The left and right hand imagery tasks of BCI-IV-2b correspond to highly overlapping EEG patterns in both spatial and frequency domains, making classification more challenging. In contrast, the four-class classification task of HGD includes both feet and resting states, and its neural representations differ more significantly from those of single-hand motor imagery, objectively reducing the possibility of confusion between categories and potentially leading to a higher overall average classification accuracy.

[0101] In summary, the advantages of the HGD dataset in terms of data quality, scale, and task separability are the core reasons why MDAT-Net achieves superior performance. Specifically, for the HGD dataset containing 14 subjects, histograms and line graphs were used to analyze the data. Figure 4 , Figure 5 This visually demonstrates the performance comparison between MDAT-Net and baseline methods. Figure 4 Presenting an overall performance comparison, Figure 5 The detailed classification accuracy of each subject is shown, collectively verifying the stable advantage of MDAT-Net in large-scale samples.

[0102] However, from Figure 5 The line graph reveals significant differences in decoding accuracy among different subjects, primarily attributed to the inherent challenge of "high inter-subject variability" in MI-EEG signals. This difference stems from multiple factors, including individual differences in brain structure and function, individual specificity of neural activity patterns during motor imagery tasks, levels of attention and fatigue during task execution, and differences in signal-to-noise ratio due to varying scalp impedance. These factors collectively result in clearer EEG signal patterns in some subjects, facilitating model learning, while other subjects may exhibit lower signal-to-noise ratios or unstable patterns, increasing the difficulty of decoding.

[0103] While MDAT-Net achieves leading average performance overall through its robust design, its decoding performance for individual subjects remains constrained by the quality of the original signal. Notably, even under this inherent challenge, the model still achieves over 90% accuracy for the majority of subjects (13 out of 14), and its overall performance and stability outperform other benchmark methods, further highlighting its strong generalization ability and application potential in processing complex and variable EEG signals.

[0104] The quantitative analysis results from three benchmark datasets show that MDAT-Net has better accuracy and robustness than existing state-of-the-art methods in motion imagery decoding tasks.

[0105] 3.3 Visualization.

[0106] To conduct an in-depth examination of the model's performance from two dimensions—individual performance and task differentiation—visual analysis was carried out.

[0107] First, to evaluate the model's ability to identify four different motion imagery tasks, Figure 6 The average classification confusion matrix for all subjects is presented. Values ​​on the diagonal of the matrix, ranging from 0.85 to 0.88, indicate that the model possesses accurate and balanced recognition capabilities for all four types of intent. All off-diagonal elements have values ​​not exceeding 0.068, indicating minimal confusion between different tasks. Even in left-hand and right-hand tasks, where neural patterns are relatively similar, the model demonstrates strong discriminative ability. These results demonstrate that the features extracted by MDAT-Net have high discriminability and can capture neural activity patterns corresponding to different motor imagery tasks, thereby achieving relatively accurate intent decoding.

[0108] After examining the task confusion of the model, Figure 7 Further visualization of the two-dimensional feature space reveals the model's intrinsic discrimination mechanism. The figure clearly presents the distribution of the learned features: data points corresponding to left hand (blue), right hand (orange), foot (green), and tongue (red) motor imagery exhibit significant intra-class aggregation and inter-class separation trends. Particularly noteworthy is that even between left and right hand tasks, which are relatively similar in neural representation, the feature regions maintain good distinguishing boundaries. This clear and orderly distribution in low-dimensional space intuitively confirms the model's powerful feature decoupling ability, providing a basis for further development. Figure 5 The extremely low classification confusion rate observed provides direct support at the spatial level, and from the perspective of feature learning, it confirms the model's high recognition of various motion imagination intentions.

[0109] After analyzing the task confusion situation Figure 7The model's intrinsic discrimination mechanism is further revealed through two-dimensional feature space visualization. The figure shows that the feature points corresponding to the motor imagery of the left hand (blue), right hand (orange), foot (green), and tongue (red) exhibit a trend of intra-class aggregation and inter-class separation. Notably, even between left- and right-hand tasks with relatively similar neural representations, the feature regions maintain good distinguishing boundaries. This clear and orderly distribution in low-dimensional space confirms the model's feature decoupling ability, providing a basis for further analysis. Figure 5 The relatively low classification confusion rate provides an explanation at the spatial feature level and further supports the model's high recognition of the intention of multiple types of motion from the perspective of feature learning.

[0110] 4. Discussion.

[0111] System experiments validated the performance of MDAT-Net in decoding motor imagery EEG signals. Results show that the model exhibits high classification accuracy and stability on multiple benchmark datasets, an advantage likely stemming from its structural modules designed specifically for the characteristics of MI-EEG signals. Compared to existing models, MDAT-Net integrates multi-scale feature extraction, differential attention, and adaptive temporal modeling mechanisms, forming a collaborative processing flow. The MFM module extracts diverse spatiotemporal features through convolutional kernels of different scales; the MDA module enhances sensitivity to key signal changes through differential operations and suppresses noise interference to some extent; and the ARSTCN module models long-range dependencies using deep temporal structures. The organic combination of these components provides a foundation for the model's performance.

[0112] 5. Conclusion.

[0113] This invention proposes an end-to-end deep learning model called MDAT-Net to address problems in decoding motor imagery EEG signals, such as insufficient feature extraction, susceptibility of attention mechanisms to noise interference, and difficulty in modeling long temporal dependencies. The model integrates a multi-branch fusion module, a differential attention mechanism, and an adaptive residual spatiotemporal convolutional network, and was evaluated on three public datasets (BCI-IV-2a, BCI-IV-2b, and HGD). Experimental results show that MDAT-Net meets or outperforms current mainstream methods in both four-class and two-class classification tasks, with an average classification accuracy of 85.73% on BCI-IV-2a, 90.04% on BCI-IV-2b, and 96.30% on HGD. This invention provides a new and effective approach for decoding MI-EEG signals and has reference value for advancing the development of high-performance brain-computer interface systems.

[0114] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A method for decoding and recognizing motor imagery electroencephalogram signals, characterized in that, The method comprises the following steps: acquiring a motor imagery electroencephalogram of a to-be-tested user; inputting the motor imagery electroencephalogram into a pre-trained multi-branch differential attention time network, the pre-trained multi-branch differential attention time network comprising: a multi-branch feature fusion module, a sliding window layer, a multi-head differential attention mechanism and an adaptive residual separable time convolution network connected in sequence; extracting the spatio-temporal features of the motor imagery electroencephalogram in parallel through the multi-branch feature fusion module by using convolution operations with different kernel sizes, so as to capture diversified neural activity patterns in the motor imagery electroencephalogram, and obtain a comprehensive spatio-temporal feature map; dividing the comprehensive spatio-temporal feature map into a plurality of continuous short sequence window features in the time dimension through the sliding window layer, each of the short sequence window features representing a neural activity pattern of the motor imagery electroencephalogram; for each short sequence window feature, performing linear transformation projection on the short sequence window feature through the multi-head differential attention mechanism module to obtain two groups of independent attention maps; determining a differential attention weight through the attention score difference of the two groups of independent attention maps, and weighting the short sequence window feature through the differential attention weight to obtain an enhanced window feature corresponding to the short sequence window feature; and splicing all the enhanced window features according to the division order to obtain a motor imagery time sequence; performing a plurality of one-dimensional separable convolution operations on the motor imagery time sequence through the adaptive residual separable time convolution network to capture long-distance time dependence of the motor imagery time sequence, and obtaining a final time sequence feature; performing classification probability prediction on the final time sequence feature through a softmax function to obtain a motor intention classification result of the to-be-tested user.

2. The method of claim 1, wherein the method further comprises: The multi-branch feature fusion module comprises a time convolution layer and a plurality of parallel feature extraction branches, and each feature extraction branch is composed of a depth separable convolution layer and a standard convolution layer or a separable convolution layer with different kernel sizes connected in series.

3. The method of claim 2, wherein the method further comprises: The method for extracting the spatio-temporal features of the motor imagery electroencephalogram in parallel through the multi-branch feature fusion module by using convolution operations with different kernel sizes, so as to capture diversified neural activity patterns in the motor imagery electroencephalogram, and obtain a comprehensive spatio-temporal feature map, specifically comprises extracting the local time pattern of the motor imagery electroencephalogram through the time convolution layer based on the following formula to obtain an initial feature map: ； wherein, is a motor imagery electroencephalogram signal, is a learnable weight of a temporal convolution layer, is a convolution operation, is a batch normalization process, is an initial feature map; for each feature extraction branch, performing spatial feature filtering on the initial feature map through the depth separable convolution layer of the feature extraction branch based on the following formula: ； wherein, is a depthwise separable convolution, is a weight of the depthwise separable convolution, is an exponential linear unit activation function, is an initial feature map after spatial feature filtering; extracting the dynamic changes of the features of the initial feature map after spatial feature filtering in different time scales through the standard convolution layer and the separable convolution layer with different kernel sizes of the feature extraction branch to obtain an output feature map; fusing all the output feature maps by using an element-by-element addition method to obtain a comprehensive spatio-temporal feature map.

4. The method of claim 3, wherein the method further comprises: extracting the dynamic changes of the features of the initial feature map after spatial feature filtering in different time scales through the standard convolution layer and the separable convolution layer with different kernel sizes of the feature extraction branch to obtain an output feature map based on the following formula: ； in, This is the initial feature map after spatial feature filtering. For the first Convolution operations for each feature extraction branch For the first The convolution kernel weights corresponding to the convolution operations of each feature extraction branch. For average pooling operation, It is the activation function of the exponential linear unit. For the first Output feature maps of each feature extraction branch; fusing all the output feature maps by using an element-by-element addition method to obtain a comprehensive spatio-temporal feature map based on the following formula: ； wherein, is the output feature map of each feature extraction branch, is the preliminary fused spatio-temporal feature map, is the comprehensive spatio-temporal feature map, is the random deactivation operation.

5. The method of claim 1, wherein the method further comprises: The integrated spatio-temporal feature map is divided into a plurality of continuous short sequence window features in a time dimension by a sliding window based on the following formula: ； wherein, is the th short sequence window feature, is the th short sequence window feature, is the starting index of the th short sequence window feature in the time dimension, is the ending index of the th short sequence window feature in the time dimension, is the th short sequence window feature, is the ending index of the th short sequence window feature in the time dimension, is the spatio-temporal feature map.

6. The method of claim 1, wherein the method further comprises: The short sequence window features are linearly transformed and projected by a multi-head differential attention mechanism module based on the following formula: ； ； ； wherein, is a short sequence window feature, is a query, is a key, is a value, , and is a learnable projection matrix; The query and the key are evenly divided into two groups of independent attention maps, and the differential attention weight is determined based on the difference between the attention scores of the two groups of independent attention maps according to the following formula: ； wherein, is a dimension of the attention head, is a learnable scalar, is an attention score of the first set of independent attention maps, is an attention score of the second set of independent attention maps; The short sequence window features are weighted by the differential attention weight based on the following formula to obtain enhanced window features corresponding to the short sequence window features: ； wherein, is the output projection matrix; All the enhanced window features are spliced in the division order based on the following formula to obtain the motor imagery time sequence: ； wherein, is the total number of windows, is the motor imagery time series.

7. The method of claim 1, wherein the method further comprises: The adaptive residual separable temporal convolution network comprises: a plurality of residual blocks connected in sequence; each residual block comprises: a one-dimensional separable convolution layer, a batch normalization layer, an activation function layer and a random dropout layer connected in sequence, and the one-dimensional separable convolution layer of each residual block is associated with the one-dimensional separable convolution layer of other residual blocks through a residual connection.

8. The method of claim 7, wherein the method further comprises: The adaptive residual separable temporal convolution network is used to perform a plurality of one-dimensional separable convolution operations on the motor imagery time sequence to capture long-distance time dependence in the motor imagery time sequence, and finally obtain the time sequence feature, specifically including: For each residual block, the input feature is subjected to depth direction convolution and point-by-point convolution by the one-dimensional separable convolution layer to obtain convolution output features; The convolution output features are subjected to distribution standardization by the batch normalization layer to obtain normalized features, and the normalized features are subjected to non-linear transformation by the activation function layer to obtain activation features; The activation features are subjected to regularization processing by the random dropout layer to obtain the output features of the residual block; the output features of each residual block are added to the corresponding input features through the residual connection to obtain the residual fusion features; The residual fusion features output by the last residual block are taken as the final time sequence feature of the motor imagery time sequence.

Images