A motor imagery electroencephalogram signal decoding method based on multi-scale filtering and attention mechanism

Abstract

A kind of motor imagination EEG signal decoding method based on multiscale filtering and attention mechanism, comprising: the original EEG data of subject is divided into single experiment electroencephalogram data and normalized;Training data is handled with time domain data enhancement strategy;EEG signal is input as four parallel networks, through a multiscale convolutional neural network including time convolution layer, spatial convolution layer and average pooling layer;All feature channels at each time point are used as token for Transform processing.The first residual sub-block of Transform is composed of layer normalization and attention mechanism, and the second residual sub-block is composed of layer normalization and channel multilayer perception structure;The space-time features output by four parallel networks are spliced along the channel dimension, and effective channel attention mechanism ECA is used for channel fusion;Two-dimensional convolution is used to classify test signal.The present application can improve the classification accuracy of motor imagination brain-computer interface and promote its practical application.

Description

Technical Field

[0001] This invention belongs to the fields of neuroscience and computer technology. More specifically, this invention relates to brain-computer interface technology and electroencephalogram (EEG) signal processing technology. Background Technology

[0002] Brain-computer interface (BCI) is a communication system that establishes a direct connection between the human brain and external devices. This system collects brain electrical activity or other neural signals by attaching electrodes to the surface of the brain or implanting chips. These signals are processed and decoded, converted into instructions or outputs that a computer can understand, allowing humans to directly control external devices using their thoughts. BCI is particularly important for people with severe motor impairments, as it allows them to connect their intentions with external devices through non-muscle channels, thereby improving their quality of life. BCI is also considered to have significant benefits for healthy subjects. It can assist healthy individuals in decision-making, improve the productivity of human-machine interfaces, and even reduce human-machine response time delays in remote operations.

[0003] Motor imagery (MI) is one of the most widely used cognitive tasks in BCI system design. MI refers to the spontaneous generation of control actions based on the user's mental activity without actual movement. When a user performs an MI task, the contralateral sensorimotor cortex of the brain exhibits event-related desynchronization (ERD), i.e., a decrease in the amplitude of the contralateral μ / β rhythm; after the user completes the MI task, the contralateral sensorimotor cortex exhibits event-related desynchronization (ERD), i.e., an increase in the amplitude of the contralateral μ / β rhythm. ERD and ERS reflect changes in the brain's electrical activity under specific tasks or stimuli. When imagining movements of different limbs, the characteristics of the electroencephalography (EEG) signals differ, and the motor imagery system effectively distinguishes the EEG signals generated by imagined movements based on these differences, thereby determining the user's motor intention.

[0004] Because EEG signals are non-stationary, with low spatial resolution and low signal-to-noise ratio, they pose a significant challenge to BCI decoding based on EEG signals. With the rapid development of deep learning technology, more and more researchers are applying deep learning to MI-BCI systems. Traditional manual feature extraction methods are not only time-consuming and laborious, but may also lose some important information from the original signal. Deep learning, on the other hand, can automatically learn effective features from the original EEG signal through an end-to-end learning approach. In deep learning-based MI-BCI research, due to individual differences among subjects and the non-stationarity of EEG signals, single-scale convolution is insufficient to extract enough information from the original EEG signal for effective recognition of the MI task. Therefore, how to extract discriminative features to improve MI-BCI decoding performance is a major challenge in MI-BCI research. Summary of the Invention

[0005] This invention aims to provide a method for decoding motor imagery EEG signals based on multi-scale filtering and attention mechanisms, named MSCA-Transformer. It introduces multi-scale convolutional kernels and an effective channel attention (ECA) mechanism to extract more discriminative feature signals. Under the condition of a small training set of samples, it can significantly improve the classification accuracy of MI-BCI, thereby promoting the practical application of MI-BCI.

[0006] The present invention discloses a method for decoding motor imagery EEG signals based on multi-scale filtering and attention mechanisms, which specifically includes the following steps:

[0007] Step 1: Data Preprocessing. Continuous EEG data from each subject are extracted into single-experiment data, and the single-experiment data are normalized.

[0008] Step 2: Time-domain data augmentation. During each training round, the training data of a single experiment of the same class is uniformly split in the time domain, and then randomly concatenated to increase the size of the dataset;

[0009] Step 3: Construction of Parallel Multi-Scale Convolutional Module. A parallel multi-scale network is introduced, using the original EEG signal as input to four parallel networks. This is achieved through a convolutional neural network comprising temporal convolutional layers, spatial convolutional layers, and average pooling layers, where the temporal convolutional kernels of the four branches have different sizes. Each branch uses two-dimensional convolution to extract the spatiotemporal features of the EEG signal. The temporal convolutional layer captures feature maps of EEG signals at different pass frequencies, the spatial convolutional layer transmits information between different electrode channels, and the average pooling layer along the time dimension is used for dimensionality reduction.

[0010] Step 4: Transformer Module Construction. The feature maps obtained from each branch of the convolutional module are rearranged, and all feature channels at each time point are used as tokens for the Transformer to process. The Transformer consists of two residual sub-blocks. The first residual sub-block consists of normalization and attention mechanisms, and the second residual sub-block consists of normalization and channel multilayer perceptron.

[0011] Step 5: Construction of the channel attention mechanism module. The spatiotemporal features output by the four parallel networks are concatenated along the channel dimension, and channel fusion is performed using the ECA channel attention mechanism;

[0012] Step 6: EEG Signal Classification Module. This module contains a convolutional layer that uses two-dimensional convolution for classification. The kernel size is the same as the input feature map size, and the output feature map channels are the number of classification task labels. Empty dimensions at the end of the input tensor are then removed to determine the result as the task label corresponding to a single test signal.

[0013] Step (1) includes the following specific steps:

[0014] (1.1) Extracting the continuous EEG signal of each subject into data for a single experiment means dividing the continuous EEG data into data for a single experiment according to the label of each experiment, that is, extracting a segment of EEG data before and after the experimental task prompt for subsequent data processing and task classification.

[0015] (1.2) The data from a single experiment are standardized using z-score, which scales the data to a range with a mean μ of 0 and a standard deviation σ of 1. The calculation formula is as follows:

[0016]

[0017] Step (2) includes the following specific steps:

[0018] (2.1) Use the EEG data collected in two different experimental periods as the training set and the test set, respectively;

[0019] (2.2) Divide the training samples belonging to the same class in each round of training data into M consecutive non-overlapping data segments (each segment contains the same number of sampling points);

[0020] (2.3) New artificial data are generated by randomly combining M segments of data, which are from the same segment of experimental data randomly selected from the same class. The additional training samples consist of segments from different experiments with the same time structure.

[0021] Step (3) includes the following specific steps:

[0022] (3.1) The original EEG signal is used as the input to four parallel networks. Each branch contains a temporal convolutional layer, a spatial convolutional layer, and an average pooling layer. The temporal convolutional layer introduces a multi-scale convolutional kernel with a kernel size of (1,K) and a stride of (1,1). The lengths K of the four branch convolutional kernels are 64, 40, 26, and 16, respectively, and the output F feature maps contain feature signals with different bandpass frequencies.

[0023] (3.2) The spatial convolutional layer has F convolutional kernels of size (C,1) and stride (1,1), where C represents the number of EEG electrode channels and the number of output feature maps (i.e. the number of feature channels) is F. This convolutional layer acts as a spatial filter to transmit information between different electrode channels.

[0024] (3.3) After the spatial convolutional layer, batch normalization is used to normalize the input data of each batch, and the rectified linear unit (ReLU) nonlinear activation function is used to increase the nonlinear expressive power of the neural network.

[0025] (3.4) The average pooling layer reduces the sampling rate of the EEG signal and retains the main features. The kernel size is (1,75) and the step size is (1,15). A new feature map is generated by averaging the values ​​of the local regions of the input feature map. That is, the values ​​of each region are averaged as the representative feature values ​​of that region.

[0026] (3.5) The extracted feature maps are rearranged, that is, the feature maps with dimensions (B,N,1,T1) are rearranged into feature maps with dimensions (B,T1,N), where B represents the batch size, N represents the feature channels, and T1 represents the number of time points after pooling. The purpose is to send all feature channels of each time point as a token into the Transformer module.

[0027] Step (4) includes the following specific steps:

[0028] (4.1) Divide the output feature map X of the previous module into h segments to form h heads, each head having a length of F. The first residual sub-block consists of layer normalization (LN) and a multi-head attention mechanism. After a linear transformation according to the following formula, the three matrices of the i-th (i = 1, 2, ..., h) head are obtained, namely query (Query, Q), key (Key, K) and value (Values, V).

[0029]

[0030] (4.2) Calculate the dot product between the key and the query of the i-th header according to the following formula, and divide each dot product by... The softmax function is applied to obtain the weights of the values, and the weights are multiplied by the values ​​to obtain the attention for the i-th head. In the formula, d... kIndicates the length of the token. This represents the scaling factor, used to avoid the gradient vanishing problem;

[0031]

[0032] (4.3) Stack the attention of all heads according to the following formula to form the final multi-head output;

[0033] MultiHead(Q,K,V)=[head1,...,head h W o

[0034] head i =Attention(Q) i ,K i V i (4)

[0035] (4.4) The second residual sub-block consists of layer normalization and channel multilayer perceptron (MLP). The channel MLP realizes information fusion in the channel domain, takes the output of the residual connection as input, and can transmit information between channels through the MLP block. It contains two fully connected layers and Gaussian rectified linear unit (GELU) activation function to generate the final output.

[0036] Step (5) includes the following specific steps:

[0037] (5.1) Concatenate the outputs of the four branches according to the channel dimension:

[0038] x = concat(x1,x2,x3,x4) (5)

[0039] In the formula, x1, x2, x3, and x4 represent the output feature maps of each branch;

[0040] (5.2) The input feature map size is (N,C,H,W), where N represents the batch size, C represents the number of feature channels, H represents spatial features, and W represents temporal features. Global average pooling is used to aggregate the information of each channel of the input feature map. After global average pooling, the output size is (N,C,1,1). Then, the channel weights are calculated through one-dimensional convolution. The convolution kernel size is an adaptive size k, which is mapped to 0-1 using a sigmoid function. Finally, the scaled weights are weighted with the input feature map to obtain the output feature map under different weights. The adaptive convolution kernel in ECA is calculated as follows:

[0041]

[0042] In the formula, C represents the feature channel, the kernel size k represents the range of local cross-channel interactions, i.e., how many neighbors participate in the attention prediction of a channel, and k is proportional to C; γ and b are two hyperparameters; |t| odd This means taking the nearest odd number to t.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] (1) This invention provides a multi-scale filtering and attention mechanism motor imagery EEG decoding method, which can better extract global information of local features, making the extracted EEG signal features more comprehensive and able to extract discriminative EEG features;

[0045] (2) The present invention can achieve high classification accuracy and classification robustness under the condition of small training dataset, which helps to build a high-precision MI-BCI system and promotes the practical application of motion imagery BCI system. Attached Figure Description

[0046] Figure 1 MSCA-Transformer Motion Imagination BCI Decoding Method.

[0047] Figure 2 Example dataset: Motion Imagination BCI Experiment Timing Scheme.

[0048] Figure 3 A schematic diagram of time-domain data augmentation strategies.

[0049] Figure 4 A schematic diagram of the network structure of a convolutional neural network.

[0050] Figure 5 A diagram illustrating the network structure of a Transformer.

[0051] Figure 6 A schematic diagram of the network structure of the ECA channel attention mechanism module.

[0052] Figure 7 MSCA-Transformer and ablation experiment classification performance diagram. Detailed Implementation

[0053] To clearly illustrate the objectives, technical solutions, and advantages of the present invention, a detailed description is provided below in conjunction with the accompanying drawings and embodiments. The specific embodiments described herein are merely illustrative of the technical solutions of the present invention and are not intended to limit the scope of the invention.

[0054] This invention provides an MSCA-Transformer network model based on multi-scale filtering and attention mechanisms. This network consists of four parts: a parallel multi-scale convolution module, a Transformer module, a channel attention mechanism (ECA) module, and an EEG signal classification module. Figure 1 As shown.

[0055] Using dataset 2a, a four-class motion imagery dataset provided by the 4th International BCI Competition, this invention illustrates the task recognition method for motion imagery BCI. This embodiment uses Python to complete all program code. A brief description of this dataset is as follows:

[0056] This dataset was recorded in an uncontrolled environment, resulting in numerous artifacts, making decoding the motor imagery task a challenging task. The dataset contains EEG data from nine healthy subjects performing four tasks: left-hand, right-hand, bilateral foot, and tongue motor imagery. Each experiment's task was given via a cue. The dataset was acquired at a sampling rate of 250 Hz, using 22 electrodes, and at two time points (T and E). Each subject underwent 288 experiments per time point, with 72 experiments per task (i.e., per category). The timing scheme for the motor imagery BCI experiment is as follows: Figure 2 As shown. The motor imagery begins with a task prompt and ends after 4 seconds of performing the imagery task. Each experiment ends after a short rest of 1.5-2.5 seconds.

[0057] The present invention discloses a method for decoding motor imagery EEG signals based on multi-scale filtering and attention mechanisms, the specific implementation steps of which are as follows:

[0058] Step 1: Data Preprocessing. Continuous EEG signals for each subject are truncated, and z-score normalization is performed on the EEG signals from a single experiment.

[0059] (1.1) The truncation of continuous EEG signals refers to dividing continuous EEG data into data from a single experiment, that is, truncating a segment of data after each experimental task prompt for subsequent data processing. For the MACA-Transformer model, the data segment is truncated from -0.5s to 4s, meaning each experiment is a matrix with a dimension of 22×1125;

[0060] (1.2) The EEG signals from a single experiment were z-score normalized to scale the data to a range with a mean of 0 and a standard deviation of 1. First, the mean and standard deviation of the original EEG data were calculated; then, the original EEG data were centered by subtracting the mean μ from each data point; finally, the centered data was divided by the standard deviation σ to scale the data amplitude. The calculation formula is as follows:

[0061]

[0062] Step 2: Time-domain data augmentation. During each training round, training samples of the same class are uniformly split and then randomly concatenated. Data augmentation strategies are as follows: Figure 3 As shown.

[0063] (2.1) The dataset contains datasets for two time periods (T and E). Data collected in time period T is used as the training set, and data collected in time period E is used as the test set.

[0064] (2.2) Divide the training samples belonging to the same class in each round of training data into M segments (M=3 in the figure) of continuous non-overlapping data (each segment contains the same number of EEG samples);

[0065] (2.3) New artificial data are generated by random combination, which are derived from the concatenation of randomly selected segments of the same class. The additional samples consist of parts of different experiments with the same time structure.

[0066] Step 3: Construction of Multi-Scale Parallel Convolutional Modules. Multi-scale convolutional modules can be used for deep learning of EEG signals induced by motion visualization, etc. The input EEG data is fed into four branch networks. Each branch contains one input layer, two convolutional layers, one batch normalization layer, one rectified linear unit, one pooling layer, and one dropout layer. A schematic diagram of the convolutional neural network is shown below. Figure 4 As shown. The two convolutions serve the following purposes: the first convolution is a temporal convolution to learn the temporal filter; the second convolution is a spatial convolution to learn the spatial filter; the pooling layer is used to reduce the sampling rate.

[0067] (3.1) The input EEG data is fed into four branch networks. The temporal convolutional layer has F convolutional kernels with a kernel size of (1,K) and a stride of (1,1), where K represents the length of the filter on the time axis. The output F feature maps contain feature signals with different pass frequencies. Multi-scale convolutional kernels are introduced in the temporal filter design, with kernel lengths set to 64, 40, 26, and 16, respectively. To ensure that the outputs of the four parallel branches are consistent after feature extraction and to facilitate subsequent feature fusion, the padding of the temporal convolution needs to be set appropriately. Specifically, the padding should be set to half the kernel size. According to the convolution output calculation formula given in the PyTorch official documentation:

[0068]

[0069] In the formula, L in L represents the input size. outThe output size is represented by padding, the number of zero-padding layers, kernel_size, stride, and dilation. The default value for dilation is 1, meaning that 2×padding and kernel_size can cancel each other out, resulting in an output size of (C, 1126). This ensures consistent output size, meaning that the output size of the four parallel branches after convolution remains the same.

[0070] (3.2) The temporal feature map is fed into the spatial convolutional layer. The spatial convolutional layer has F convolutional kernels of size (C,1) and stride of (1,1). C represents the number of electrode channels of the EEG signal. The channel size of the output feature map is F. This convolutional layer acts as a spatial filter to transmit information between different electrode channels.

[0071] (3.3) The batch normalization method is used to normalize the input data of each batch, and the ReLU nonlinear activation function is used to increase the nonlinear expressive power of the neural network;

[0072] (3.4) The average pooling layer reduces the sampling rate of the EEG signal and retains the main features. The kernel size is (1,75) and the stride is (1,15). The output feature map is generated by averaging the values ​​of the local regions of the input feature map. That is, the values ​​of each region are averaged as the representative feature values ​​of that region.

[0073] (3.5) The extracted feature maps are rearranged, that is, the feature maps with dimensions (B,N,1,T1) are rearranged into feature maps with dimensions (B,T1,N), where B represents the batch size, N represents the number of feature channels, and T1 represents the number of time points. The purpose is to send all feature channels of each time point as tokens to the next module.

[0074] Step 4: Transformer Module Construction. Each branch of the Transformer module contains two residual sub-blocks, and its network structure diagram is shown below. Figure 5 As shown in (a), the first residual sub-block consists of a normalization and multi-head attention (MHA) module, as follows. Figure 5 As shown in (b), the Scaled Dot-ProductAttention module is as follows: Figure 5 As shown in (c), the second residual sub-block consists of a normalized multi-layer perceptron (MLP). The Transformer network can be represented by the following formula:

[0075] Y = MHA(Norm(X)) + X (3)

[0076] Z=σ(Norm(Y)W1)W2+Y (4)

[0077] In the formula, X represents the output token of a single convolutional module, Norm(·) represents normalization, W1 and W2 represent the learnable parameters in the channel MLP, and σ(·) represents the activation function.

[0078] (4.1) Treat all channels corresponding to each time point as a token, divide all tokens at all time points into h segments, and the length of each segment is F, where h represents the number of heads in the multi-head attention mechanism, and F represents the number of feature channels. Normalize each segment before inputting it into the multi-head attention mechanism;

[0079] (4.2) After performing Layer Normalization (LN) on the input feature map X, it is projected into three matrices: Query (Q), Key (K) and Values ​​(V);

[0080]

[0081] In the formula, i represents the i-th head, W q W k With W v These are the projection matrices for Q, K, and V, respectively.

[0082] (4.3) Calculate all bonds K i With query Q i The dot product between them, each dot product divided by And apply the softmax function to obtain the value V. i The weights. d k It is the length of the token. This represents the scaling factor, used to avoid the gradient vanishing problem;

[0083]

[0084] (4.4) The outputs of the head are connected together to form the final output according to the following formula;

[0085] MultiHead(Q,K,V)=[head1,...,head h W o

[0086] head i =Attention(Q) i ,K i Vi (7)

[0087] (4.5) The output of the first residual sub-block is used as the input of the second residual sub-block. Normalization is performed first, and then the information is transferred through the channel MLP. It contains two fully connected layers and the GELU activation function to generate the final output.

[0088] Step 5: Construction of the Channel Attention Mechanism Module. The spatiotemporal features output by the four parallel networks are concatenated along the channel dimension. An Efficient Channel Attention (ECA) module is used for channel fusion, which significantly improves the classification performance of the entire network with only a few additional parameters. ECA mainly optimizes two aspects: first, it removes the fully connected layers in the Squeeze-and-Excitation (SE) network, using one-dimensional convolutions to achieve a local cross-channel interaction strategy without dimensionality reduction; second, it uses an adaptive kernel size selection method to determine the range of cross-channel interaction, allowing the model to dynamically adjust the kernel size based on the number of channels in the input feature map. The network structure of the ECA-based channel attention mechanism module is as follows: Figure 6 As shown.

[0089] (5.1) Concatenate the outputs of the four branches according to the channel dimension:

[0090] x = concat(x1,x2,x3,x4) (8)

[0091] In the formula, x1, x2, x3, and x4 represent the output feature maps of the four branches;

[0092] (5.2) x is used as the input to the ECA module, with a size of (N,C,H,W), where N represents the batch size, C represents the number of channels, H represents the spatial features, and W represents the temporal features. Global Average Pooling (GAP) is used to aggregate the channel information of the input features, and the output size after GAP is (N,C,1,1). Then, the channel weights are calculated through one-dimensional convolution, with the convolution kernel size being an adaptive size k. The Sigmoid function is used to map the features to the range of 0-1. Finally, the scaled weights are applied to the input feature map to obtain the output feature map under different weights. The formula for calculating the adaptive convolution kernel in ECA is as follows:

[0093]

[0094] In the formula, C represents the feature channel, the kernel size k represents the range of local cross-channel interactions, i.e., how many neighbors participate in the attention prediction of a channel, and k is proportional to C; γ and b are two hyperparameters, set to 2 and 1 respectively; |t| odd This means taking the nearest odd number to t.

[0095] Step 6: Classification Module Construction. This module contains a convolutional layer that uses two-dimensional convolution for classification. The kernel size is the same as the input feature map size, and the output feature map channels represent the number of classification task labels. Empty dimensions at the end of the input tensor are then removed to determine the result as the task label corresponding to a single test signal.

[0096] Step 7: Performance Evaluation. In the MI-based BCI research, to verify the effectiveness of each part of the MSCA-Transformer model, ablation experiments were conducted, comparing four algorithms: 1) data augmentation removed; 2) parallel Transformer module removed; 3) ECA channel attention mechanism module removed, i.e., only concat is used to stack feature maps by channel; 4) the complete MSCA-Transformer model. To evaluate the actual effect of the signal decoding method provided by this invention, we compared the classification performance of these four algorithms.

[0097] Figure 7 The graph shows the classification accuracy for each subject using these four algorithms, compared to the average accuracy. As can be seen from the graph, except for subjects A02 and A09, the classification accuracy of the MSCA-Transformer is greater than or equal to that of the other methods. On average for each subject, the classification accuracies of data augmentation removal, Transformer, ECA channel attention removal, and MSCA-Transformer are 77.13%, 78.55%, 78.43%, and 80.21%, respectively. The latter is 3.08%, 1.66%, and 1.78% higher than the other three, respectively.

[0098] Using paired Student's t-tests (confidence probability set at 95%), we performed statistical analysis on the average classification accuracy of these four methods, such as... Figure 7 As shown in the figure. ** indicates p < 0.01, and * indicates p < 0.05. The t-test results show that the classification accuracy of MSCA-Transformer is significantly higher than that of the other three algorithms.

[0099] These results fully demonstrate that the method provided by this invention, based on the deep learning network MSCA-Transformer with multi-scale filtering and attention mechanisms, for decoding motion imagery BCI signals is extremely effective and helps to improve the classification performance and practical application of BCI systems.

Claims

1. A method for decoding motor imagery EEG signals based on multi-scale filtering and attention mechanisms, characterized by: Includes the following steps: Step 1: Data preprocessing: Extract the continuous EEG data of each subject into single-experiment data and normalize the single-experiment data. Step 2: Time-domain data augmentation: During each round of training, the training data of a single experiment of the same class is uniformly divided in the time domain, and then randomly concatenated to increase the size of the dataset. Step 3: Parallel Multi-Scale Convolution Module Construction: A parallel multi-scale network is introduced, using the original EEG signal as input to four parallel networks. This is achieved through a convolutional neural network comprising temporal convolutional layers, spatial convolutional layers, and average pooling layers. The temporal convolutional kernels of the four branches have different sizes. Each branch uses two-dimensional convolution to extract the spatiotemporal features of the EEG signal. The temporal convolutional layer captures feature maps of EEG signals with different pass frequencies, the spatial convolutional layer transmits information between different electrode channels, and the average pooling layer along the time dimension is used for dimensionality reduction. Step 4: Transformer module construction: The feature maps obtained from each branch of the convolution module are rearranged, and all feature channels at each time point are used as tokens for the Transformer to process. The Transformer consists of two residual sub-blocks. The first residual sub-block consists of normalization and attention mechanisms, and the second residual sub-block consists of normalization and channel multilayer perception mechanisms. Step 5: Construction of the channel attention mechanism module: The spatiotemporal features output by the four parallel networks are spliced ​​along the channel dimension, and the channel fusion is performed using the ECA channel attention mechanism; Step 6: EEG signal classification module: This module contains a convolutional layer that uses two-dimensional convolution for classification. The size of the convolutional kernel is the same as the size of the input feature map. The number of channels in the output feature map is the number of classification task labels. The empty dimensions at the end of the input tensor are then removed so that the result can be judged as the task label corresponding to a single test signal.

2. The method for decoding motor imagery EEG signals based on multi-scale filtering and attention mechanisms according to claim 1, characterized in that: Step 1 includes the following steps: (1.1) Extracting continuous EEG signals from each subject into data for a single experiment means dividing continuous EEG data into data for a single experiment according to the label of each experiment, that is, extracting a segment of EEG data before and after the experimental task prompt for subsequent data processing and task classification. (1.2) Use z-score to standardize the data from a single experiment, scaling the data to a scale with a mean. 0 and standard deviation For values ​​within the range of 1, the calculation formula is as follows: (1) 3. The method for decoding motor imagery EEG signals based on multi-scale filtering and attention mechanisms according to claim 1, characterized in that: Step 2 includes the following steps: (2.1) Use the EEG data collected in two different experimental periods as the training set and the test set, respectively; (2.2) Divide the training samples belonging to the same class in each round of training data into M consecutive non-overlapping data segments, each segment containing the same number of sampling points; (2.3) New artificial data are generated by randomly combining M segments of data. These data come from the same segment of experimental data randomly selected from the same class. The additional training samples are composed of segments from different experiments with the same time structure.

4. The method for decoding motor imagery EEG signals based on multi-scale filtering and attention mechanisms according to claim 1, characterized in that: Step 3 includes the following steps: (3.1) The original EEG signal is used as the input of four parallel networks. Each branch contains a temporal convolutional layer, a spatial convolutional layer and an average pooling layer. The temporal convolutional layer introduces a multi-scale convolutional kernel with a kernel size of (1, K) and a stride of (1, 1). The lengths K of the four branch convolutional kernels are 64, 40, 26 and 16, respectively. The output F feature maps contain feature signals with different bandpass frequencies. (3.2) The spatial convolutional layer has F convolutional kernels of size (C, 1) with stride (1, 1), where C represents the number of EEG electrode channels and F represents the number of output feature maps. This convolutional layer acts as a spatial filter to transmit information between different electrode channels. (3.3) After the spatial convolutional layer, batch normalization is used to normalize the input data of each batch, and the ReLU nonlinear activation function of the rectified linear unit is used to increase the nonlinear expressive power of the neural network. (3.4) The average pooling layer reduces the sampling rate of the EEG signal and retains the main features. The kernel size is (1, 75) and the step size is (1, 15). A new feature map is generated by averaging the values ​​of local regions of the input feature map. That is, the values ​​of each region are averaged as the representative feature values ​​of that region. (3.5) The extracted feature maps are rearranged, that is, the feature maps with dimensions (B, N, 1, T1) are rearranged into feature maps with dimensions (B, T1, N), where B represents the batch size, N represents the feature channels, and T1 represents the number of time points after pooling. The purpose is to send all feature channels of each time point as a token into the Transformer module.

5. The method for decoding motor imagery EEG signals based on multi-scale filtering and attention mechanisms according to claim 1, characterized in that: Step 4 includes the following steps: (4.1) Divide the output feature map X of the previous module into h segments to form h heads, each head having a length of F; the first residual sub-block is composed of layer normalized LN and multi-head attention mechanism, and the three matrices of the i-th head are obtained through the linear transformation of the following formula. That is, query, Q, key, K, and value, V; (2) (4.2) Calculate the dot product between the key and the query of the i-th header according to the following formula, and divide each dot product by... The softmax function is applied to obtain the weights of the values, and the weights are multiplied by the values ​​to obtain the attention of the i-th head; where Indicates the length of the token. This represents the scaling factor, used to avoid the gradient vanishing problem; (3) (4.3) Stack the attention of all heads according to the following formula to form the final multi-head output; (4) (4.4) The second residual sub-block consists of layer normalization and channel multilayer perceptron (MLP). The channel MLP realizes information fusion in the channel domain, takes the output of the residual connection as input, and can transmit information between channels through the MLP block. It contains two fully connected layers and Gaussian rectified linear unit (GELU) activation function to generate the final output.

6. The method for decoding motor imagery EEG signals based on multi-scale filtering and attention mechanisms according to claim 1, characterized in that: Step 5 includes the following steps: (5.1) Connect the outputs of the four branches according to the channel dimension: (5) In the formula, This represents the output feature map of each branch; (5.2) The input feature map size is (N, C, H, W), where N represents the batch size, C represents the number of feature channels, H represents the spatial features, and W represents the temporal features. Global average pooling is used to aggregate the channel information of the input features. After global average pooling, the output size is (N, C, 1, 1). Then, the channel weights are calculated through one-dimensional convolution. The convolution kernel size is an adaptive size k, which is mapped to 0-1 using Sigmoid. Finally, the scaled weights are weighted with the input feature map to obtain the output feature map under different weights. The adaptive convolution kernel in ECA is calculated as follows: (6) In the formula, C represents the feature channel, the kernel size k represents the range of local cross-channel interaction, that is, how many neighbors participate in the attention prediction of a channel, and k is proportional to C; b are two hyperparameters; This means taking the nearest odd number to t.

Images