Motor imagery action recognition system based on lightweight attention network

Abstract

The invention provides a motor imagery action recognition system based on a lightweight attention network, and belongs to the field of biomedical engineering. According to the system, on the basis of an EEGNet network, two unique attention modules, namely a channel attention module and a deep attention module, are integrated, so that features are effectively integrated from multiple dimensions to extract more discriminative features, and the performance in a motor imagery classification task is remarkably improved. The deep learning method is adopted, the manual feature extraction step in a traditional method is avoided, the satisfactory classification effect is achieved while low calculation cost is kept, and the system has important application value and potential academic significance in the related fields such as neural rehabilitation and human-computer interaction.

Description

Technical Field

[0001] The present invention belongs to the field of biomedical engineering, and in particular relates to a motor imagery action recognition system based on a lightweight attention network. Background Art

[0002] Brain Computer Interface (BCI) can realize information interaction between the brain and external devices by extracting EEG signals and converting them into control commands that can be understood by external devices. It is of great significance to improve the quality of life of people with disabilities and promote the development of human-computer interaction. Motor imagery is an important paradigm to drive BCI, and robust feature extraction algorithms play a key role in improving the performance and application of motor imagery BCI systems. However, due to the high-dimensional and variable spatiotemporal characteristics and complex topological distribution of EEG signals, researchers usually need to use highly specialized data processing techniques to extract classification features closely related to specific actions. The subsequent feature screening process relies on the professional experience and judgment of researchers, lacks objectivity and requires a lot of manpower and time costs. In recent years, deep neural networks have been widely used due to their end-to-end characteristics and excellent automatic feature extraction performance, which greatly reduces the repetitive work of researchers so that they can focus more on the study of mechanisms and models. However, it should be noted that due to the non-stationarity and nonlinearity of EEG signals, traditional neural network methods may perform poorly in motor imagery action recognition. In addition, existing studies generally believe that the internal structure of EEG signals is relatively simple, and too deep or too complex structures will reduce their classification performance. Based on this, the present invention proposes a motor imagery action recognition system based on a lightweight attention network. The system cleverly integrates the channel attention module and the deep attention module in the forward propagation process of the EEGNet neural network, and then integrates the high-dimensional spatiotemporal information of EEG data with efficient computing performance by using deep convolution (DepthwiseConv2D), separable convolution (SeparableConv2D) and pointwise convolution (PointwiseConv2D), while significantly improving the accuracy and efficiency of the motor imagery classification system. Compared with traditional methods, the system uses the attention mechanism to focus the recognition on information that is useful for system classification, overcoming challenges such as low signal-to-noise ratio of EEG signals and difficulty in information integration. At the same time, in response to problems such as high computational cost and high model complexity of deep neural networks, the system greatly reduces the computing cost with a small number of computing parameters and extremely fast computing speed. While improving the performance of motor imagery action recognition, it can provide users with a more accurate and smooth interactive experience, which is of great significance for promoting the application of BCI technology in sports rehabilitation, neuroinformatics research and human-computer interaction. Summary of the invention

[0003] In order to address the problems of low signal-to-noise ratio of EEG signals, difficulty in information integration, high computational cost of deep neural networks, and high model complexity, the present invention proposes a motor imagery action recognition system based on a lightweight attention network.

[0004] The present invention proposes a motor imagery action recognition system based on a lightweight attention network, comprising the following steps:

[0005] Step S1: Perform the following preprocessing steps on the original EEG signal:

[0006] Apply 4-38HZ bandpass filtering to the collected EEG signals to extract the frequency band of interest, normalize and scale the filtered data to the range of [-1,1], and whiten the original signal using the Euclidean alignment method;

[0007] Step S2: Implementation and model initialization of the baseline network EEGNet:

[0008] The benchmark network EEGNet includes: Block1, Block2, Block3 in sequence; Block1 includes: two-dimensional convolution layer, deep convolution layer; Block2 includes: deep convolution layer, point convolution layer; Block3 includes: classification module with linear layer; The benchmark network EEGNet is implemented in the pytorch environment, and the Xavier network initialization operation is applied to the model;

[0009] Step S3: Construction of channel attention module:

[0010] Construct a channel attention module and embed it between Block2 and Block3 of EEGNet;

[0011] Step S4: Construction of deep attention module:

[0012] Build a deep attention module and embed it between the channel attention module and Block3;

[0013] Step S5: Model training and classification system output:

[0014] The data processed in step S1 is divided into training set, validation set and test set. The Adam optimizer is used to train the model with the cross entropy loss function, and the model parameters are saved. The saved optimal model is applied to the test set, and the classification accuracy of each category, the total accuracy and the Kappa value of the model are output.

[0015] Furthermore, the data preprocessing step in step S1 is as follows:

[0016] Step S11: using a 4-38 Hz finite impulse response filter to extract frequency band data of interest in the motor imagery task;

[0017] Step S12: In order to facilitate the subsequent neural network training, the filtered data is scaled to the range of [-1,1] using maximum normalization: where x i,j represents the data value of the i-th row and j-th column after filtering, x i,j ' represents the data in the i-th row and j-th column after normalization, max(x 1,1 |,x 1,2 |,…x i,j |) represents the maximum absolute value of the sample data after filtering;

[0018] Step S13: Use the Euclidean alignment method to whiten the data normalized in step S12: in is the data after whitening, represents the average covariance of the normalized data, x i represents the i-th data sample, Represents x i is the transpose of , and N is the number of samples.

[0019] Furthermore, the implementation and model initialization steps of the reference network EEGNet in step S2 are as follows:

[0020] Step S21: The EEGNet network is divided into three blocks. In Block1, two deep convolutional layers are executed sequentially. First, F1 temporal filters are used, and then they are sent to the deep convolutional layer to learn spatial filters. Since the deep convolutional layer does not use all feature maps, the trainable parameters are reduced to a certain extent. In Block2, deep convolution and F2 point-by-point convolutions with kernel size (1,1) are used to further reduce model parameters and integrate the information within and between feature maps. In Block3, the obtained features are sent to the linear layer and the N-type Softmax function. Output the final result;

[0021] Step S22: The model applies Xavier initialization to the deep convolutional layer and the linear layer. By setting the initial weights within a reasonable range, it can ensure that the activation values ​​at the beginning of the neural network model training are kept at an appropriate scale; if the convolutional layer and the linear layer contain bias terms, then setting their initial values ​​to zero helps the neural network model maintain a small output value at the beginning of training;

[0022] Step S23: Set the input channel and output channel of the two-dimensional convolution layer in Block 1 to 1 and 8 respectively, where the convolution kernel size is (1,64), the step size is 1, and the padding size is 32, and then normalize the convolution result and output it;

[0023] Step S24: Set the input channel and output channel of the deep convolution layer in Block 1 to 8 and 16 respectively, where the convolution kernel size is (number of input data channels, 1) and the number of groups is 8; after that, normalize the output and use the ELU activation function: Where x is the input data; and the average pooling layer of size (1,4) with a random dropout rate of 0.5;

[0024] Step S25: Set the input and output channels of the deep convolution layer in Block 2 to 16, the convolution kernel size to (1, 16), the step size to 1, the number of groups to 16, and the padding size to 8;

[0025] Step S26: Set the input and output channels of the point-wise convolution in Block 2 to 16, with the convolution kernel size of (1,1) and the step size of 1. After that, normalize the output and use the ELU activation function: (where x is the input data) and an average pooling layer of size (1,8) with a dropout rate of 0.5;

[0026] Step S27: Send the result obtained in Block 2 to the classification module of Block 3 to obtain the final classification result.

[0027] Furthermore, the steps of constructing the channel attention module in step S3 are as follows:

[0028] Step S31: The time series of each channel in the EEG data may contain important information, and the channel attention module can capture the main trends in the time series. The channel attention module uses an adaptive global maximum pooling layer and an average pooling layer to extract the maximum and average values ​​of each channel from the input data, and then concatenates these statistics into a feature vector; then, a linear layer sequence is used to calculate the attention weight of each channel; these weights are then multiplied with the original input data to generate weighted data as the final output of this module;

[0029] Step S32: using adaptive average pooling and adaptive maximum pooling of size (1, 1) for the data of the input channel attention module respectively;

[0030] Step S33: remove the last two dimensions of the two types of pooling results through the squeeze function;

[0031] Step S34: concatenate the two types of results obtained in step S33 along the channel dimension, and expand the size of the channel dimension to twice the size of the module input data;

[0032] Step S35: Send the result obtained in S34 to the linear layer without bias, convert the number of channels into the number of module input data channels, and then pass the Relu activation function: f(x)=max(0,x);

[0033] Step S36: Send the result obtained in S35 to the linear layer without bias, and its input and output are both the number of module input data channels;

[0034] Step S37: Send the result obtained in S36 to the Sigmoid activation function: Get the attention weight of the channel attention module:

[0035] Step S38: Reshape the attention weights obtained in S37 to match the shape of the module input data, and apply them to the module input data through element-by-element multiplication to achieve weighted adjustment of each channel.

[0036] Further, the steps of constructing the deep attention module in step S4 are as follows:

[0037] Step S41: For the deep attention module, first use the adaptive average pooling layer to perform global pooling operations on the spatial dimension of the module input data while maintaining the integrity of the time dimension; then the convolution layer further extracts the depth information of the pooled result; the adjustable convolution kernel slides on the depth dimension, captures the correlation between features of different dimensions through local cross-depth interaction, and filters out information related to the decoding task; finally, the module uses the Softmax function to perform a probabilistic transformation on the result of the convolution processing on the last dimension, generates the deep attention weight, and then uses the interpolation function to reshape it into the shape of the module input data, and multiplies it with the module input data and parameter b (parameter b value is 16), so as to generate weighted data as the final output of this module;

[0038] Step S42: using adaptive average pooling of size (1, a) on the data input to the deep attention module, where parameter a is an adjustable parameter;

[0039] Step S43: Send the result obtained in S42 to a two-dimensional convolution layer with input and output channels of 16 and 1 respectively, where the convolution kernel size is (7, 1), the step size is 1, and the padding size is 3;

[0040] Step S44: Send the result obtained in S43 to The function normalizes the convolution output;

[0041] Step S45: interpolate the result obtained in S44 to restore it to the module input data shape, and obtain the attention weight of the deep attention module;

[0042] Step S46: Multiply the deep attention weight obtained in S45 by the module input data element by element, and introduce parameter b (parameter b value is 16) during the multiplication process to scale the module output to prevent the weight from being too small and affecting the module weighting process.

[0043] Furthermore, the steps of model training and classification system output in step S5 are as follows:

[0044] Step S51: In order to better train and verify the data, the data preprocessed in step S1 is divided into a training set, a validation set and a test set. At the same time, the training set adopts a ten-fold cross-validation method; before the model training starts, the parameters are sent to the Adam optimizer to facilitate adaptive adjustment of the learning rate. When the data is sent to the neural network training, the cross entropy loss function can be obtained: Where c is the total number of categories, y i is the true i-th class label, is the model’s predicted probability for the i-th class;

[0045] Step S52: Split the preprocessed data into a training set, a validation set, and a test set. Use the training set to train the model, use the validation set to monitor the model training process, and use the test set to output the system classification results;

[0046] Step S53: The system uses the Adam optimizer and the cross entropy loss function: At the same time, a ten-fold cross-validation method is used during training to evaluate the stability and robustness of the model;

[0047] Step S54: when the model is subjected to ten-fold cross validation, if the accuracy of a certain validation set is greater than the highest validation set accuracy currently recorded, the current neural network model parameters are saved as the optimal parameters and the highest validation set accuracy is recorded as the current value;

[0048] Step S55: Apply the optimal model obtained in step S54 to the test set segmented in step S52. At the same time, in order to compare the recognition difficulty and overall performance of each category in N categories, this system takes the accuracy rate of each category and the average accuracy rate as test criteria to test the performance of the final model;

[0049] Step S56: In addition, in order to evaluate the classification performance of the model in the N-classification task, this method uses the Kappa coefficient as one of the criteria for judging the quality of the classification results. If the total number of samples is n, then Where p0 is the sum of the number of correctly classified samples divided by the total number of samples, that is, the overall classification accuracy. The number of real samples in each category is A1, A2, A3…, and the predicted number of samples in each category is B1, B2, B3….

[0050] The advantages of the present invention are:

[0051] By applying deep convolution and separable convolution, the parameters of the neural network to be trained are reduced. On this basis, the channel attention module and the deep attention module are integrated to extract the multi-scale spatiotemporal information of the EEG signals. Excellent classification performance is achieved at a low computational cost, and accurate recognition of various actions under motor imagery tasks is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A schematic diagram of the steps of a motor imagery action recognition system based on a lightweight attention network provided by the present invention.

[0053] Figure 2 This is a schematic diagram of data preprocessing involved in the present invention.

[0054] Figure 3 This is an overall schematic diagram of the neural network provided by the present invention.

[0055] Figure 4 Schematic diagram of the reference network EEGNet in the neural network provided by the present invention.

[0056] Figure 5 Schematic diagram of the channel attention module in the neural network provided by the present invention.

[0057] Figure 6 Schematic diagram of the deep attention module in the neural network provided by the present invention.

[0058] Figure 7 This is a diagram of the ablation experiment results performed by the present invention on a public dataset, where EEGNet_channel is a network with only the channel attention module added, EEGNet_depth is a network with only the depth attention module added, and EEGNet_attention is the final network with both the channel attention module and the depth attention module added. DETAILED DESCRIPTION

[0059] The implementation of the present invention is further described below in conjunction with the accompanying drawings.

[0060] In the first aspect, the present invention proposes a motor imagery action recognition system based on a lightweight attention network, see Figure 1 , including the following steps:

[0061] Step S1: Perform the following preprocessing steps on the original EEG signal:

[0062] Apply 4-38HZ bandpass filtering to the collected EEG signals to extract the frequency band of interest, normalize and scale the filtered data to the range of [-1,1], and whiten the original signal using the Euclidean alignment method;

[0063] Step S2: Implementation and model initialization of the baseline network EEGNet:

[0064] The benchmark network EEGNet includes three blocks, namely Block 1: two-dimensional convolution layer, deep convolution layer, Block 2: deep convolution layer, point convolution layer, Block 3: classification module containing linear layer; the present invention implements it in the pytorch environment and applies network initialization operations such as Xavier to the model;

[0065] Step S3: Construction of channel attention module:

[0066] Construct a channel attention module and embed it after the second block of EEGNet, followed by the deep attention module and the third block of EEGNet (classification module);

[0067] Step S4: Construction of deep attention module:

[0068] Build a deep attention module and embed it after the channel attention module, followed by the third block of EEGNet (classification module);

[0069] Step S5: Model training and classification system output:

[0070] The data processed in step S1 is divided into training set, validation set and test set, and the model is trained using the Adam optimizer and the cross entropy loss function, and the model parameters are saved. The saved optimal model is applied to the test set, and the classification accuracy, total accuracy and Kappa value of each category of the model are output;

[0071] For further information, see Figure 2 , the data preprocessing steps in step S1 are as follows:

[0072] Step S11: using a 4-38 Hz finite impulse response filter to extract frequency band data of interest in the motor imagery task;

[0073] Step S12: In order to facilitate the subsequent neural network training, the filtered data is scaled to the range of [-1,1] using maximum normalization: where x i,j represents the data value of the i-th row and j-th column after filtering, x i,j ' represents the data in the i-th row and j-th column after normalization, max(x 1,1 |,x1,2 |,…x i,j |) represents the maximum absolute value of the sample data after filtering;

[0074] Step S13: Use the Euclidean alignment method to whiten the data normalized in step S12: in is the data after whitening, represents the average covariance of the normalized data, x i represents the i-th data sample, Represents x i The transpose of , N is the number of samples;

[0075] For further information, see Figure 4 , the implementation and model initialization steps of step S2 benchmark network EEGNet are as follows:

[0076] Step S21: The EEGNet network is divided into three blocks. In the first block, two convolutional layers are executed sequentially. First, F1 temporal filters are applied, and then the deep convolutional layer is sent to learn spatial filters. Since the deep convolutional layer does not use all feature maps, the trainable parameters are reduced to a certain extent. In the second block, deep convolution and F2 point-by-point convolutions with kernel size (1,1) are used to further reduce model parameters and integrate information within and between feature maps. In the third block, the obtained features are sent to the linear layer and the N-type Softmax function. Output the final result;

[0077] Step S22: The model applies Xavier initialization to the convolutional layer and the fully connected layer. By setting the initial weights within a reasonable range, it can ensure that the activation values ​​at the beginning of the neural network model training remain at an appropriate scale. If the convolutional layer and the linear layer contain bias terms, their initial values ​​will be set to zero, which helps the neural network model maintain a small output value at the beginning of training.

[0078] Step S23: Set the input channel and output channel of the two-dimensional convolution layer in Block 1 to 1 and 8 respectively, where the convolution kernel size is (1,64), the step size is 1, and the padding size is 32, and then normalize the convolution result and output it;

[0079] Step S24: Set the input and output channels of the deep convolution layer in Block 1 to 8 and 16 respectively, where the convolution kernel size is (number of input data channels, 1) and the number of groups is 8. After that, normalize the output and use the ELU activation function: (where x is the input data) and an average pooling layer of size (1,4) with a dropout rate of 0.5;

[0080] Step S25: Set the input and output channels of the deep convolution layer in Block 2 to 16, the convolution kernel size to (1, 16), the step size to 1, the number of groups to 16, and the padding size to 8;

[0081] Step S26: Set the input and output channels of the point-wise convolution in Block 2 to 16, with the convolution kernel size of (1,1) and the step size of 1. After that, normalize the output and use the ELU activation function: (where x is the input data) and an average pooling layer of size (1,8) with a dropout rate of 0.5;

[0082] Step S27: Send the result obtained in Block 2 to the classification module of Block 3 to obtain the final classification result;

[0083] For further information, see Figure 5 ,The steps of constructing the channel attention module in step S3 are as follows:

[0084] Step S31: The time series of each channel in the EEG data may contain important information, and the channel attention module can capture the main trends in the time series. This module uses an adaptive global maximum pooling layer and an average pooling layer to extract the maximum and average values ​​of each channel from the input data, and then concatenates these statistics into a feature vector. The attention weight of each channel is then calculated through a sequence of linear layers. These weights are then multiplied with the original input data to generate weighted data as the final output of this module;

[0085] Step S32: using adaptive average pooling and adaptive maximum pooling of size (1, 1) for the data of the input channel attention module respectively;

[0086] Step S33: remove the last two dimensions of the two types of pooling results through the squeeze function;

[0087] Step S34: concatenate the two types of results obtained in step S33 along the channel dimension, and expand the size of the channel dimension to twice the size of the module input data;

[0088] Step S35: Send the result obtained in S34 to the linear layer without bias, convert the number of channels into the number of module input data channels, and then pass the Relu activation function: f(x)=max(0,x);

[0089] Step S36: Send the result obtained in S35 to the linear layer without bias, and its input and output are both the number of module input data channels;

[0090] Step S37: Send the result obtained in S36 to the Sigmoid activation function: Get the attention weight of the channel attention module:

[0091] Step S38: Reshape the attention weights obtained in S37 to match the shape of the module input data, and apply them to the module input data through element-by-element multiplication to achieve weighted adjustment of each channel.

[0092] For further information, see Figure 6 ,The steps of building the deep attention module in step S4 are as follows:

[0093] Step S41: For the deep attention module, the adaptive average pooling layer is first used to perform global pooling operations on the spatial dimension of the module input data while maintaining the integrity of the temporal dimension. The convolution layer then further extracts the depth information from the pooled result. The adjustable convolution kernel slides on the depth dimension, capturing the correlation between features of different dimensions through local cross-depth interactions, and filtering out information related to the decoding task. The module finally uses the Softmax function The result after convolution processing is probabilistically transformed in the last dimension, and after generating the deep attention weight, it is reshaped into the shape of the module input data using the interpolation function, and multiplied with the module input data and parameter b (parameter b value is 16) to generate weighted data as the final output of this module;

[0094] Step S42: using adaptive average pooling of size (1, a) on the data input to the deep attention module, where parameter a is an adjustable parameter;

[0095] Step S43: Send the result obtained in S42 to a two-dimensional convolution layer with input and output channels of 16 and 1 respectively, where the convolution kernel size is (7, 1), the step size is 1, and the padding size is 3;

[0096] Step S44: Send the result obtained in S43 to The function normalizes the convolution output;

[0097] Step S45: interpolate the result obtained in S44 to restore it to the module input data shape, and obtain the attention weight of the deep attention module;

[0098] Step S46: multiply the deep attention weight obtained in S45 by the module input data element by element, and introduce parameter b (parameter b value is 16) in the multiplication process to scale the module output to prevent the weight from being too small and thus affecting the module weighting process;

[0099] For further information, see Figure 7,Table 1,Step S5,Model training and classification system output steps are as follows:

[0100] Step S51: In order to better train and verify the data, this method divides the data obtained after step S1 into a training set, a verification set, and a test set. At the same time, the training set adopts a ten-fold cross-validation method. Before the model training starts, the parameters are sent to the Adam optimizer to facilitate adaptive adjustment of the learning rate. When the data is sent to the neural network training, the cross entropy loss function can be obtained: Where c is the total number of categories, y i is the true i-th class label, is the model’s predicted probability for the i-th class;

[0101] Step S52: Split the preprocessed data into a training set, a validation set, and a test set. Use the training set to train the model, use the validation set to monitor the model training process, and use the test set to output the system classification results;

[0102] Step S53: The system uses the Adam optimizer and the cross entropy loss function: At the same time, a ten-fold cross-validation method is used during training to evaluate the stability and robustness of the model;

[0103] Step S54: when the model is subjected to ten-fold cross validation, if the accuracy of a certain validation set is greater than the highest validation set accuracy currently recorded, the current neural network model parameters are saved as the optimal parameters and the highest validation set accuracy is recorded as the current value;

[0104] Step S55: Apply the optimal model obtained in step S54 to the test set segmented in step S52. At the same time, in order to compare the recognition difficulty and overall performance of each category in N categories, this system takes the accuracy rate of each category and the average accuracy rate as test criteria to test the performance of the final model;

[0105] Step S56: In addition, in order to evaluate the classification performance of the model in the N-classification task, this method uses the Kappa coefficient as one of the criteria for judging the quality of the classification results. If the total number of samples is n, then Where p0 is the sum of the number of correctly classified samples divided by the total number of samples, that is, the overall classification accuracy. The number of real samples in each category is A1, A2, A3…, and the predicted number of samples in each category is B1, B2, B3….

[0106] Table 1 is a detailed comparison table of the results of the present invention on the public data set with the benchmark network, other networks and ablation experiment networks.

[0107]

Claims

1. A motor imagery action recognition system based on a lightweight attention network, comprising the following steps: Step S1: Perform the following preprocessing steps on the original EEG signal: Apply 4-38HZ bandpass filtering to the collected EEG signals to extract the frequency band of interest, normalize and scale the filtered data to the range of [-1,1], and whiten the original signal using the Euclidean alignment method; Step S2: Implementation and model initialization of the baseline network EEGNet: The benchmark network EEGNet includes: Block1, Block2, Block3 in sequence; Block1 includes: two-dimensional convolution layer, deep convolution layer; Block2 includes: deep convolution layer, point convolution layer; Block3 includes: classification module with linear layer; The benchmark network EEGNet is implemented in the pytorch environment, and the Xavier network initialization operation is applied to the model; Step S3: Construction of channel attention module: Construct a channel attention module and embed it between Block2 and Block3 of EEGNet; Step S4: Construction of deep attention module: Build a deep attention module and embed it between the channel attention module and Block3; Step S5: Model training and classification system output: The data processed in step S1 is divided into training set, validation set and test set. The Adam optimizer is used to train the model with the cross entropy loss function, and the model parameters are saved. The saved optimal model is applied to the test set, and the classification accuracy of each category, the total accuracy and the Kappa value of the model are output.

2. A motor imagery action recognition system based on a lightweight attention network as claimed in claim 1, characterized in that: The data preprocessing steps in step S1 are as follows: Step S11: using a 4-38 Hz finite impulse response filter to extract frequency band data of interest in the motor imagery task; Step S12: In order to facilitate the subsequent neural network training, the filtered data is scaled to the range of [-1,1] using maximum normalization: where x i,j represents the data value of the i-th row and j-th column after filtering, x i,j ' represents the data in the i-th row and j-th column after normalization, max(x 1,1 |,x 1,2 |,…x i,j |) represents the maximum absolute value of the sample data after filtering; Step S13: Use the Euclidean alignment method to whiten the data normalized in step S12: where x' i is the data after whitening, represents the average covariance of the normalized data, x i represents the i-th data sample, Represents x i is the transpose of , and N is the number of samples.

3. A motor imagery action recognition system based on a lightweight attention network as claimed in claim 1, characterized in that: The implementation and model initialization steps of the reference network EEGNet in step S2 are as follows: Step S21: The EEGNet network is divided into three blocks. In Block1, two deep convolutional layers are executed sequentially. First, F1 temporal filters are used, and then they are sent to the deep convolutional layer to learn spatial filters. In Block2, deep convolution and F2 point-by-point convolutions with kernel size (1,1) are used. In Block3, the obtained features are sent to the linear layer and the N-type Softmax function. Output the final result; Step S22: The model applies Xavier initialization to the deep convolutional layer and the linear layer; if the convolutional layer and the linear layer contain bias terms, then their initial values ​​are set to zero, which helps the neural network model maintain a small output value in the early stage of training; Step S23: Set the input channel and output channel of the two-dimensional convolution layer in Block 1 to 1 and 8 respectively, where the convolution kernel size is (1,64), the step size is 1, and the padding size is 32, and then normalize the convolution result and output it; Step S24: Set the input channel and output channel of the deep convolution layer in Block 1 to 8 and 16 respectively, where the convolution kernel size is (number of input data channels, 1) and the number of groups is 8; after that, normalize the output and use the ELU activation function: Where x is the input data; and the average pooling layer of size (1,4) with a random dropout rate of 0.5; Step S25: Set the input and output channels of the deep convolution layer in Block 2 to 16, the convolution kernel size to (1, 16), the step size to 1, the number of groups to 16, and the padding size to 8; Step S26: Set the input and output channels of the point-wise convolution in Block 2 to 16, with the convolution kernel size of (1,1) and the step size of 1. After that, normalize the output and use the ELU activation function: (where x is the input data) and an average pooling layer of size (1,8) with a dropout rate of 0.5; Step S27: Send the result obtained in Block 2 to the classification module of Block 3 to obtain the final classification result.

4. A motor imagery action recognition system based on a lightweight attention network as claimed in claim 1, characterized in that: The steps for constructing the channel attention module in step S3 are as follows: Step S31: The channel attention module uses an adaptive global maximum pooling layer and an average pooling layer to extract the maximum and average values ​​of each channel from the input data, and then concatenates these statistics into a feature vector; then, the attention weight of each channel is calculated through a sequence of linear layers; These weights are then multiplied with the original input data to generate weighted data as the final output of this module; Step S32: using adaptive average pooling and adaptive maximum pooling of size (1, 1) for the data of the input channel attention module respectively; Step S33: remove the last two dimensions of the two types of pooling results through the squeeze function; Step S34: concatenate the two types of results obtained in step S33 along the channel dimension, and expand the size of the channel dimension to twice the size of the module input data; Step S35: Send the result obtained in S34 to the linear layer without bias, convert the number of channels into the number of module input data channels, and then pass the Relu activation function: f(x)=max(0,x); Step S36: Send the result obtained in S35 to the linear layer without bias, and its input and output are both the number of module input data channels; Step S37: Send the result obtained in S36 to the Sigmoid activation function: Get the attention weight of the channel attention module: Step S38: Reshape the attention weights obtained in S37 to match the shape of the module input data, and apply them to the module input data through element-by-element multiplication to achieve weighted adjustment of each channel.

5. The motor imagery action recognition system based on lightweight attention network as claimed in claim 1, characterized in that: The steps for constructing the deep attention module in step S4 are as follows: Step S41: For the deep attention module, first use the adaptive average pooling layer to perform global pooling operations on the spatial dimension of the module input data while maintaining the integrity of the time dimension; then the convolution layer further extracts the depth information of the pooled result; the adjustable convolution kernel slides on the depth dimension, captures the correlation between features of different dimensions through local cross-depth interaction, and filters out information related to the decoding task; finally, the module uses the Softmax function to perform a probabilistic transformation on the result of the convolution processing on the last dimension, generates the deep attention weight, and then uses the interpolation function to reshape it into the shape of the module input data, and multiplies it with the module input data and parameter b (parameter b value is 16), so as to generate weighted data as the final output of this module; Step S42: using adaptive average pooling of size (1, a) on the data input to the deep attention module, where parameter a is an adjustable parameter; Step S43: Send the result obtained in S42 to a two-dimensional convolution layer with input and output channels of 16 and 1 respectively, where the convolution kernel size is (7, 1), the step size is 1, and the padding size is 3; Step S44: Send the result of S43 to Softmax The function normalizes the convolution output; Step S45: interpolate the result obtained in S44 to restore it to the module input data shape, and obtain the attention weight of the deep attention module; Step S46: Multiply the deep attention weight obtained in S45 by the module input data element by element, and introduce parameter b (parameter b value is 16) during the multiplication process to scale the module output to prevent the weight from being too small and affecting the module weighting process.

6. A motor imagery action recognition system based on a lightweight attention network as claimed in claim 1, characterized in that: The steps of model training and classification system output in step S5 are as follows: Step S51: In order to better train and verify the data, the data preprocessed in step S1 is divided into a training set, a validation set and a test set. At the same time, the training set adopts a ten-fold cross-validation method; before the model training starts, the parameters are sent to the Adam optimizer to facilitate adaptive adjustment of the learning rate. When the data is sent to the neural network training, the cross entropy loss function can be obtained: Where c is the total number of categories, y i is the true i-th class label, is the model’s predicted probability for the i-th class; Step S52: Split the preprocessed data into a training set, a validation set, and a test set. Use the training set to train the model, use the validation set to monitor the model training process, and use the test set to output the system classification results; Step S53: The system uses the Adam optimizer and the cross entropy loss function: At the same time, a ten-fold cross-validation method is used during training to evaluate the stability and robustness of the model; Step S54: when the model is subjected to ten-fold cross validation, if the accuracy of a certain validation set is greater than the highest validation set accuracy currently recorded, the current neural network model parameters are saved as the optimal parameters and the highest validation set accuracy is recorded as the current value; Step S55: Apply the optimal model obtained in step S54 to the test set segmented in step S52. At the same time, in order to compare the recognition difficulty and overall performance of each category in N categories, this system takes the accuracy rate of each category and the average accuracy rate as test criteria to test the performance of the final model; Step S56: In addition, in order to evaluate the classification performance of the model in the N-classification task, this method uses the Kappa coefficient as one of the criteria for judging the quality of the classification results. If the total number of samples is n, then Where p0 is the sum of the number of correctly classified samples divided by the total number of samples, that is, the overall classification accuracy. The number of real samples in each category is A1, A2, A3…, and the predicted number of samples in each category is B1, B2, B3….

Images