A motor imagery action recognition system based on a lightweight attention network

Abstract

The application provides a motor imagery action recognition system based on a lightweight attention network, and belongs to the field of biomedical engineering.The system is based on an EEGNet network, two unique attention modules, a channel attention module and a deep attention module, are integrated, features are effectively integrated from multiple dimensions, more discriminative features are extracted, and the performance in a motor imagery classification task is significantly improved.The application adopts a deep learning method, avoids a manual feature extraction step in a traditional method, and achieves satisfactory classification effects while keeping a low calculation cost.The system has important application value and potential academic significance in the fields of neural rehabilitation and human-computer interaction.

Description

Technical Field

[0001] The present invention belongs to the field of biomedical engineering, and particularly relates to a motor imagery action recognition system based on a lightweight attention network. Background Art

[0002] Brain-computer interfaces (BCIs) enable information exchange between the brain and external devices by extracting EEG signals and converting them into understandable control commands. This approach holds significant promise for improving the quality of life for people with disabilities and advancing the development of human-computer interaction. Motor imagery is a key paradigm driving BCIs, and robust feature extraction algorithms play a key role in improving the performance and application of motor imagery BCI systems. However, due to the high-dimensional, variable spatiotemporal characteristics and complex topological distribution of EEG signals, researchers typically require highly specialized data processing techniques to extract classification features closely associated with specific movements. The subsequent feature selection process relies heavily on the researcher's expertise and judgment, lacking objectivity and requiring significant labor and time. In recent years, deep neural networks have gained widespread application due to their end-to-end nature and excellent automatic feature extraction performance. This significantly reduces repetitive work for researchers, allowing them to focus on mechanisms and models. However, it is important to note that due to the non-stationary and nonlinear nature of EEG signals, traditional neural network methods may perform poorly in motor imagery action recognition. In addition, existing studies generally believe that the internal structure of EEG signals is relatively simple, and that structures that are too deep or too complex will instead reduce their classification performance. Based on this, the present invention proposes a motor imagery action recognition system based on a lightweight attention network. The system cleverly integrates the channel attention module and the depth attention module into the forward propagation process of the EEGNet neural network, and then uses deep convolution (DepthwiseConv2D), separable convolution (SeparableConv2D) and pointwise convolution (PointwiseConv2D) to integrate the high-dimensional spatiotemporal information of EEG data with efficient computing performance, while significantly improving the accuracy and efficiency of the motor imagery classification system. Compared with traditional methods, this system uses the attention mechanism to focus the recognition focus precisely on information that is beneficial to the system classification, overcoming challenges such as low signal-to-noise ratio of EEG signals and difficulty in information integration; at the same time, in response to problems such as high computational cost and high model complexity of deep neural networks, the system greatly reduces the computational cost with a small number of computational parameters and extremely fast computing speed. While improving the performance of motor imagery action recognition, it can provide users with a more accurate and smooth interactive experience, which is of great significance for promoting the application of BCI technology in sports rehabilitation, neuroinformatics research and human-computer interaction. Summary of the Invention

[0003] To cope with the problems of low signal-to-noise ratio of electroencephalogram signals, difficulty in information integration, high operation cost of deep neural network and high model complexity, the application proposes a motor imagery action recognition system based on a lightweight attention network.

[0004] The application proposes a motor imagery action recognition system based on a lightweight attention network, comprising the following steps:

[0005] Step S1: The original electroencephalogram signal is preprocessed as follows:

[0006] The collected electroencephalogram signal is subjected to band-pass filtering of 4-38 Hz to extract the frequency band of interest, the filtered data is normalized and scaled to the range of [-1, 1], and the original signal is subjected to whitening processing using the Euclidean alignment method.

[0007] Step S2: Implementation and model initialization of the reference network EEGNet:

[0008] The reference network EEGNet sequentially includes Block1, Block2 and Block3; Block1 includes a two-dimensional convolution layer and a deep convolution layer; Block2 includes a deep convolution layer and a point convolution layer; Block3 includes a classification module containing a linear layer; the reference network EEGNet is implemented in the pytorch environment, and the model is subjected to the Xavier network initialization operation;

[0009] Step S3: Construction of the channel attention module:

[0010] The channel attention module is constructed and embedded between Block2 and Block3 of the EEGNet.

[0011] Step S4: Construction of the depth attention module:

[0012] The depth attention module is constructed and embedded between the channel attention module and Block3.

[0013] Step S5: Model training and classification system output:

[0014] The data processed in step S1 is divided to obtain a training set, a validation set and a test set, the model is trained using the Adam optimizer and the cross-entropy loss function, and the model parameters are saved; the saved optimal model is applied to the test set, and the classification accuracy, total accuracy and Kappa value of each class of the model are output.

[0015] Further, the data preprocessing step S1 is as follows:

[0016] Step S11: using a 4-38 Hz finite impulse response filter to extract frequency band data of interest in the motor imagery task;

[0017] Step S12: To facilitate subsequent neural network training, the filtered data is scaled to the range of [-1, 1] using maximum normalization: where x i,j represents the data value of row i and column j after filtering, x i,j ' represents the data in row i and column j after normalization, max(x 1,1 |,x 1,2 |,…x i,j |) represents the maximum absolute value of the sample data after filtering;

[0018] Step S13: Use the Euclidean alignment method to whiten the data normalized in step S12: in is the data after whitening, represents the average covariance of the normalized data, x i represents the i-th data sample, Represents x i is the transpose of , and N is the number of samples.

[0019] Furthermore, the implementation and model initialization steps of the reference network EEGNet in step S2 are as follows:

[0020] Step S21: The EEGNet network is divided into three blocks. In Block 1, two deep convolutional layers are executed sequentially. First, F1 temporal filters are applied, and then the deep convolutional layer is used to learn spatial filters. Since the deep convolutional layer does not use all feature maps, the trainable parameters are reduced to a certain extent. In Block 2, deep convolution and F2 point-by-point convolutions with kernel size (1,1) are used to further reduce the model parameters and integrate the information within and between feature maps. In Block 3, the obtained features are sent to the linear layer and the N-type Softmax function. Output the final result;

[0021] Step S22: The model applies Xavier initialization to the deep convolutional and linear layers. By setting the initial weights within a reasonable range, it ensures that the activation values ​​at the beginning of neural network model training are maintained at an appropriate scale. If the convolutional and linear layers contain bias terms, then their initial values ​​are set to zero to help the neural network model maintain a small output value in the early stages of training.

[0022] Step S23: Set the input and output channels of the two-dimensional convolution layer in Block1 to 1 and 8 respectively, where the convolution kernel size is (1, 64), the step size is 1, and the padding size is 32. Then normalize the convolution result and output;

[0023] Step S24: Set the input and output channels of the deep convolution layer in Block1 to 8 and 16 respectively, where the convolution kernel size is (input data channel number, 1), and the group number is 8. Then normalize the output and use the ELU activation function: where x is the input data, and the average pooling layer with a size of (1, 4) has a random inactivation rate of 0.5;

[0024] Step S25: Set the input and output channels of the deep convolution layer in Block2 to 16, where the convolution kernel size is (1, 16), the step size is 1, the group number is 16, and the padding size is 8;

[0025] Step S26: Set the input and output channels of the pointwise convolution in Block2 to 16, where the convolution kernel size is (1, 1) and the step size is 1. Then normalize the output and use the ELU activation function: where x is the input data, and the average pooling layer with a size of (1, 8) has a random inactivation rate of 0.5;

[0026] Step S27: Send the result obtained in Block2 to the Block3 classification module to obtain the final classification result.

[0027] Further, the construction steps of the channel attention module in step S3 are as follows:

[0028] Step S31: The time series of each channel in the electroencephalogram data may contain important information, and the channel attention module can capture the main trend in the time series. The channel attention module uses adaptive global max pooling and average pooling layers to extract the maximum and average values of each channel from the input data, and then concatenates these statistics into a feature vector. Then, a linear layer sequence is used to calculate the attention weights of each channel. These weights are then multiplied with the original input data to generate weighted data as the final output of this module;

[0029] Step S32: Use adaptive average pooling and adaptive max pooling with a size of (1, 1) on the data input to the channel attention module;

[0030] Step S33: Remove the last two dimensions of the two types of pooling results by using the squeeze function;

[0031] Step S34: concatenate the two types of results obtained in step S33 along the channel dimension, and expand the size of the channel dimension to twice the size of the module input data;

[0032] Step S35: Send the result obtained in S34 to the unbiased linear layer, convert the number of channels to the number of module input data channels, and then pass it through the ReLU activation function: f(x) = max(0,x);

[0033] Step S36: Send the result obtained in S35 to the linear layer without bias, and its input and output are both the number of module input data channels;

[0034] Step S37: Send the result obtained in S36 to the Sigmoid activation function: Get the attention weight of the channel attention module:

[0035] Step S38: Reshape the attention weights obtained in S37 to match the shape of the module input data, and apply them to the module input data through element-by-element multiplication to achieve weighted adjustment of each channel.

[0036] Furthermore, the steps of constructing the deep attention module in step S4 are as follows:

[0037] Step S41: For the deep attention module, the adaptive average pooling layer is first used to perform a global pooling operation on the spatial dimension of the module input data while maintaining the integrity of the time dimension; the convolution layer then further extracts the depth information of the pooled result; the adjustable convolution kernel slides on the depth dimension, captures the correlation between features of different dimensions through local cross-depth interaction, and filters out information related to the decoding task; the module finally uses the Softmax function to perform a probabilistic transformation on the last dimension of the convolution result, generates the deep attention weight, and then uses the interpolation function to reshape it into the shape of the module input data, and multiplies it with the module input data and the parameter b (the parameter b value is 16), thereby generating the weighted data as the final output of this module;

[0038] Step S42: Apply adaptive average pooling of size (1, a) to the data input to the deep attention module, where parameter a is an adjustable parameter;

[0039] Step S43: Send the result obtained in S42 to a two-dimensional convolutional layer with 16 input and 1 output channels respectively, where the convolution kernel size is (7, 1), the stride is 1, and the padding size is 3;

[0040] Step S44: Send the result obtained in S43 to The function normalizes the convolution output;

[0041] Step S45: the results obtained in S44 are interpolated to restore the module input data shape to obtain the attention weight of the depth attention module;

[0042] Step S46: the depth attention weight obtained in S45 is multiplied element by element with the module input data, and a parameter b (the value of parameter b is 16) is introduced in the multiplication process to scale the module output, so as to prevent the weight from being too small and affecting the module weighting process.

[0043] Further, the steps of the model training and the classification system output are as follows:

[0044] Step S51: in order to better train and verify the data, the data preprocessed in step S1 is divided into a training set, a verification set and a test set, and the training set adopts a ten-fold cross-validation method; before the model training starts, the parameters are sent to the Adam optimizer for adaptive adjustment of the learning rate, and after the data is sent to the neural network training, the cross-entropy loss function can be obtained: Where c is the total number of categories, y i is the true i-th class label, is the prediction probability of the model for the i-th class.

[0045] Step S52: the preprocessed data is divided into a training set, a verification set and a test set. The training set is used to train the model, the verification set is used to monitor the model training process, and the test set is used to output the system classification result;

[0046] Step S53: the system adopts the Adam optimizer and the cross-entropy loss function: At the same time, the ten-fold cross-validation method is used to evaluate the stability and robustness of the model during training;

[0047] Step S54: when the model is subjected to ten-fold cross-validation, if the accuracy of a certain fold verification set is greater than the highest verification set accuracy recorded at present, the current neural network model parameters are saved as the optimal parameters and the highest verification set accuracy is recorded as the current value;

[0048] Step S55: the optimal model obtained in step S54 is applied to the test set segmented in step S52, and in order to compare the recognition difficulty of each category and the overall performance in N categories, the system takes the category accuracy and the average accuracy as the test standard to test the performance of the final model;

[0049] Step S56: in addition, in order to evaluate the classification performance of the model in the N classification task, the Kappa coefficient is taken as one of the standards for judging the pros and cons of the classification result. If the total number of samples is n, then Where p0 is the sum of the number of correctly classified samples divided by the total number of samples, i.e. the overall classification accuracy, Wherein the number of each type of real sample is A1, A2, A3…, and the number of each type of predicted sample is B1, B2, B3…

[0050] The advantages of the present application are:

[0051] By applying deep convolution and separable convolution, the parameters to be trained of the neural network are reduced, and on this basis, the channel attention module and the deep attention module are integrated to extract the multi-scale spatial and temporal information of the electroencephalogram signal, so that excellent classification performance is achieved at a low operation cost, and accurate recognition of each action under the motor imagination task is realized. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 A step schematic diagram of a motor imagination action recognition system based on a lightweight attention network is provided.

[0053] Figure 2 A data preprocessing schematic diagram related to the present application.

[0054] Figure 3 A neural network overall schematic diagram is provided.

[0055] Figure 4 A reference network EEGNet schematic diagram in the neural network is provided.

[0056] Figure 5 A channel attention module schematic diagram in the neural network is provided.

[0057] Figure 6 A deep attention module schematic diagram in the neural network is provided.

[0058] Figure 7 An ablation experiment result graph on a public data set is provided, wherein EEGNet_channel is a network only adding a channel attention module, EEGNet_depth is a network only adding a deep attention module, and EEGNet_attention is the final network adding a channel attention module and a deep attention module. DETAILED DESCRIPTION

[0059] The embodiments of the present application will be further described below in combination with the drawings.

[0060] In a first aspect, the present application proposes a motor imagination action recognition system based on a lightweight attention network, please refer to Figure 1 , comprising the following steps:

[0061] Step S1: the original electroencephalogram signal is preprocessed as follows:

[0062] The collected electroencephalogram signal is applied with a band-pass filter of 4-38HZ to extract the interested frequency band, the filtered data is normalized and scaled to the range of [-1, 1], and the original signal is whitened by using the Euclidean alignment method;

[0063] Step S2: implementation of the reference network EEGNet and model initialization:

[0064] The reference network EEGNet includes three blocks, namely Block1: two-dimensional convolution layer, deep convolution layer, Block2: deep convolution layer, point convolution layer, and Block3: classification module containing a linear layer; the present application is implemented in the pytorch environment, and the model is applied with the Xavier network initialization operation;

[0065] Step S3: construction of the channel attention module:

[0066] The channel attention module is constructed and embedded into the second Block of the EEGNet, followed by the deep attention module and the third Block (classification module) of the EEGNet;

[0067] Step S4: construction of the deep attention module:

[0068] The deep attention module is constructed and embedded into the channel attention module, followed by the third Block (classification module) of the EEGNet;

[0069] Step S5: model training and classification system output:

[0070] The data processed in step S1 is divided to obtain a training set, a validation set and a test set, the model is trained using the Adam optimizer and the cross-entropy loss function, and the model parameters are saved. The saved optimal model is applied to the test set, and the classification accuracy of each class of the model, the total accuracy and the Kappa value are output;

[0071] Further, please refer to Figure 2 , the step S1 data preprocessing step is as follows:

[0072] Step S11: the interested frequency band data of the motor imagery task is extracted by using a 4-38HZ finite impulse response filter;

[0073] Step S12: in order to facilitate the subsequent training of the neural network, the filtered data is scaled to the range of [-1, 1] by using maximum normalization: Where x i,j represents the data value of the i-th row and the j-th column after filtering, x i,j ' represents the i-th row and the j-th column of the normalized data, max(x 1,1 |,x1,2 |,…x i,j | represents the maximum value of the absolute value of the filtered sample data;

[0074] Step S13: using the Euclidean alignment method to whiten the data normalized in step S12: wherein is the whitened data, represents the average covariance of the normalized data, x i represents the ith data sample, represents the transpose of x i , and N is the number of samples;

[0075] Further, please refer to Figure 4 , the implementation and model initialization steps of the reference network EEGNet are as follows:

[0076] Step S21: the EEGNet network is divided into three Blocks, and two convolutional layers are sequentially executed in the first Block, first F1 time filter, and then sent to the deep convolutional layer to learn the spatial filter. Since the deep convolutional layer does not use all the feature maps, it reduces the trainable parameters to a certain extent. In the second Block, a deep convolution and F2 point-by-point convolution with a convolution kernel size of (1,1) are used, which further reduces the model parameters and integrates the information within and between feature maps. In the third Block, the derived features are sent to the linear layer and the N-class Softmax function to output the final result;

[0077] Step S22: the model applies Xavier initialization in the convolutional layer and the fully connected layer. By setting the initial weight in a reasonable range, it can ensure that the activation value of the neural network model at the beginning of training is maintained at a proper scale. If the convolutional layer and the linear layer contain bias terms, their initial values will be set to zero, which helps the neural network model to maintain a small output value at the beginning of training.

[0078] Step S23: set the input channel and output channel of the two-dimensional convolutional layer in Block1 to 1 and 8 respectively, wherein the convolution kernel size is (1,64), the step size is 1, and the padding size is 32, and then output the normalized result of the convolution;

[0079] Step S24: set the input channel and output channel of the deep convolutional layer in Block1 to 8 and 16 respectively, wherein the convolution kernel size is (input data channel number, 1), and the number of groups is 8. After that, normalize the output and use the ELU activation function: (where x is the input data) and the average pooling layer with a size of (1,4) and a random inactivation rate of 0.5;

[0080] Step S25: Set the input and output channels of the depth convolution layer in Block2 to be 16, the convolution kernel size to be (1, 16), the step to be 1, the number of groups to be 16, and the padding size to be 8;

[0081] Step S26: Set the input and output channels of the dotwise convolution in Block2 to be 16, the convolution kernel size to be (1, 1), and the step to be 1. After that, normalize the output and use the ELU activation function: (where x is the input data) and an average pooling layer with a size of (1, 8) and a random inactivation rate of 0.5;

[0082] Step S27: Send the result obtained in Block2 to the Block3 classification module to obtain the final classification result;

[0083] Further, please refer to Figure 5 , the steps of constructing the channel attention module in step S3 are as follows:

[0084] Step S31: The time series of each channel in the electroencephalogram data may contain important information, and the channel attention module can capture the main trend in the time series. This module uses adaptive global max pooling and average pooling layers to extract the maximum value and average value of each channel from the input data, and then concatenates these statistics into a feature vector. Then a linear layer sequence is used to calculate the attention weight of each channel. These weights are then multiplied with the original input data to generate weighted data as the final output of this module;

[0085] Step S32: Use adaptive average pooling and adaptive max pooling with a size of (1, 1) on the data input into the channel attention module;

[0086] Step S33: Remove the last two dimensions of the two types of pooling results by using the squeeze function;

[0087] Step S34: Concatenate the two results obtained in step S33 along the channel dimension, and expand the channel dimension to twice the size of the module input data;

[0088] Step S35: Send the result obtained in S34 to a linear layer without bias, convert the channel number to the size of the module input data channel, and then pass it through the Relu activation function: f(x) = max(0, x);

[0089] Step S36: Send the result obtained in S35 to a linear layer without bias, and the input and output are both the channel number of the module input data;

[0090] Step S37: send the result of S36 into Sigmoid activation function: Get the attention weight of the channel attention module:

[0091] Step S38: reshape the attention weight of S37 to match the shape of the module input data, and apply it to the module input data through element-wise multiplication, so as to realize the weighted adjustment of each channel.

[0092] Further, please refer to Figure 6 , the steps of constructing the deep attention module in step S4 are as follows:

[0093] Step S41: for the deep attention module, first use the adaptive average pooling layer to perform global pooling operation on the spatial dimension of the module input data while keeping the time dimension intact. The subsequent convolution layer further extracts the deep information of the pooled result. The adjustable convolution kernel slides in the depth dimension, captures the correlation between different dimensional features through local cross-depth interaction, and filters out the information related to the decoding task. The module finally uses the Softmax function to perform probabilistic conversion on the result of convolution processing in the last dimension, generate deep attention weight, and then reshape it to the shape of the module input data using the interpolation function, and multiply it with the module input data and parameter b (the value of parameter b is 16), so as to generate the weighted data as the final output of this module;

[0094] Step S42: use adaptive average pooling with size (1, a) to process the data input into the deep attention module, where parameter a is an adjustable parameter;

[0095] Step S43: send the result of S42 into a two-dimensional convolution layer with input and output channels of 16 and 1 respectively, where the convolution kernel size is (7, 1), the step is 1, and the padding size is 3;

[0096] Step S44: send the result of S43 into function to normalize the convolution output;

[0097] Step S45: interpolate the result of S44 to restore the shape of the module input data to get the attention weight of the deep attention module;

[0098] Step S46: element-wise multiply the deep attention weight of S45 with the module input data, and introduce parameter b (the value of parameter b is 16) in the multiplication process to scale the module output, so as to prevent the weight from being too small and affecting the module weighting process;

[0099] Further, please refer to Figure 7, Table 1, step S5 model training and classification system output steps as follows:

[0100] Step S51: In order to better train and verify the data, the method divides the data obtained after step S1 into training set, validation set and test set, and the training set adopts ten-fold cross validation method. Before the model training starts, the parameters are sent to the Adam optimizer to adaptively adjust the learning rate. When the data is sent to the neural network training, the cross entropy loss function can be obtained: Where c is the total number of categories, y i is the real i-th class label, is the prediction probability of the i-th class of the model.

[0101] Step S52: The preprocessed data is divided into training set, validation set and test set. The training set is used to train the model, the validation set is used to monitor the model training process, and the test set is used to output the system classification result;

[0102] Step S53: The system uses Adam optimizer and cross entropy loss function: At the same time, ten-fold cross validation method is used to evaluate the stability and robustness of the model;

[0103] Step S54: When the model is ten-fold cross validated, if the accuracy of a certain fold validation set is greater than the highest validation set accuracy recorded so far, the current neural network model parameters are saved as the optimal parameters and the highest validation set accuracy is recorded as the current value;

[0104] Step S55: The optimal model obtained in step S54 is applied to the test set segmented in step S52. In order to compare the recognition difficulty of each category in N categories and the overall performance, the system takes the accuracy of each category and the average accuracy as the test standard to test the performance of the final model;

[0105] Step S56: In addition, in order to evaluate the classification performance of the model in N classification task, the method takes Kappa coefficient as one of the standards to judge the classification result. If the total number of samples is n, then Where p0 is the sum of the number of correctly classified samples divided by the total number of samples, i.e. the overall classification accuracy, Where the number of real samples of each category is A1, A2, A3…, and the number of predicted samples of each category is B1, B2, B3…

[0106] Table 1 is a detailed comparison table of the results of the present application with the benchmark network, other network and ablation experiment network on the public data set.

[0107]

Claims

1. A motor imagery action recognition system based on a lightweight attention network, comprising the following steps: Step S1: Perform the following preprocessing steps on the original EEG signal: Apply 4-38 Hz bandpass filtering to the collected EEG signals to extract the frequency band of interest, normalize the filtered data to the range of [-1, 1], and whiten the original signal using the Euclidean alignment method; Step S2: Implementation and model initialization of the baseline network EEGNet: The baseline network EEGNet consists of Block 1, Block 2, and Block 3. Block 1 includes a two-dimensional convolutional layer and a depthwise convolutional layer. Block 2 includes a depthwise convolutional layer and a pointwise convolutional layer. Block 3 includes a classification module with a linear layer. The baseline network EEGNet is implemented in the PyTorch environment, and the Xavier network initialization operation is applied to the model. Step S3: Construction of channel attention module: Construct a channel attention module and embed it between Block2 and Block3 of EEGNet; The construction steps of the channel attention module are as follows: Step S31: The channel attention module uses an adaptive global max pooling layer and an average pooling layer to extract the maximum and average values ​​of each channel from the input data, and then concatenates these statistics into a feature vector. It then calculates the attention weights of each channel through a sequence of linear layers. These weights are then multiplied with the original input data to generate the weighted data as the final output of this module. Step S32: Adaptive average pooling and adaptive maximum pooling of size (1, 1) are applied to the data of the input channel attention module respectively; Step S33: remove the last two dimensions of the two types of pooling results through the squeeze function; Step S34: concatenate the two types of results obtained in step S33 along the channel dimension, and expand the size of the channel dimension to twice the size of the module input data; Step S35: Send the result obtained in S34 to the unbiased linear layer to convert the number of channels into the module input data channel size, and then pass it through the ReLU activation function: f(x) = max(0,x), where x is the input data; Step S36: Send the result obtained in S35 to the linear layer without bias, and its input and output are both the number of module input data channels; Step S37: Send the result obtained in S36 to the Sigmoid activation function: Get the attention weight of the channel attention module: Step S38: reshape the attention weights obtained in S37 to match the shape of the module input data, and apply them to the module input data through element-by-element multiplication to achieve weighted adjustment of each channel; Step S4: Construction of deep attention module: Build a deep attention module and embed it between the channel attention module and Block3; The construction steps of the deep attention module are as follows: Step S41: For the deep attention module, the adaptive average pooling layer is first used to perform a global pooling operation on the spatial dimension of the module input data while maintaining the integrity of the time dimension; the convolution layer then further extracts the depth information of the pooled result; the adjustable convolution kernel slides on the depth dimension, capturing the correlation between features of different dimensions through local cross-depth interaction, and filtering out information related to the decoding task; the module finally uses the Softmax function to perform a probabilistic transformation on the last dimension of the convolution result, generates the deep attention weight, and then uses the interpolation function to reshape it into the shape of the module input data, and multiplies it with the module input data and the parameter b to generate the weighted data as the final output of this module, and the parameter b value is 16; Step S42: Apply adaptive average pooling of size (1, a) to the data input to the deep attention module, where parameter a is an adjustable parameter; Step S43: Send the result obtained in S42 to a two-dimensional convolutional layer with 16 input and 1 output channels respectively, where the convolution kernel size is (7, 1), the stride is 1, and the padding size is 3; Step S44: Send the result of S43 to Softmax, Indicates that the function normalizes the convolution output; Step S45: interpolate the result obtained in S44 to restore it to the module input data shape, and obtain the attention weight of the deep attention module; Step S46: Multiply the deep attention weight obtained in S45 by the module input data element by element, and introduce parameter b in the multiplication process to scale the module output to prevent the weight from being too small and thus affecting the module weighting process. The parameter b value is 16; Step S5: Model training and classification system output: The data processed in step S1 is divided into training set, validation set and test set. The Adam optimizer is used to train the model while adopting the cross entropy loss function, and the model parameters are saved. The saved optimal model is applied to the test set, and the classification accuracy of each category, the total accuracy and the Kappa value of the model are output.

2. The motor imagery action recognition system based on a lightweight attention network according to claim 1, wherein: The data preprocessing steps in step S1 are as follows: Step S11: using a 4-38 Hz finite impulse response filter to extract frequency band data of interest in the motor imagery task; Step S12: To facilitate subsequent neural network training, the filtered data is scaled to the range of [-1, 1] using maximum normalization: where x i,j represents the data value of row i and column j after filtering, x i,j ' represents the data in row i and column j after normalization, max(|x 1,1 |,|x 1,2 |,…|x i,j |) represents the maximum absolute value of the sample data after filtering; Step S13: Use the Euclidean alignment method to whiten the data normalized in step S12: where x' i is the data after whitening, represents the average covariance of the normalized data, x i represents the i-th data sample, Represents x i is the transpose of , and N is the number of samples.

3. The motor imagery action recognition system based on a lightweight attention network as claimed in claim 2, characterized in that: The implementation and model initialization steps of the reference network EEGNet in step S2 are as follows: Step S21: The EEGNet network is divided into three blocks. In Block 1, two deep convolutional layers are executed sequentially. First, F1 temporal filters are applied, and then the spatial filters are fed into the deep convolutional layer. In Block 2, deep convolution and F2 point-by-point convolution with kernel size (1,1) are used. In Block 3, the features obtained are fed into the linear layer and the N-type Softmax function. Output the final result; Step S22: The model applies Xavier initialization to the depthwise convolutional and linear layers; if the convolutional and linear layers contain bias terms, their initial values ​​are set to zero; Step S23: Set the input channel and output channel of the two-dimensional convolution layer in Block 1 to 1 and 8 respectively, with the convolution kernel size of (1, 64), the stride of 1, and the padding size of 32. Then, normalize the convolution result and output it. Step S24: Set the input and output channels of the deep convolution layer in Block 1 to 8 and 16 respectively, where the convolution kernel size is (number of input data channels, 1) and the number of groups is 8; then normalize the output and use the ELU activation function: Where x is the input data; and the average pooling layer of size (1,4) with a random dropout rate of 0.5; Step S25: Set the input and output channels of the depthwise convolution layer in Block 2 to 16, the convolution kernel size to (1, 16), the stride to 1, the number of groups to 16, and the padding size to 8; Step S26: Set the input and output channels of the point-wise convolution in Block 2 to 16, with a convolution kernel size of (1,1) and a stride of 1. After that, normalize the output and use the ELU activation function: Where x is the input data, and the average pooling layer is of size (1,8) with a random dropout rate of 0.5; Step S27: Send the result obtained in Block 2 to the classification module of Block 3 to obtain the final classification result.

4. The motor imagery action recognition system based on a lightweight attention network as claimed in claim 1, characterized in that: The steps of step S5 model training and classification system output are as follows: Step S51: In order to better train and verify the data, the data preprocessed in step S1 is divided into a training set, a validation set, and a test set. At the same time, the training set adopts a ten-fold cross-validation method; before the model training begins, the parameters are sent to the Adam optimizer to facilitate adaptive adjustment of the learning rate. When the data is sent to the neural network training, the cross entropy loss function can be obtained: Where c is the total number of categories, y i is the true i-th class label, is the model’s predicted probability for the i-th class; Step S52: Split the pre-processed data into a training set, a validation set, and a test set. Use the training set to train the model, use the validation set to monitor the model training process, and use the test set to output the system classification results. Step S53: The system uses the Adam optimizer and the cross entropy loss function: At the same time, a ten-fold cross-validation method is used during training to evaluate the stability and robustness of the model; Step S54: When the model is subjected to ten-fold cross validation, if the accuracy of a certain fold validation set is greater than the highest validation set accuracy currently recorded, the current neural network model parameters are saved as the optimal parameters and the highest validation set accuracy is recorded as the current value; Step S55: Apply the optimal model obtained in step S54 to the test set segmented in step S52. At the same time, in order to compare the recognition difficulty and overall performance of each category in N categories, this system takes the accuracy rate of each category and the average accuracy rate as test criteria to test the performance of the final model; Step S56: In addition, in order to evaluate the classification performance of the model in the N classification task, this method uses the Kappa coefficient as one of the criteria for judging the quality of the classification results; if the total number of samples is n, then Where p0 is the sum of the number of correctly classified samples divided by the total number of samples, which is the overall classification accuracy. The number of real samples in each category is A1, A2, A3…, and the predicted number of samples in each category is B1, B2, B3….

Images