Classification method and system for motor imagery electroencephalogram signals and storage medium

Abstract

The invention discloses a motor imagery electroencephalogram signal classification method and system and a storage medium, and belongs to the technical field of electroencephalogram signal analysis. The method comprises the following steps of: performing MODWT decomposition on a time domain signal of each channel in an electroencephalogram signal, splicing the electroencephalogram signal and an MODWT decomposition layer signal, and inputting a motor imagery signal classification network model after weighting in a depth dimension and a channel dimension; after the model is trained, the electroencephalogram signals can be identified and classified, and a motor imagery classification result is obtained. According to the method, the lightweight network model is adopted, the overfitting problem caused by insufficient training samples is reduced, and good motor imagery signal classification performance can be achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of electroencephalogram signal analysis, and relates to a classification method, system and storage medium for motor imagery electroencephalogram signals of a MODWT time-frequency fusion lightweight neural network. Background Art

[0002] Motor imagery is one of the classic paradigms of brain-computer interface (BCI). When the subject imagines a certain limb movement, the electroencephalogram (EEG) signal in a specific area of ​​the brain related to movement is activated. By using non-invasive electrode equipment to collect the EEG signals during motor imagery, the corresponding human movement intention can be extracted from it. Since motor imagery brain-computer interface has the advantages of no need for external stimulation and active control, it has received widespread attention in the fields of smart medical care, assisted control and rehabilitation treatment.

[0003] The EEG signals collected by motor imagery are easily disturbed by environmental noise, and have the characteristics of low signal-to-noise ratio, non-stationarity and high inter-subject variability, which greatly affect the signal classification accuracy of motor imagery tasks. Traditional motor imagery signal classification methods mainly consist of two parts: feature extraction and feature classification. Feature extraction mainly uses methods such as wavelet transform, independent component analysis (ICA) or common spatial pattern (CSP), and classification algorithms use methods such as linear discriminant analysis (LDA) and support vector machine (SVM). However, these traditional classification methods have limitations such as reliance on prior knowledge, manual feature selection, and too few training samples. The classification accuracy and generalization performance in motor imagery tasks are poor.

[0004] As deep learning has made great progress in fields such as vision and natural language processing, applying deep learning to motor imagery classification methods has become a current research hotspot. Compared with traditional methods, deep learning models can better help understand brain neural activity. Such methods can automatically extract high-dimensional spatiotemporal features from data. For example, models such as ConvNet, EEGNet, TCNet, graph networks, and residual networks have achieved many results in EEG signal classification accuracy and have good generalization performance. However, many current deep learning classification models have complex network structures, are limited by the number of training samples, are prone to overfitting, and have high computational complexity, which is not conducive to real-time and embedded motor imagery brain-computer interface systems. Summary of the invention

[0005] In order to solve the shortcomings of the prior art, the present invention proposes an EEG signal classification method and system based on MODWT time-frequency fusion lightweight neural network, which adopts a lightweight network model to reduce the overfitting problem caused by insufficient training samples and can achieve better motor imagery signal classification performance.

[0006] The technical solution adopted by the present invention is as follows:

[0007] A method for classifying motor imagery electroencephalogram signals comprises the following steps:

[0008] S1: Collect multi-channel raw EEG signals and perform preprocessing;

[0009] S2: for the preprocessed EEG signal obtained in step S1, extract the time domain signal of each channel and perform maximum overlap discrete wavelet transform (MODWT) decomposition respectively, and perform splicing and expansion in the depth dimension to obtain the MODWT decomposition layer signal;

[0010] S3: copy the EEG signal obtained in multiple layers of step S1, combine it with the MODWT decomposition layer signal obtained in step S2, and concatenate them in the depth dimension to form a high-dimensional input signal;

[0011] S4: In the depth dimension and channel dimension, the input signal obtained in step S3 is weighted using the attention mechanism;

[0012] S5: Construct a motor imagery signal classification network model, which includes two time domain convolution modules, one spatial convolution module, one channel convolution module, one fully connected layer and one linear classifier;

[0013] S6: Using the input signal obtained in step S3 to construct a labeled training set and a validation set, using the training set to train the motor imagery signal classification network model, and testing the trained network on the validation set;

[0014] S7: Obtain the EEG signal to be classified, process it according to steps S1-S3, and input it into the trained motor imagery signal classification network model for identification and classification to obtain the motor imagery classification result.

[0015] Furthermore, the step of preprocessing the EEG signal in step S1 includes:

[0016] First, a 5th-order Butterworth bandpass filter with a bandwidth of [4,40] Hz is used to bandpass filter each channel of the original EEG signal;

[0017] Then, the baseline of the EEG signal is removed, the average value of the signal is removed from the amplitude of each channel signal, and the signal amplitude is linearly normalized according to the peak value to linearly map the EEG signal to the [0,1] interval.

[0018] Furthermore, the formula for MODWT decomposition in step S2 is as follows:

[0019]

[0020] Where x(n) is the EEG signal, h(n) is the wavelet filter function, g(n) is the scale function, n is the time variable, and w j (k) and v j (k) represents the wavelet decomposition coefficient of the jth layer at time k, L j Indicates the length of the j-th layer signal.

[0021] Furthermore, the processing steps of the motor imagery signal classification network model in step S5 include:

[0022] 1) Input to the first time domain convolution module and process as follows:

[0023] The input EEG signal first passes through a 1×1 convolutional network to increase the dimension of the input signal depth;

[0024] Use two layers of separable convolutional networks to perform multi-layer convolution on the input signal, and add a batch normalization layer after each separable convolutional network;

[0025] Then the output data is regularized through the activation function GELU layer and Dropout layer;

[0026] Then use a 1×1 convolutional network to reduce the depth of the input signal;

[0027] 2) Input to the second time domain convolution module and process as follows:

[0028] Use a 1×1 convolutional network layer to reduce the depth dimension;

[0029] Use a layer of separable convolutional network to extract deep temporal features;

[0030] 3) Input to the spatial domain convolution module, which includes two convolution layers, each followed by a batch normalization layer, and finally an activation function GELU layer to extract spatial local features;

[0031] 4) Input to the channel convolution module, which includes a maximum pooling layer to compress spatiotemporal features and reduce the amount of calculation;

[0032] 5) Use fully connected layers to combine spatiotemporal features;

[0033] 6) Use a linear classifier to classify the features.

[0034] Furthermore, in step S6, the motor imagery signal classification network model uses a cross entropy loss function to minimize the classification difference during training.

[0035] Furthermore, the training process uses the Adam optimizer with a learning rate of 0.001.

[0036] A classification system for motor imagery EEG signals comprises a memory and a processor. The memory stores a computer program, and the processor implements the steps of the above classification method when executing the computer program.

[0037] A computer-readable storage medium stores a computer program, which implements the steps of the above classification method when executed.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] The present invention combines the MODWT wavelet decomposition method with a lightweight neural network, fully exploits the time-frequency characteristics in the EEG signal, and effectively integrates the time-frequency characteristics with the time-space domain characteristics by combining the wavelet decomposition signal with the time domain signal in the network input layer. At the same time, deep separable convolution is used in the network layer to reduce the computational complexity of the network model, and multi-layer time domain convolution is used to reduce the network overfitting problem, thereby achieving higher-performance motor imagery signal classification accuracy with fewer model parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a processing flow chart of the classification method of motor imagery EEG signals proposed by the present invention.

[0041] Figure 2 This is a structural diagram of the motor imagery signal classification network model proposed by the present invention. DETAILED DESCRIPTION

[0042] In order to make the various technical features and advantages or technical effects in the above technical solutions of the present invention more obvious and easy to understand, they are described in detail below with reference to the accompanying drawings.

[0043] The embodiment of the present invention specifically proposes a classification method for motor imagery signals based on MODWT time-frequency fusion lightweight neural network (abbreviated as MODWT-Net). The data processing flow of the method is as follows: Figure 1 As shown, the specific steps include:

[0044] S1: Preprocess the original EEG signals of multiple channels, including filtering and normalization. The data of different channels extracted from the EEG signals contain various types of noise. First, a 5th-order Butterworth filter with a bandwidth of [4,40] Hz is used to perform bandpass filtering on each channel of the original EEG signal, retaining only the information of the useful frequency band; then, the baseline error is removed, and the average value is subtracted from the amplitude of each channel signal to reduce the influence of the data difference value; finally, the data is normalized, and the EEG signal is linearly mapped to the [0,1] interval to speed up the network convergence. In addition, the preprocessed data is divided into a certain proportion to construct the training set and the validation set.

[0045] S2: Maximum Overlap Discrete Wavelet Transform (MODWT) decomposition. Aiming at the problem that the existing network model does not make sufficient use of frequency domain features, this method performs MODWT decomposition on the EEG signals of each channel after preprocessing, and merges the signals of each decomposition layer into the depth dimension, thereby extracting the time-frequency features of the EEG signals.

[0046] The formula for MODWT decomposition is as follows:

[0047]

[0048] Where x(n) is the EEG signal, h(n) is the wavelet filter function, g(n) is the scaling function, and w j and v j represents the wavelet decomposition coefficient of the jth layer;

[0049] The specific parameters use 5 layers of wavelet decomposition, and the wavelet filter function uses sym4 wavelet. Each channel signal is processed separately. It is assumed that the dimension of the preprocessed EEG signal two-dimensional data is {C, N}, where C is the number of EEG channels and N is the length of the EEG signal. The MODWT decomposition signals of each layer are spliced ​​in the depth dimension to obtain a {1,5,C,N}-dimensional signal.

[0050] S3: Time-frequency fusion. Since the time-frequency characteristics of some data are not obvious enough, they still rely on the time domain characteristics of the signal. The EEG signal after multiple layers of preprocessing is copied, and after the depth dimension is expanded, it is spliced ​​and merged with the MODWT decomposition signal to obtain a high-dimensional input signal, thereby effectively fusing the time-frequency feature information. Signal splicing For example, the {C, N}-dimensional EEG signal is copied 5 layers along the depth dimension to obtain a {1, 5, C, N}-dimensional signal, and then spliced ​​and combined with the wavelet decomposition layer signal {1, 5, C, N}-dimensional MODWT decomposition result along the depth dimension to obtain a {1, 10, C, N}-dimensional signal. Optionally, the number of wavelet decomposition layers and the number of EEG signal replication layers can be adjusted according to the specific task to achieve the optimal network model classification performance.

[0051] S4: Attention mechanism weighting. In view of the different roles played by different time-frequency components and channel components in the motor imagery signal classification task, the attention mechanism is used to weight the depth dimension and channel dimension, multiply them by a randomly initialized weight, and continuously update them during the network training process. For example, the depth and channel dimensions of the multidimensional signal {1,10,C,N} are respectively multiplied by the weights {α1,α2,…,α 10} and {β1,β2,…,β C}, initialized with a uniformly distributed random function and used as network parameters for training.

[0052] S5: Network model construction. Based on EEGNet, a lightweight neural network model is constructed using separable convolution, which includes two time domain convolutions, one spatial convolution and one fully connected layer. The network model is constructed based on Pytorch. The model structure is as follows: Figure 2 As shown in the figure, the input layer refers to the time-frequency fused EEG signal of input {1,10,C,N}, which mainly includes:

[0053] Time domain convolution module 1: A 1×1 convolutional network layer is first used for the input {1,10,C,N} signal to increase the depth dimension and output the signal dimension {1,25,C,N} to enrich the time-frequency fusion features of the depth dimension. Then, two consecutive convolutional layers are stacked to perform multiple convolutions on the input signal, which can better extract the time domain features and reduce the overfitting problem caused by a single convolution kernel that is too large. Both convolutions use depthwise separable convolutions with kernel sizes of (1,60) and (1,10) respectively. The group size is the depth dimension (for example, 20), the padding value is 0, the stride is set to 1, and each convolution layer is followed by a BatchNorm batch normalization layer. The last layer of time domain convolution module 1 is a GELU layer and a Dropout layer with a dropout rate of 0.3 to regularize the output data and reduce the overfitting problem in the training process. A 1×1 convolutional network is then used to reduce the depth of the input signal and the depth dimension of the output signal becomes 12.

[0054] Time domain convolution module 2: The upper layer output uses a 1×1 convolutional network layer to reduce the depth dimension; then a separable convolution layer is used to further extract deep time domain features. The convolution kernel and group size are both the depth dimension size after dimensionality reduction. The group size is 10, the Padding value is 0, the Stride is set to 1, and there is a BatchNorm batch normalization layer after both convolution layers.

[0055] Spatial convolution: The upper layer output is passed through a spatial convolution layer with a convolution kernel size of (C, 1), where C is the number of channels, the Padding value is 0, and the Stride is set to 1, so as to extract spatial local features;

[0056] Channel convolution: Use maximum pooling and discarding to compress spatiotemporal features and reduce the amount of calculation. The pooling rate is 25 and the discarding rate is 0.5 to improve the generalization ability of the model.

[0057] Fully connected layer: After flattening all features in the depth dimension, the input is stacked into the fully connected layer to fuse the time-frequency and space-time features;

[0058] Linear Classifier: Input to a linear classifier to classify the features.

[0059] S6: The MODWT-Net network model is trained using the training set and validation set data of the motor imagery EEG signal. The cross entropy loss function is used in the training process. The network uses the cross entropy loss function to minimize the classification difference. The function is as follows:

[0060]

[0061] Where C is the number of categories, N is the number of samples, and y ij Indicates whether the i-th sample belongs to the j-th category, p ij It represents the probability that the i-th sample is predicted to be of the j-th class.

[0062] The Adam optimizer was used to optimize the model parameters with a learning rate of 0.001 and an epoch of 300.

[0063] Experimental test:

[0064] To further illustrate the advantages of the method of the present invention, the MODWT-Net model proposed in the present invention is experimented on the BCI-Competition IV-2a dataset. The dataset contains motor imagery EEG signals of 9 subjects, each of whom participated in 4 motor imagery tasks, including the imaginary movement of the left hand, right hand, feet and tongue. Two sessions were recorded using a sampling rate of 250Hz and 22 channel electrodes. Each session contained 288 data trials. The data of the first session was used for training, and the data of the second session was used for testing.

[0065] Considering the large differences between subjects, the experiment classified the EEG signal data of 9 subjects separately. The training set and validation set of the data set were preprocessed and input into the network. The EEG signal dimension was (22, 1125), 22 was the number of channels, and 1125 was the data time length. The EEG signal was decomposed by MODWT using a 5-layer sysm4 wavelet, and expanded into a (1, 5, 22, 1125)-dimensional signal in the depth dimension. At the same time, the original signal was copied 5 layers in the depth dimension, and then spliced ​​with the wavelet decomposition result in the depth dimension to obtain the input layer signal dimension (1, 10, 22, 1125). In the time domain convolution module 1, a 1×1 convolution was used to increase the input signal depth dimension from 10 to 25, and then passed through two separable convolution layers and a GELU layer. In the time domain convolution module 2, a 1×1 convolution was used to reduce the input signal depth dimension from 25 to 12, and then a separable convolution layer was used. Then, channel features are extracted in spatial convolution, the output channel dimension is 1, and pooling is performed using the Maxpool layer with a pooling rate of 25. Finally, the output signal of depth dimension 12 is flattened and classified through a fully connected layer and a linear classifier. The training process uses the Adam optimizer, with a learning rate of 0.001, a batch size of 64, and an epoch of 300.

[0066] In order to evaluate the model performance, the average classification accuracy (Acc) and Kappa coefficient of 9 subjects were used as model evaluation indicators, and the method of the present invention was compared with the currently popular convolutional network models FBCSP, EEGNet, DeepConvNet, ShallowConvNet and Conformer models. The comparison results are shown in Table 1.

[0067] Table 1 Statistics of classification recognition rate and Kappa coefficient of the present invention and other comparative network models

[0068]

[0069] It can be seen from Table 1 that the average classification accuracy of the MODWT-Net proposed in the present invention is 78.7%, and the Kappa coefficient is 0.71, both of which are better than the comparison network model. In addition, the network parameter of MODWT-Net is only 4794, which is only higher than the EEGNet model (4080) and FBCNet (4180) in the above models, and far lower than other models, which proves the effectiveness and superiority of the lightweight neural network model proposed in the present invention.

[0070] Although the present invention has been disclosed as above by way of embodiments, it is not intended to limit the present invention. Appropriate modifications or equivalent substitutions of the technical solutions of the present invention made by ordinary technicians in the field should all be included in the protection scope of the present invention. The protection scope of the present invention shall be based on what is defined in the claims.

Claims

1. A method for classifying motor imagery EEG signals, characterized in that: The following steps are involved: S1: Collect multi-channel raw EEG signals and perform preprocessing; S2: for the preprocessed EEG signal obtained in step S1, extract the time domain signal of each channel and perform maximum overlap discrete wavelet transform MODWT decomposition respectively, perform splicing and expansion in the depth dimension, and obtain the MODWT decomposition layer signal; S3: copy the EEG signal obtained in multiple layers of step S1, combine it with the MODWT decomposition layer signal obtained in step S2, and concatenate them in the depth dimension to form a high-dimensional input signal; S4: In the depth dimension and channel dimension, the input signal obtained in step S3 is weighted using the attention mechanism; S5: Construct a motor imagery signal classification network model, which includes two time domain convolution modules, one spatial convolution module, one channel convolution module, one fully connected layer and one linear classifier; S6: Using the input signal obtained in step S3 to construct a labeled training set and a validation set, using the training set to train the motor imagery signal classification network model, and testing the trained network on the validation set; S7: Obtain the EEG signal to be classified, process it according to steps S1-S3, and input it into the trained motor imagery signal classification network model for identification and classification to obtain the motor imagery classification result.

2. The classification method according to claim 1, characterized in that: The step of preprocessing the EEG signal in step S1 includes: First, a 5th-order Butterworth bandpass filter with a bandwidth of [4,40] Hz is used to bandpass filter each channel of the original EEG signal; Then, the baseline of the EEG signal is removed, the average value of the signal is removed from the amplitude of each channel signal, and the signal amplitude is linearly normalized according to the peak value to linearly map the EEG signal to the [0,1] interval.

3. The classification method according to claim 1, characterized in that: The formula for MODWT decomposition in step S2 is as follows: Where x(n) is the EEG signal, h(n) is the wavelet filter function, g(n) is the scale function, n is the time variable, and w j (k) and v j (k) represents the wavelet decomposition coefficient of the jth layer at time k, L j Indicates the length of the j-th layer signal.

4. The classification method according to claim 1, characterized in that: The processing steps of the motor imagery signal classification network model in step S5 include: 1) Input to the first time domain convolution module and process as follows: The input EEG signal first passes through a 1×1 convolutional network to increase the dimension of the input signal depth; Use two layers of separable convolutional networks to perform multi-layer convolution on the input signal, and add a batch normalization layer after each separable convolutional network; Then the output data is regularized through the activation function GELU layer and Dropout layer; Then use a 1×1 convolutional network to reduce the depth of the input signal; 2) Input to the second time domain convolution module and process as follows: Use a 1×1 convolutional network layer to reduce the depth dimension; Use a layer of separable convolutional network to extract deep temporal features; 3) Input to the spatial domain convolution module, which includes two convolution layers, each followed by a batch normalization layer, and finally an activation function GELU layer to extract spatial local features; 4) Input to the channel convolution module, which includes a maximum pooling layer to compress spatiotemporal features and reduce the amount of calculation; 5) Use fully connected layers to combine spatiotemporal features; 6) Use a linear classifier to classify the features.

5. The classification method according to claim 1, characterized in that: In step S6, when training the motor imagery signal classification network model, a cross entropy loss function is used to minimize the classification difference.

6. The classification method according to claim 5, characterized in that: The cross entropy loss function is as follows: Where C is the number of categories, N is the number of samples, and y ij Indicates whether the i-th sample belongs to the j-th category, p ij It represents the probability that the i-th sample is predicted to be of the j-th class.

7. The classification method according to claim 1, characterized in that: The training process uses the Adam optimizer with a learning rate of 0.

001.

8. A classification system for motor imagery EEG signals, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the classification method according to any one of claims 1 to 7 when executing the computer program.

9. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed, the steps of the classification method according to any one of claims 1 to 7 are implemented.

Images