A method, system and storage medium for classifying motor imagery electroencephalogram signals

Abstract

The application discloses a motor imagery electroencephalogram signal classification method and system and a storage medium, and belongs to the technical field of electroencephalogram signal analysis. The time domain signal of each channel in the electroencephalogram signal is subjected to MODWT decomposition, the electroencephalogram signal is spliced with the MODWT decomposition layer signal, and after being weighted in the depth dimension and the channel dimension, the motor imagery signal classification network model is input; the model can be trained to recognize and classify the electroencephalogram signal and obtain a motor imagery classification result. The light network model is adopted, the overfitting problem caused by insufficient training samples is reduced, and good motor imagery signal classification performance can be achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of electroencephalogram (EEG) signal analysis, and relates to a classification method, system, and storage medium for motor imagery EEG signals using a MODWT time-frequency fusion lightweight neural network. Background Art

[0002] Motor imagery is one of the classic paradigms of brain-computer interface (BCI). When the subject imagines a certain limb movement, it activates the electroencephalogram (EEG) signals in specific areas of the brain related to movement. By using non-invasive electrode equipment to collect the EEG signals during motor imagery, the corresponding human movement intention can be extracted from them. Since the motor imagery brain-computer interface has the advantages of not requiring external stimulation and active control, it has received widespread attention in the fields of smart medical care, auxiliary control and rehabilitation treatment.

[0003] EEG signals collected during motor imagery are susceptible to environmental noise interference and exhibit low signal-to-noise ratios, non-stationarity, and high inter-subject variability, significantly impacting signal classification accuracy in motor imagery tasks. Traditional motor imagery signal classification methods primarily consist of feature extraction and feature classification. Feature extraction primarily utilizes methods such as wavelet transform, independent component analysis (ICA), or co-spatial patterns (CSP), while classification algorithms utilize linear discriminant analysis (LDA) and support vector machines (SVM). However, these traditional classification methods suffer from limitations such as reliance on prior knowledge, manual feature selection, and limited training samples. Consequently, they exhibit poor classification accuracy and generalization performance in motor imagery tasks.

[0004] Due to the significant progress made in deep learning in fields such as vision and natural language processing, applying deep learning to motor imagery classification has become a current research hotspot. Compared to traditional methods, deep learning models can better understand brain neural activity. These methods can automatically extract high-dimensional spatiotemporal features from data. Models such as ConvNet, EEGNet, TCNet, graph networks, and residual networks have achieved significant results in EEG signal classification accuracy and exhibit good generalization performance. However, many current deep learning classification models have complex network structures, are limited by the number of training samples, are prone to overfitting, and have high computational complexity, making them unsuitable for real-time and embedded motor imagery brain-computer interface systems. Summary of the Invention

[0005] In order to address the shortcomings of the existing technology, the present invention proposes an EEG signal classification method and system based on the MODWT time-frequency fusion lightweight neural network. The lightweight network model is adopted to reduce the overfitting problem caused by insufficient training samples, and can achieve better motor imagery signal classification performance.

[0006] The technical solution adopted in the present invention is as follows:

[0007] A method for classifying motor imagery EEG signals comprises the following steps:

[0008] S1: Collect multi-channel raw EEG signals and perform preprocessing;

[0009] S2: For the preprocessed EEG signal obtained in step S1, extract the time domain signal of each channel and perform maximum overlap discrete wavelet transform (MODWT) decomposition on each channel, and perform splicing and expansion in the depth dimension to obtain the MODWT decomposition layer signal;

[0010] S3: Copy the EEG signals obtained in multiple layers of step S1, combine them with the MODWT decomposition layer signals obtained in step S2, and splice them in the depth dimension to form a high-dimensional input signal;

[0011] S4: In the depth dimension and channel dimension, the input signal obtained in step S3 is weighted using the attention mechanism;

[0012] S5: Construct a motor imagery signal classification network model, which includes two temporal convolution modules, a spatial convolution module, a channel convolution module, a fully connected layer, and a linear classifier.

[0013] S6: Using the input signal obtained in step S3, construct a labeled training set and a validation set, use the training set to train the motor imagery signal classification network model, and test the trained network on the validation set;

[0014] S7: Obtain the EEG signal to be classified, process it according to steps S1-S3, and input it into the trained motor imagery signal classification network model for recognition and classification to obtain the motor imagery classification result.

[0015] Furthermore, the step of pre-processing the EEG signal in step S1 includes:

[0016] First, a 5th-order Butterworth bandpass filter with a bandwidth of [4,40] Hz is used to bandpass filter each channel of the original EEG signal.

[0017] Then, the baseline of the EEG signal is removed, the average value of the signal is removed from the amplitude of each channel signal, and the signal amplitude is linearly normalized according to the peak value to linearly map the EEG signal to the [0,1] interval.

[0018] Furthermore, the formula for MODWT decomposition in step S2 is as follows:

[0019]

[0020] Where x(n) is the EEG signal, h(n) is the wavelet filter function, g(n) is the scale function, n is the time variable, and w j (k) and v j (k) represents the wavelet decomposition coefficient of the jth layer at time k, L j Indicates the length of the j-th layer signal.

[0021] Furthermore, the processing steps of the motor imagery signal classification network model in step S5 include:

[0022] 1) Input to the first time domain convolution module and process as follows:

[0023] The input EEG signal first passes through a 1×1 convolutional network to increase the depth of the input signal;

[0024] Use two layers of separable convolutional networks to perform multi-layer convolution on the input signal, and add a batch normalization layer after each separable convolutional network;

[0025] Then, the output data is regularized through the activation function GELU layer and Dropout layer;

[0026] Then use a 1×1 convolutional network to reduce the depth of the input signal;

[0027] 2) Input to the second time domain convolution module and process as follows:

[0028] Use a 1×1 convolutional network layer to reduce the depth dimension;

[0029] Use a separable convolutional network to extract deep temporal features;

[0030] 3) Input to the spatial domain convolution module, which consists of two convolutional layers, each followed by a batch normalization layer, and finally an activation function GELU layer to extract spatial local features;

[0031] 4) Input to the channel convolution module, which includes a maximum pooling layer to compress spatiotemporal features and reduce the amount of computation;

[0032] 5) Use fully connected layers to combine spatiotemporal features;

[0033] 6) Use a linear classifier to classify the features.

[0034] Furthermore, in step S6, the motor imagery signal classification network model is trained using a cross entropy loss function to minimize classification differences.

[0035] Furthermore, the training process uses the Adam optimizer with a learning rate of 0.001.

[0036] A classification system for motor imagery EEG signals includes a memory and a processor. The memory stores a computer program, and the processor implements the steps of the above classification method when executing the computer program.

[0037] A computer-readable storage medium stores a computer program, which implements the steps of the above classification method when executed.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] The present invention combines the MODWT wavelet decomposition method with a lightweight neural network, fully exploiting the time-frequency characteristics of EEG signals, and effectively integrating the time-frequency characteristics with the spatiotemporal domain characteristics by combining the wavelet decomposition signal with the time domain signal in the network input layer. At the same time, the use of depthwise separable convolution in the network layer reduces the computational complexity of the network model, and by adopting multi-layer time domain convolution, the network overfitting problem is reduced, achieving higher-performance motor imagery signal classification accuracy with fewer model parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a processing flow chart of the classification method for motor imagery EEG signals proposed by the present invention.

[0041] Figure 2 This is a structural diagram of the motor imagery signal classification network model proposed by the present invention. DETAILED DESCRIPTION

[0042] In order to make the various technical features and advantages or technical effects of the above technical solutions of the present invention more obvious and easy to understand, they are described in detail below with reference to the accompanying drawings.

[0043] The embodiment of the present invention specifically proposes a classification method for motor imagery signals based on MODWT time-frequency fusion lightweight neural network (abbreviated as MODWT-Net). The data processing flow of this method is as follows: Figure 1 As shown, the specific steps include:

[0044] S1: Preprocess the raw multi-channel EEG signals, including filtering and normalization. The data extracted from the different channels of the EEG signals contain various types of noise. First, a 5th-order Butterworth filter with a bandwidth of [4, 40] Hz is used to bandpass filter each channel of the raw EEG signals, retaining only information in the useful frequency band. Next, baseline error is removed by subtracting the mean value from the amplitude of each channel signal to reduce the impact of data variance. Finally, the data is normalized, linearly mapping the EEG signals to the [0, 1] interval to accelerate network convergence. Furthermore, the preprocessed data is divided into training and validation sets according to a certain ratio.

[0045] S2: Maximum Overlap Discrete Wavelet Transform (MODWT) decomposition. To address the problem of insufficient utilization of frequency domain features in existing network models, this method performs MODWT decomposition on each channel of the preprocessed EEG signal and merges the signals of each decomposition layer into the depth dimension to extract the time-frequency features of the EEG signal.

[0046] The formula for MODWT decomposition is as follows:

[0047]

[0048] Where x(n) is the EEG signal, h(n) is the wavelet filter function, g(n) is the scaling function, and w j and v j represents the wavelet decomposition coefficient of the jth layer;

[0049] The specific parameters use 5 layers of wavelet decomposition, and the wavelet filter function uses sym4 wavelet. Each channel signal is processed separately. It is assumed that the dimension of the preprocessed EEG signal two-dimensional data is {C, N}, where C is the number of EEG channels and N is the length of the EEG signal. The MODWT decomposition signals of each layer are spliced ​​in the depth dimension to obtain a {1, 5, C, N}-dimensional signal.

[0050] S3: Time-Frequency Fusion. Because the time-frequency characteristics of some data are not obvious, they still rely on the signal's time-domain characteristics. Multiple layers of preprocessed EEG signals are replicated, expanded in the depth dimension, and then spliced ​​and merged with the MODWT decomposition signal to obtain a high-dimensional input signal, effectively fusing the time-frequency feature information. For example, signal splicing involves replicating the {C, N}-dimensional EEG signal along the depth dimension for five layers to obtain a {1, 5, C, N}-dimensional signal. This signal is then spliced ​​and combined with the wavelet decomposition layer signal's {1, 5, C, N}-dimensional MODWT decomposition result along the depth dimension to obtain a {1, 10, C, N}-dimensional signal. Optionally, the number of wavelet decomposition layers and the number of EEG signal replication layers can be adjusted according to the specific task to achieve optimal network model classification performance.

[0051] S4: Attention mechanism weighting. To address the problem that different time-frequency components and channel components play different roles in the motor imagery signal classification task, the attention mechanism is used to weight the depth dimension and channel dimension, multiply them by a randomly initialized weight, and continuously update them during the network training process. For example, the depth and channel dimensions of the multidimensional signal {1,10,C,N} are multiplied by the weights {α1,α2,…,α 10} and {β1,β2,……,β C}, initialized with a uniformly distributed random function and used as network parameters for training.

[0052] S5: Network model construction. Drawing on EEGNet, we use separable convolution to build a lightweight neural network model, which includes two time domain convolutions, one spatial convolution and one fully connected layer. We also build the network model based on Pytorch. The model structure is as follows: Figure 2 As shown in the figure, the input layer refers to the time-frequency fused EEG signal of input {1,10,C,N}, which mainly includes:

[0053] Time domain convolution module 1: A 1×1 convolutional network layer is first used for the input {1,10,C,N} signal to increase the depth dimension and output the signal dimension {1,25,C,N} to enrich the time-frequency fusion features of the depth dimension; then two consecutive convolutional layers are stacked to perform multiple convolutions on the input signal, which can better extract the time domain features and reduce the overfitting problem caused by a single convolution kernel being too large. Both convolutions use depthwise separable convolutions with kernel sizes of (1,60) and (1,10) respectively. The group size is the depth dimension (for example, 20), the padding value is 0, the stride is set to 1, and each convolution layer is followed by a BatchNorm batch normalization layer; the time domain convolution module 1 finally has a GELU layer and a Dropout layer with a dropout rate of 0.3 to regularize the output data and reduce the overfitting problem that occurs during training; then a 1×1 convolutional network is used to reduce the depth dimension of the input signal, and the depth dimension of the output signal becomes 12.

[0054] Temporal convolution module 2: The upper layer output is passed through a 1×1 convolutional network layer to reduce the depth dimension. A separable convolution layer is then used to further extract deep temporal features. The convolution kernel and group size are both the depth dimension size after dimensionality reduction. The group size is 10, the padding value is 0, and the stride is set to 1. Both convolution layers are followed by a BatchNorm layer.

[0055] Spatial convolution: The upper layer output is passed through a spatial convolution layer with a convolution kernel size of (C, 1), where C is the number of channels, the padding value is 0, and the stride is set to 1, thereby extracting spatial local features;

[0056] Channel convolution: Perform maximum pooling and dropout on features to compress spatiotemporal features and reduce computational complexity. The pooling rate is 25 and the dropout rate is 0.5, which improves the generalization ability of the model.

[0057] Fully connected layer: After flattening all features in the depth dimension, the input is stacked into the fully connected layer to fuse the time-frequency and spatiotemporal features;

[0058] Linear classifier: Input to a linear classifier to classify the features.

[0059] S6: The MODWT-Net network model is trained using the training and validation data sets of motor imagery EEG signals. The cross entropy loss function is used in the training process. The network uses the cross entropy loss function to minimize the classification difference. The function is as follows:

[0060]

[0061] Where C is the number of categories, N is the number of samples, and y ij Indicates whether the i-th sample belongs to the j-th category, p ij It represents the probability that the i-th sample is predicted to be of the j-th class.

[0062] The Adam optimizer was used to optimize the model parameters with a learning rate of 0.001 and an epoch of 300.

[0063] Experimental test:

[0064] To further illustrate the advantages of the proposed method, the proposed MODWT-Net model was tested on the BCI-Competition IV-2a dataset. This dataset contains motor imagery EEG signals from nine subjects. Each subject participated in four motor imagery tasks, including imagined movements of the left hand, right hand, feet, and tongue. Two recording sessions were conducted using 22 electrodes with a sampling rate of 250 Hz. Each session contained 288 data trials. The first session was used for training, and the second for testing.

[0065] Considering the significant inter-subject variability, the EEG signal data of nine subjects were individually classified. The training and validation sets of this dataset were preprocessed and input to the network. The EEG signal dimensions were (22, 1125), where 22 represents the number of channels and 1125 represents the data time length. The EEG signals were decomposed using a five-layer sysm4 wavelet, expanding the depth dimension to a signal with dimensions of (1, 5, 22, 1125). The original signal was replicated five layers in the depth dimension and then concatenated with the wavelet decomposition result in the depth dimension, resulting in an input signal with dimensions of (1, 10, 22, 1125). In the time-domain convolution module 1, a 1×1 convolution was used to increase the input signal's depth dimension from 10 to 25, followed by two separable convolutional layers and a GELU layer. In the time-domain convolution module 2, a 1×1 convolution was used to reduce the input signal's depth dimension from 25 to 12, followed by a separable convolutional layer. The channel features are then extracted using spatial convolution, with the output channel dimension set to 1. This is then pooled using a Maxpool layer with a pooling rate of 25. Finally, the output signal, with a depth dimension of 12, is flattened and classified using a fully connected layer and a linear classifier. Training uses the Adam optimizer with a learning rate of 0.001, a batch size of 64, and 300 epochs.

[0066] In order to evaluate the model performance, the average classification accuracy (Acc) and Kappa coefficient of 9 subjects were used as model evaluation indicators. The method of the present invention was compared with the currently popular convolutional network models FBCSP, EEGNet, DeepConvNet, ShallowConvNet and Conformer models. The comparison results are shown in Table 1.

[0067] Table 1 Statistics of classification recognition rate and Kappa coefficient of the present invention and other comparative network models

[0068]

[0069] As can be seen from Table 1, the average classification accuracy of the MODWT-Net proposed in the present invention is 78.7%, and the Kappa coefficient is 0.71, both of which are better than the comparison network model. In addition, the network parameter of MODWT-Net is only 4794, which is only higher than the EEGNet model (4080) and FBCNet (4180) among the above models, and far lower than the other models, which proves the effectiveness and superiority of the lightweight neural network model proposed in the present invention.

[0070] Although the present invention has been disclosed as above by way of embodiments, they are not intended to limit the present invention. Any appropriate modification or equivalent substitution of the technical solution of the present invention by a person skilled in the art should be included in the protection scope of the present invention. The protection scope of the present invention shall be based on that defined in the claims.

Claims

1. A method for classifying motor imagery EEG signals, characterized in that: The following steps are involved: S1: Collect multi-channel raw EEG signals and perform preprocessing; S2: For the preprocessed EEG signal obtained in step S1, extract the time domain signal of each channel and perform maximum overlap discrete wavelet transform (MODWT) decomposition on each channel, perform splicing and expansion in the depth dimension, and obtain the MODWT decomposition layer signal; S3: Copy the EEG signals obtained in multiple layers of step S1, combine them with the MODWT decomposition layer signals obtained in step S2, and splice them in the depth dimension to form a high-dimensional input signal; S4: In the depth dimension and channel dimension, the input signal obtained in step S3 is weighted using the attention mechanism; S5: Construct a motor imagery signal classification network model, which includes two temporal convolution modules, a spatial convolution module, a channel convolution module, a fully connected layer, and a linear classifier. S6: Using the input signal obtained in step S3, construct a labeled training set and a validation set, use the training set to train the motor imagery signal classification network model, and test the trained network on the validation set; S7: Obtain the EEG signal to be classified, process it according to steps S1-S3, and input it into the trained motor imagery signal classification network model for recognition and classification to obtain the motor imagery classification result.

2. The classification method according to claim 1, wherein: The steps of pre-processing the EEG signal in step S1 include: First, a 5th-order Butterworth bandpass filter with a bandwidth of [4,40] Hz is used to bandpass filter each channel of the original EEG signal; Then, the baseline of the EEG signal is removed, the average value of the signal is removed from the amplitude of each channel signal, and the signal amplitude is linearly normalized according to the peak value to linearly map the EEG signal to the [0,1] interval.

3. The classification method according to claim 1, wherein: The formula for MODWT decomposition in step S2 is as follows: Where x(n) is the EEG signal, h(n) is the wavelet filter function, g(n) is the scale function, n is the time variable, and w j (k) and v j (k) represents the wavelet decomposition coefficient of the jth layer at time k, L j Indicates the length of the j-th layer signal.

4. The classification method according to claim 1, wherein: The processing steps of the motor imagery signal classification network model in step S5 include: 1) Input to the first time domain convolution module and process as follows: The input EEG signal first passes through a 1×1 convolutional network to increase the depth of the input signal; Use two layers of separable convolutional networks to perform multi-layer convolution on the input signal, and add a batch normalization layer after each separable convolutional network; Then, the output data is regularized through the activation function GELU layer and Dropout layer; Then use a 1×1 convolutional network to reduce the depth of the input signal; 2) Input to the second time domain convolution module and process as follows: Use a 1×1 convolutional network layer to reduce the depth dimension; Use a separable convolutional network to extract deep temporal features; 3) Input to the spatial domain convolution module, which consists of two convolutional layers, each followed by a batch normalization layer, and finally an activation function GELU layer to extract spatial local features; 4) Input to the channel convolution module, which includes a maximum pooling layer to compress spatiotemporal features and reduce the amount of computation; 5) Use fully connected layers to combine spatiotemporal features; 6) Use a linear classifier to classify the features.

5. The classification method according to claim 1, wherein: In step S6, the motor imagery signal classification network model is trained using a cross entropy loss function to minimize the classification difference.

6. The classification method according to claim 5, wherein: The cross entropy loss function is as follows: Where C is the number of categories, N is the number of samples, and y ij Indicates whether the i-th sample belongs to the j-th category, p ij It represents the probability that the i-th sample is predicted to be of the j-th class.

7. The classification method according to claim 1, wherein: The training process uses the Adam optimizer with a learning rate of 0.

001.

8. A classification system for motor imagery EEG signals, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the classification method according to any one of claims 1 to 7 when executing the computer program.

9. A computer-readable storage medium, characterized in that A computer program is stored, and when the computer program is executed, the steps of the classification method according to any one of claims 1 to 7 are implemented.

Images