A method and system for classifying and recognizing EEG signals based on deep learning

Abstract

A method and system for classifying and identifying EEG signals based on deep learning. The present invention relates to a method and system for classifying and identifying EEG signals based on deep learning. The present invention addresses the problem of low EEG signal classification and recognition accuracy due to factors such as a low EEG signal-to-noise ratio. The process is as follows: 1: obtaining an original EEG signal dataset and processing it into a one-dimensional original EEG signal feature vector; 2: using an EMD decomposition algorithm to expand the EEG signal to obtain new EEG signal data samples; 3: constructing an improved CNN convolutional neural network to obtain one-dimensional EEG spatial features; 4: constructing an improved LSTM recurrent neural network to obtain one-dimensional EEG temporal features; 5: constructing an autoencoder to reconstruct the spliced ​​spatiotemporal features, which are then input into a classifier to complete the classification and recognition of a motor imagery task; and 6: collecting the EEG signal to be tested from the subject to complete the classification and recognition of the motor imagery task. The present invention belongs to the field of brain-computer interfaces.

Description

Technical Field

[0001] The present invention belongs to the field of brain-computer interface, and specifically relates to a method and system for classifying and recognizing EEG signals based on deep learning. Background Art

[0002] A brain-computer interface (BCI), sometimes also called a "direct neural interface" (DNI) or "brain-machine interface" (BMI), establishes a new type of real-time communication and control system between the biological (human or animal) brain and external devices or the environment, thereby enabling direct interaction between the brain and external devices, playing the role of monitoring, replacement, improvement / recovery, enhancement, and supplementation. Focusing on the five major functions mentioned above, brain-computer interface technology is applied in medical care, education, entertainment, smart homes, military, and other fields.

[0003] Among mainstream non-invasive EEG signals, the motor imagery EEG paradigm has been extensively studied. Brain-computer interfaces (BCIs) based on motor imagery provide patients with movement disorders or those in a completely locked-out state with an interface to interact with their environment by controlling robotic prostheses, wheelchairs, and other devices. MI BCIs have a wide range of applications, including controlling wheelchairs in two- and three-dimensional space, virtual reality, neurorehabilitation, and controlling devices like quadcopters. Motor imagery (MI), as the name suggests, involves activation of specific brain regions when a person imagines moving their limbs (or muscles) without actually producing movement. By analyzing EEG signals and detecting and identifying activation effects in different brain regions, user intent can be determined, enabling direct communication and control between the human brain and external devices. Currently, common motor imagery involves the left hand, right hand, both hands, both feet, and the tongue.

[0004] The primary focus of EEG signal classification research is to develop a classification model with high recognition accuracy and robustness. Research in this area has been ongoing for decades. Traditional classification algorithms have achieved promising results. However, due to the tedious feature extraction process inherent in these algorithms and the continuous advancement of computer hardware technology, researchers, driven by the rise of deep learning, have turned their attention to EEG signal classification algorithms based on deep learning, achieving significant results. Deep learning simulates the information processing mechanisms of the human brain to process the vast amount of nonlinear data available in real life. It can also mine deep features in the data that are difficult to describe directly in words. The efficiency of deep learning is unmatched by traditional classification algorithms. Compared to traditional algorithms, deep learning algorithms offer the advantage of eliminating the need for complex and extensive feature engineering, and the extracted features are more representative and more similar to the data itself. On the other hand, EEG signals, with their extremely low signal-to-noise ratio, are not only affected by ambient noise during acquisition, but also by artifacts generated by the subject's own muscle movements and eye movements. These signals have similar characteristics to EEG signals, and noise reduction methods are inadequate.

[0005] In summary, based on the feature extraction and classification recognition of EEG signals, it is crucial to effectively extract information that can represent the characteristics of brain activity and train a classifier with good generalization ability. Summary of the Invention

[0006] The present invention addresses the problem of low accuracy in EEG signal classification and recognition due to reasons such as low EEG signal-to-noise ratio, and proposes a method and system for EEG signal classification and recognition based on deep learning.

[0007] A method for EEG signal classification and recognition based on deep learning has the following specific process:

[0008] Step 1: Obtain the original EEG signal dataset, that is, the public dataset: PhysioNet eegmidb database;

[0009] Preprocessing the original EEG signal data into a one-dimensional original EEG signal feature vector;

[0010] Step 2: Use the EMD decomposition algorithm to expand the one-dimensional original EEG signal obtained in step 1 to obtain a new EEG signal data sample;

[0011] Step 3: Construct an improved CNN convolutional neural network and use it to extract EEG spatial features from the new EEG signal data samples obtained in step 2 to obtain a one-dimensional EEG spatial feature X s ;

[0012] Step 4: Construct an improved LSTM recurrent neural network RNN ​​and use it to extract the EEG time feature of the new EEG signal in step 2 to obtain a one-dimensional EEG time feature X t ;

[0013] Step 5: Concatenate the one-dimensional EEG spatial features and the one-dimensional EEG temporal features to obtain the concatenated spatiotemporal features;

[0014] An automatic encoder (AE) is constructed to reconstruct the spliced ​​spatiotemporal features. The reconstructed features are then input into the classifier to complete the classification and recognition of the motor imagery task.

[0015] Step 6: The subject wears the BCI2000 EEG device to collect the subject's EEG signal to be measured, and the EEG signal to be measured is represented as a one-dimensional vector;

[0016] The one-dimensional vector is input into the trained improved CNN convolutional neural network, and the output of the second fully connected layer of the improved CNN convolutional neural network is used as the spatial feature extracted by the improved CNN convolutional neural network, which is called the one-dimensional EEG spatial feature;

[0017] The one-dimensional vector is input into the trained improved LSTM recurrent neural network RNN, and the data output by the second LSTM of the improved LSTM recurrent neural network RNN ​​is used as the time feature extracted by the improved LSTM recurrent neural network RNN, which is called the one-dimensional EEG time feature;

[0018] The obtained one-dimensional EEG spatial features and the obtained one-dimensional EEG temporal features are spliced ​​together to obtain a sample;

[0019] The obtained samples are input into the trained autoencoder AE, and the output features of the autoencoder AE are input into the classifier to complete the classification and recognition of the motor imagery task.

[0020] A deep learning-based EEG signal classification and recognition system is used to execute a deep learning-based EEG signal classification and recognition method.

[0021] The beneficial effects of the present invention are:

[0022] This paper designs and implements a deep learning-based EEG signal classification method and system. Using a modified convolutional neural network combined with a modified recurrent neural network model, temporal and spatial features extracted from preprocessed EEG signals are superimposed. An autoencoder (AE) layer is designed to learn an abstract representation of the superimposed data and perform sample reconstruction. Finally, a LightGBM classifier is used for intent recognition.

[0023] The algorithm designed by the present invention has better classification effect than traditional classification algorithms such as support vector machine (SVM).

[0024] The data enhancement method designed in the present invention based on EMD decomposition of EEG signals also has a good effect on data expansion of EEG signal datasets.

[0025] The LightGBM classification algorithm designed by the present invention has short training time and high classification accuracy.

[0026] The method of the present invention was evaluated on the PhysioNet eegmidb dataset. The results showed that the classification accuracy of the method of the present invention was 96.32%, which can effectively complete the classification of EEG signals and enable the classification effect to be visualized on the web page. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is the architecture diagram of convolutional neural network CNN;

[0028] Figure 2 This is the LSTM architecture diagram of the recurrent neural network;

[0029] Figure 3 This is a comparison chart of the accuracy and loss rate results of recurrent neural networks with different structures;

[0030] Figure 4 This is a diagram showing the effect of different learning rates on the AE layer of the autoencoder;

[0031] Figure 5 This is the effect of the data enhancement EMD method on the CNN network;

[0032] Figure 6 This is the effect of the data enhancement EMD method on the LSTM network;

[0033] Figure 7a This is a comparison chart of the final autoencoder confusion matrix classification results before data enhancement;

[0034] Figure 7b This is a comparison chart of the final autoencoder confusion matrix classification results after data enhancement;

[0035] Figure 8 It is the final confusion matrix diagram obtained by the entire network structure;

[0036] Figure 9 This is the system login interface picture;

[0037] Figure 10 This is the CNN network model parameter adjustment interface diagram;

[0038] Figure 11This is the classification result interface of the EEG signal classification and recognition system. DETAILED DESCRIPTION

[0039] Specific implementation method 1: This implementation method is a method for classifying and identifying EEG signals based on deep learning. The specific process is as follows:

[0040] Step 1: Obtain the original EEG signal dataset, i.e. the public dataset: PhysioNet eegmidb (EEG motor movement / image database);

[0041] Preprocessing the original EEG signal data into a one-dimensional original EEG signal feature vector;

[0042] Step 2: Since EEG signal acquisition is difficult and the size of the dataset is a key factor affecting the accuracy of the deep learning network, the original dataset is expanded;

[0043] The EMD decomposition algorithm is used to expand the one-dimensional original EEG signal obtained in step 1 to obtain a new EEG signal data sample;

[0044] The EEG signals in step 1 and step 2 are labeled EEG signals;

[0045] Step 3: Construct an improved CNN convolutional neural network and use it to extract EEG spatial features from the new EEG signal data samples obtained in step 2 to obtain a one-dimensional EEG spatial feature X s In the basic model, we adjusted the number of convolutional layers, pooling layers, and learning rate, and used classification accuracy, loss rate, ROC curve, and confusion matrix to identify the optimal network structure, and analyzed and compared the experimental results.

[0046] Step 4: Construct an improved LSTM recurrent neural network RNN ​​and use it to extract the EEG time feature of the new EEG signal in step 2 to obtain a one-dimensional EEG time feature X t Based on the basic network model structure, we adjusted the learning rate and the number of LSTM neuron layers to design four network models with different structures. We used classification accuracy, loss rate, ROC curve, and confusion matrix to find the optimal network model for extracting time features.

[0047] Step 5: Concatenate the one-dimensional EEG spatial features and the one-dimensional EEG temporal features to obtain the concatenated spatiotemporal features;

[0048] An automatic encoder (AE) is constructed to reconstruct the spliced ​​spatiotemporal features. The reconstructed features are then input into the classifier to complete the classification and recognition of the motor imagery task.

[0049] The abstract reconstruction representation of the input features is achieved through two steps: encoding and decoding;

[0050] Set different learning rates to compare the reconstruction effect of the AE layer on the input features;

[0051] Step 6: The subject wears the BCI2000 EEG device to collect the subject's EEG signal to be measured, and the EEG signal to be measured is represented as a one-dimensional vector;

[0052] The one-dimensional vector is input into the trained improved CNN convolutional neural network, and the output of the second fully connected layer of the improved CNN convolutional neural network is used as the spatial feature extracted by the improved CNN convolutional neural network, which is called the one-dimensional EEG spatial feature;

[0053] The one-dimensional vector is input into the trained improved LSTM recurrent neural network RNN, and the data output by the second LSTM of the improved LSTM recurrent neural network RNN ​​is used as the time feature extracted by the improved LSTM recurrent neural network RNN, which is called the one-dimensional EEG time feature;

[0054] The obtained one-dimensional EEG spatial features and the obtained one-dimensional EEG temporal features are spliced ​​together to obtain a sample;

[0055] The obtained samples are input into the trained autoencoder AE, and the output features of the autoencoder AE are input into the classifier to complete the classification and recognition of the motor imagery task.

[0056] Specific embodiment 2: This embodiment differs from specific embodiment 1 in that the original EEG signal dataset is obtained in step 1, i.e., the public dataset: PhysioNet eegmidb (EEG motor movement / image database);

[0057] Preprocessing the original EEG signal data into a one-dimensional original EEG signal feature vector;

[0058] The EEG signals in step 1 and step 2 are labeled EEG signals;

[0059] The specific process is:

[0060] Step 1.1: Each subject (109) wore a BCI2000 EEG device, sat in front of a computer screen, and performed certain actions or imagined motor tasks according to the prompts on the screen. The BCI2000 EEG device used 64 channels to record each subject's EEG signals at a sampling rate of 160 Hz.

[0061] The EEG signals recorded by the BCI2000 EEG device correspond to the type of motor imagery task undertaken by the subjects;

[0062] The motor imagery task types are divided into five categories:

[0063] The first category is rest;

[0064] The second category is motor imagery of the left hand;

[0065] The third category is motor imagery of the right hand;

[0066] The fourth category is motor imagery of both hands;

[0067] The fifth category is motor imagery of the feet;

[0068] Each subject underwent 14 experiments, and the 14 experiments were as follows:

[0069] 1 time, baseline, eyes open

[0070] 2 times, baseline, eyes closed

[0071] 3 times, Task 1: Open and close the left or right hand;

[0072] 4 times, Task 2: Imagine opening and closing your left or right hand;

[0073] 5 times, Task 3: Open and close your hands or feet;

[0074] 6 times, Task 4: Imagine opening and closing your hands or feet;

[0075] 7 times, Task 1: Open and close the left or right hand;

[0076] 8 times, Task 2: Imagine opening and closing your left or right hand;

[0077] 9 times, Task 3: Open and close your hands or feet;

[0078] 10 times, Task 4: Imagine opening and closing your hands or feet;

[0079] 11 times, Task 1: Open and close the left or right hand;

[0080] 12 times, Task 2: Imagine opening and closing your left or right hand;

[0081] 13 times, Task 3: Open and close your hands or feet;

[0082] 14 times, Task 4: Imagine opening and closing your hands or feet;

[0083] The EEG signals generated by the subjects performing real movements to open and close their left hand were the same as those generated by just imagining opening and closing their left hand. The EEG signals generated by these two methods belong to the same category, namely the second type of movement imagining the left hand;

[0084] The EEG signal dataset includes three label annotation codes T0, T1 or T2;

[0085] T0 represents rest, which corresponds to the entire 1st and 2nd trials (one minute each) and the rest in trials 3–14 for each subject (rest (T0) and motor imagery (T1 or T2) in trials 3–14 were performed alternately for 2 minutes, with a rest period of approximately 4 seconds);

[0086] T1 represents the subjects' imagined or actual opening and closing of their left hand, corresponding to the 3rd, 4th, 7th, 8th, 11th, and 12th trials for each subject (one imagined or actual opening and closing of the left hand lasted approximately 4 seconds); T1 represents the subjects' imagined or actual opening and closing of both hands, corresponding to the 5th, 6th, 9th, 10th, 13th, and 14th trials for each subject (one imagined or actual opening and closing of both hands lasted approximately 4 seconds);

[0087] T2 represents the subjects' imagined or actual opening and closing of their right hand, corresponding to the 3rd, 4th, 7th, 8th, 11th, and 12th trials for each subject (one imagined or actual opening and closing of the right hand lasted approximately 4 seconds); T2 represents the subjects' imagined or actual opening and closing of their feet, corresponding to the 5th, 6th, 9th, 10th, 13th, and 14th trials for each subject (one imagined or actual opening and closing of the feet lasted approximately 4 seconds);

[0088] The experimental process taking subject S001 as an example is as follows:

[0089] In the first experiment, the eyes were open for one minute;

[0090] In the second experiment, the eyes were closed for one minute;

[0091] The third experiment lasted two minutes, in which rest (T0) and motor imagery task (T1 or T2) were performed alternately for a total of 2 minutes. The order in which T1 and T2 alternated with T0 was uncertain. This was the third experiment, so T1 represented the real opening and closing of the left hand, and T2 represented the real opening and closing of the right hand. The specific process was as follows: rest 4s, real opening and closing of the right hand 4s, rest 4s, real opening and closing of the left hand 4s, rest 4s, real opening and closing of the left hand 4s, rest 4s, real opening and closing of the right hand 4s, rest 4s, real opening and closing of the right hand 4s, rest 4s, real opening and closing of the right hand 4s, rest 4s, real Open and close the left hand for 4s, rest for 4s, actually open and close the left hand for 4s, rest for 4s, actually open and close the right hand for 4s, rest for 4s, actually open and close the left hand for 4s, rest for 4s, actually open and close the right hand for 4s, rest for 4s, actually open and close the right hand for 4s, rest for 4s, actually open and close the right hand for 4s, rest for 4s, actually open and close the left hand for 4s, rest for 4s, actually open and close the left hand for 4s, rest for 4s, actually open and close the right hand for 4s, rest for 4s, actually open and close the left hand for 4s; in total, there are 15 rests, 15 actual opening and closings of the right and left hands, 30 total times for a total of 120s;

[0092] The fourth experiment lasted two minutes, in which rest (T0) and motor imagery tasks (T1 or T2) were performed alternately for a total of 2 minutes. The order in which T1 and T2 alternated with T0 was uncertain. This was the fourth experiment, so T1 represented imagining opening and closing the left hand, and T2 represented imagining opening and closing the right hand. The specific process was as follows: rest for 4 seconds, imagine opening and closing the right hand for 4 seconds, rest for 4 seconds, imagine opening and closing the left hand for 4 seconds, rest for 4 seconds, imagine opening and closing the left hand for 4 seconds, rest for 4 seconds, imagine opening and closing the right hand for 4 seconds, rest for 4 seconds, imagine opening and closing the right hand for 4 seconds, rest for 4 seconds, imagine opening and closing the right hand for 4 seconds, rest for 4 seconds, imagine Open and close left hand for 4 seconds, rest for 4 seconds, imagine opening and closing right hand for 4 seconds, rest for 4 seconds, imagine opening and closing left hand for 4 seconds, rest for 4 seconds, imagine opening and closing right hand for 4 seconds, rest for 4 seconds, imagine opening and closing left hand for 4 seconds, rest for 4 seconds, imagine opening and closing left hand for 4 seconds, rest for 4 seconds, imagine opening and closing right hand for 4 seconds, rest for 4 seconds, imagine opening and closing left hand for 4 seconds; 15 times in total, rest, imagine opening and closing right and left hands for 15 times in total, 30 times in total, 120 seconds in total;

[0093] The fifth experiment lasted two minutes, in which rest (T0) and motor imagery task (T1 or T2) were performed alternately for a total of 2 minutes. The order in which T1 and T2 alternated with T0 was uncertain. This was the fifth experiment, so T1 represented the real opening and closing of the hands, and T2 represented the real opening and closing of the feet. The specific process was as follows: rest 4s, real opening and closing of hands, rest 4s, real opening and closing of feet 4s, rest 4s, real opening and closing of hands 4s, rest 4s, real opening and closing of feet 4s, rest 4s, real opening and closing of hands 4s, rest 4s, real opening and closing of feet 4s, Open and close the legs for 4 seconds, rest for 4 seconds, actually open and close the legs for 4 seconds, rest for 4 seconds, actually open and close the hands for 4 seconds, rest for 4 seconds, actually open and close the hands for 4 seconds, rest for 4 seconds, actually open and close the legs for 4 seconds, rest for 4 seconds, actually open and close the legs for 4 seconds, rest for 4 seconds, actually open and close the legs for 4 seconds, rest for 4 seconds, actually open and close the legs for 4 seconds, rest for 4 seconds, actually open and close the legs for 4 seconds; in total, 15 rests, 15 actual opening and closings of the legs and hands, 30 reps for a total of 120 seconds;

[0094] The sixth experiment lasted two minutes, in which rest (T0) and motor imagery tasks (T1 or T2) were performed alternately for a total of 2 minutes. The order in which T1 and T2 alternated with T0 was uncertain. This was the sixth experiment, so T1 represented imagining opening and closing the hands, and T2 represented imagining opening and closing the feet. The specific process was as follows: rest for 4 seconds, imagine opening and closing the hands for 4 seconds, rest for 4 seconds, imagine opening and closing the feet for 4 seconds, rest for 4 seconds, imagine opening and closing the feet for 4 seconds, rest for 4 seconds, imagine opening and closing the hands for 4 seconds, rest for 4 seconds, imagine opening and closing the feet for 4 seconds, rest for 4 seconds, imagine opening and closing the hands for 4 seconds, rest for 4 seconds, imagine opening and closing the feet for 4 seconds, rest for 4 seconds, imagine Open and close your hands for 4 seconds, rest for 4 seconds, imagine opening and closing your hands for 4 seconds, rest for 4 seconds, imagine opening and closing your feet for 4 seconds, rest for 4 seconds, imagine opening and closing your feet for 4 seconds, rest for 4 seconds, imagine opening and closing your hands for 4 seconds, rest for 4 seconds, imagine opening and closing your hands for 4 seconds, rest for 4 seconds, imagine opening and closing your feet for 4 seconds, rest for 4 seconds, imagine opening and closing your feet for 4 seconds, rest for 4 seconds, imagine opening and closing your hands for 4 seconds, rest for 4 seconds, imagine opening and closing your hands for 4 seconds; a total of 15 rests, imagine opening and closing your hands and feet for a total of 15 times, a total of 30 times for a total of 120 seconds;

[0095] Experiments 7-14 were similar;

[0096] The 14 experiments for other subjects followed a similar procedure;

[0097] Step 1.2: 10 subjects were randomly selected from the subjects, and a total of N (280,000) labeled EEG signals were selected from the 10 subjects (28,000 samples per subject); each EEG signal sample is a vector of 64 elements, each element corresponding to a channel of the EEG data;

[0098] The EEG signal data is represented as a one-dimensional vector; the signal feature dimension is [1,64], where 64 is the number of electrode channels in the original EEG signal and 1 is a time point;

[0099] Next, input is given to the RNN structure and CNN structure for feature learning.

[0100] Other steps and parameters are the same as those in the first embodiment.

[0101] Specific embodiment three: This embodiment differs from specific embodiments one or two in that, in step 2, since EEG signal acquisition is difficult and the size of the data set is a key factor affecting the accuracy of the deep learning network, the original data set is expanded;

[0102] The EMD decomposition algorithm is used to expand the one-dimensional original EEG signal obtained in step 1 to obtain a new EEG signal data sample;

[0103] The EEG signals in step 1 and step 2 are labeled EEG signals;

[0104] Verify the improved effect of data augmentation in the improved deep learning model proposed in steps 3 and 4;

[0105] The specific process is:

[0106] Step 2.1: Perform EMD decomposition on the one-dimensional original EEG signal data of the five types of motor imagery obtained in step 1 to obtain the IMF components of different channels of each type of motor imagery EEG signal sample;

[0107] Step 2.2: Randomly select the IMF components of different channels of each motor imagery EEG signal sample belonging to the same category, and perform the same processing on the training set and the test set;

[0108] Step 2.3: Superimpose the IMF components of the randomly selected corresponding channels (the same channel). The sum of the components can obtain the properties of the original EEG signal and obtain a new EEG signal data sample;

[0109] After the data is expanded, the classification effects on the improved convolutional neural network and the improved recurrent neural network are compared before and after data enhancement.

[0110] Other steps and parameters are the same as those in the first or second embodiment.

[0111] Specific embodiment 4: This embodiment differs from any one of the specific embodiments 1 to 3 in that in step 3, an improved CNN convolutional neural network is constructed, and the improved CNN convolutional neural network is used to extract EEG spatial features from the new EEG signal data sample obtained in step 2 to obtain a one-dimensional EEG spatial feature X. s In the basic model, we adjusted the number of convolutional layers, pooling layers, and learning rate, and used classification accuracy, loss rate, ROC curve, and confusion matrix to identify the optimal network structure, and analyzed and compared the experimental results.

[0112] The specific process is:

[0113] Step 3.1: Build an improved CNN convolutional neural network:

[0114] The improved CNN convolutional neural network includes: input layer, first convolution layer, first pooling layer, first fully connected layer, second fully connected layer and output layer;

[0115] Improve the output data of the i′th layer of the CNN convolutional neural network with X i′ Indicates that i′=1, 2, …, 6, the first layer is the input layer, the second layer is the first convolutional layer, the third layer is the first pooling layer, the fourth layer is the first fully connected layer, the fifth layer is the second fully connected layer, and the sixth layer is the output layer;

[0116] Step 3.2: Input the new EEG signal data sample obtained in step 2 into the input layer of the improved CNN convolutional neural network. The input layer outputs the EEG signal in the shape of [1,64].

[0117] 1 is the dimension, 64 is the number of electrode channels;

[0118] Step 3.3: First convolutional layer operation:

[0119] Set the convolution filter with kernel size [1,1] and stride size [1,1], where stride represents the x- and y-movement distances of the convolution kernel.

[0120] The padding method is chosen to be zero padding of the same shape, which keeps the sample shape unchanged in the convolution calculation;

[0121] After the first convolutional layer operation, the shape of X2 is [1, 64, 2], where 1 is the height, 64 is the width, and 2 is the number of channels;

[0122] The Relu activation function is then used to process the convolution result. It has sparseness and can effectively alleviate the vanishing gradient. The mathematical expression of the activation function Relu is as follows:

[0123]

[0124] Step 3.4: First pooling layer operation:

[0125] Maximize pooling to reduce the spatial size of input EEG features and reduce overfitting;

[0126] Set the pooling window size to [1, 2] and the stride size to [1, 2], without changing the depth. The stride represents the x- and y-movement distances of the pooling window.

[0127] After the first pooling layer operation, the shape of X3 is [1,32,2]; 1 is the height, 32 is the width, and 2 is the number of channels;

[0128] Step 3.5: First fully connected layer operation: Calculate the output data according to the conventional neural network operation;

[0129] Step 3.4: The output data of shape [1,32,2] is flattened into a vector of shape [1,64].

[0130] Step 3.6: Second fully connected layer operation:

[0131] The first fully connected layer operation outputs data of shape [1,64] which is flattened into a vector of shape [1,120];

[0132] Finally, the output layer obtains the result of the improved CNN convolutional neural network with a shape of [1,5];

[0133] 5 represents the five types of motor imagery tasks;

[0134] Step 3.7: Train the improved CNN convolutional neural network. The specific process is as follows:

[0135] Use cross entropy loss function; optimize the loss function through AdamOptimizer algorithm;

[0136] The weight matrix and bias matrix parameter update process of the convolutional neural network and the following recurrent neural network model is as follows:

[0137] In the first iteration, the weight matrix and bias matrix parameters are randomly initialized. As the number of iterations increases and the number of data samples fed into the network increases, the weight matrix is ​​updated.

[0138] Update the weights and biases of the network, where W represents the weight matrix and b represents the bias matrix;

[0139] The calculation formula for the weight and bias of the network in back-propagation learning is as follows:

[0140]

[0141]

[0142] in, To obtain partial derivatives, l represents the learning rate of the improved CNN convolutional neural network; W t+1 is the weight matrix of the t+1th iteration, W t is the weight matrix of the tth iteration, b t+1 is the bias matrix of the t+1th iteration, b t is the bias matrix of the t-th iteration, Find the partial derivative of E with respect to W, Find the partial derivative of E with respect to b; * is the multiplication sign;

[0143] Obtaining the trained improved CNN convolutional neural network, that is, obtaining the parameters corresponding to the trained improved CNN convolutional neural network;

[0144] Step 3.8: The second fully connected layer has a direct relationship with the output layer and the prediction results;

[0145] The output of the second fully connected layer of the trained improved CNN convolutional neural network is used as the spatial feature extracted by the improved CNN convolutional neural network, which is called the one-dimensional EEG spatial feature X s ; This method can automatically learn the spatial features of one-dimensional EEG signals through CNN convolutional neural network.

[0146] The other steps and parameters are the same as those in the first to third embodiments.

[0147] Specific embodiment 5: This embodiment differs from any one of the specific embodiments 1 to 4 in that in step 4, an improved LSTM recurrent neural network RNN ​​is constructed, and the improved LSTM recurrent neural network RNN ​​is used to extract the EEG time feature of the new EEG signal in step 2 to obtain a one-dimensional EEG time feature X t Based on the basic network model structure, we adjusted the learning rate and the number of LSTM neuron layers to design four network models with different structures. We used classification accuracy, loss rate, ROC curve, and confusion matrix to find the optimal network model for extracting time features.

[0148] The step 4 is specifically as follows:

[0149] Step 4.1: Build an improved LSTM recurrent neural network RNN:

[0150] The improved LSTM recurrent neural network RNN ​​includes an input layer, a first fully connected layer, a second fully connected layer, a third fully connected layer, a first LSTM, a second LSTM, and an output layer in sequence;

[0151] Improve the output data of the i-th layer of the LSTM recurrent neural network RNN ​​structure with X i Indicates that i = 1, 2, ..., 7, the first layer is the input layer, the second layer is the first fully connected layer, the third layer is the second fully connected layer, the fourth layer is the third fully connected layer, the fifth layer is the first LSTM, the sixth layer is the second LSTM, and the seventh layer is the output layer;

[0152] Step 4.2: Input the new EEG signal data sample obtained in step 2 into the input layer of the improved LSTM recurrent neural network RNN. The input layer outputs the EEG signal in the shape of [1, 1, 64].

[0153] 1 is the height, 1 is the width, and 64 is the number of channels;

[0154] Step 4.3: The EEG signal output by the input layer is sequentially input into the first fully connected layer, the second fully connected layer, and the third fully connected layer;

[0155] Use W i,i+1 represents the weight between the i-th layer and the i+1-th layer, b i Represents the deviation of the i-th layer, the data of the i-th layer is X i , then the calculation between the three fully connected layers can be expressed as:

[0156] X i+1 =X i *W i,i+1 +b i

[0157] Step 4.4: The EEG signal output by the third fully connected layer is input into the first LSTM and the second LSTM in sequence;

[0158] The second LSTM outputs the result of LSTM with shape [1,1,5];

[0159] 5 represents the five types of motor imagery tasks;

[0160] Step 4.5: Train the improved LSTM recurrent neural network RNN ​​to obtain the trained improved LSTM recurrent neural network RNN; the process is:

[0161] Use cross entropy as the loss function;

[0162] Optimize the loss function through the AdamOptimizer algorithm;

[0163] Get the trained improved LSTM recurrent neural network RNN;

[0164] Step 4.6: The data X6 output by the second LSTM layer is directly related to the output layer and the prediction result; therefore, the data X6 output by the second LSTM in the trained improved LSTM recurrent neural network RNN ​​obtained in step 4.5 is used as the time feature extracted by the improved LSTM recurrent neural network RNN, which is called the one-dimensional EEG time feature X t ; This method can automatically learn the temporal features of one-dimensional EEG signals through the LSTM structure.

[0165] The other steps and parameters are the same as those in the first to fourth embodiments.

[0166] Specific embodiment 6: This embodiment differs from any one of specific embodiments 1 to 5 in that in step 5, the one-dimensional EEG spatial feature and the one-dimensional EEG temporal feature are spliced ​​to obtain the spliced ​​spatiotemporal feature;

[0167] An automatic encoder (AE) is constructed to reconstruct the spliced ​​spatiotemporal features. The reconstructed features are then input into the classifier to complete the classification and recognition of the motor imagery task.

[0168] The abstract reconstruction representation of the input features is achieved through two steps: encoding and decoding;

[0169] Set different learning rates to compare the reconstruction effect of the AE layer on the input features;

[0170] The specific process is:

[0171] Step 5.1: The one-dimensional EEG spatial feature X obtained in step 3 s And the one-dimensional EEG time feature X obtained in step 4 t Perform splicing to obtain sample X;

[0172] Step 5.2: Design the autoencoder AE; the specific process is:

[0173] Let sample X = R d*n , d is the dimension, n is the dimension;

[0174] The encoding process of sample X is expressed as follows:

[0175] H=g(W m *X+b m )

[0176] Where W m is the coding layer weight matrix, b m is the bias matrix of the coding layer, g is the activation function;

[0177] X is encoded and input into the hidden layer;

[0178] After the code is obtained, it is decoded by the decoder, and the hidden layer output sample H is decoded and output

[0179] The decoding process of sample H is expressed as follows:

[0180]

[0181] Where T represents transpose; W d represents the decoding layer weight matrix, b d represents the bias matrix of the decoding layer, and g represents the activation function;

[0182] Step 5.3: Train the autoencoder AE; the specific process is:

[0183] The training goal of the autoencoder AE is to minimize the loss function. The formula is as follows:

[0184] argminJ(W,b)

[0185] Where J(W,b) represents the loss function; W represents the weight matrix of the autoencoder AE, and b represents the bias matrix of the autoencoder AE;

[0186] In the autoencoder AE, the loss function adopts the cross entropy loss function;

[0187] Assume that the learning rate of the autoencoder AE is L. Like the deep learning network model, the gradient descent and back propagation algorithms are used to update the autoencoder AE parameters.

[0188] Get the trained autoencoder AE.

[0189] The other steps and parameters are the same as those in the first to fifth embodiments.

[0190] Specific embodiment seven: This embodiment differs from any one of specific embodiments one to six in that the classifier is a lightGBM classifier.

[0191] When the lightGBM classifier is used to address the high overhead and overfitting issues of XGBoost, experimental results show that the lightGBM classification method has higher classification accuracy and significantly reduces training time due to memory optimization. Compared with traditional classification algorithms such as support vector machines (SVM), this method has better classification results.

[0192] The other steps and parameters are the same as those in the first to sixth embodiments.

[0193] Specific embodiment eight: In this embodiment, an EEG signal classification and recognition system based on deep learning is used to execute an EEG signal classification and recognition method based on deep learning.

[0194] Example

[0195] Example 1:

[0196] Step 1: Obtain the original EEG signal dataset, i.e. the public dataset: PhysioNet eegmidb (EEG motor movement / image database);

[0197] Preprocessing the original EEG signal data into a one-dimensional original EEG signal feature vector;

[0198] The EEG signals in step 1 and step 2 are labeled EEG signals;

[0199] Step 1.1: Each subject (109) wore a BCI2000 EEG device, sat in front of a computer screen, and performed certain actions or imagined motor tasks according to the prompts on the screen. The BCI2000 EEG device used 64 channels to record each subject's EEG signals at a sampling rate of 160 Hz.

[0200] The EEG signals recorded by the BCI2000 EEG device correspond to the type of motor imagery task undertaken by the subjects;

[0201] The motor imagery task types are divided into five categories:

[0202] The first category is rest;

[0203] The second category is motor imagery of the left hand;

[0204] The third category is motor imagery of the right hand;

[0205] The fourth category is motor imagery of both hands;

[0206] The fifth category is motor imagery of the feet;

[0207] Each subject underwent 14 experiments, and the 14 experiments were as follows:

[0208] 1 time, baseline, eyes open

[0209] 2 times, baseline, eyes closed

[0210] 3 times, Task 1: Open and close the left or right hand;

[0211] 4 times, Task 2: Imagine opening and closing your left or right hand;

[0212] 5 times, Task 3: Open and close your hands or feet;

[0213] 6 times, Task 4: Imagine opening and closing your hands or feet;

[0214] 7 times, Task 1: Open and close the left or right hand;

[0215] 8 times, Task 2: Imagine opening and closing your left or right hand;

[0216] 9 times, Task 3: Open and close your hands or feet;

[0217] 10 times, Task 4: Imagine opening and closing your hands or feet;

[0218] 11 times, Task 1: Open and close the left or right hand;

[0219] 12 times, Task 2: Imagine opening and closing your left or right hand;

[0220] 13 times, Task 3: Open and close your hands or feet;

[0221] 14 times, Task 4: Imagine opening and closing your hands or feet;

[0222] The EEG signals generated by the subjects performing real movements to open and close their left hand were the same as those generated by just imagining opening and closing their left hand. The EEG signals generated by these two methods belong to the same category, namely the second type of movement imagining the left hand;

[0223] The EEG signal dataset includes three label annotation codes T0, T1 or T2;

[0224] T0 represents rest, which corresponds to the entire 1st and 2nd trials (one minute each) and the rest in trials 3–14 for each subject (rest (T0) and motor imagery (T1 or T2) in trials 3–14 were performed alternately for 2 minutes, with a rest period of approximately 4 seconds);

[0225] T1 represents the subjects' imagined or actual opening and closing of their left hand, corresponding to the 3rd, 4th, 7th, 8th, 11th, and 12th trials for each subject (one imagined or actual opening and closing of the left hand lasted approximately 4 seconds); T1 represents the subjects' imagined or actual opening and closing of both hands, corresponding to the 5th, 6th, 9th, 10th, 13th, and 14th trials for each subject (one imagined or actual opening and closing of both hands lasted approximately 4 seconds);

[0226] T2 represents the subjects' imagined or actual opening and closing of their right hand, corresponding to the 3rd, 4th, 7th, 8th, 11th, and 12th trials for each subject (one imagined or actual opening and closing of the right hand lasted approximately 4 seconds); T2 represents the subjects' imagined or actual opening and closing of their feet, corresponding to the 5th, 6th, 9th, 10th, 13th, and 14th trials for each subject (one imagined or actual opening and closing of the feet lasted approximately 4 seconds);

[0227] The experimental process taking subject S001 as an example is as follows:

[0228] In the first experiment, the eyes were open for one minute;

[0229] In the second experiment, the eyes were closed for one minute;

[0230] The third experiment lasted two minutes, in which rest (T0) and motor imagery task (T1 or T2) were performed alternately for a total of 2 minutes. The order in which T1 and T2 alternated with T0 was uncertain. This was the third experiment, so T1 represented the real opening and closing of the left hand, and T2 represented the real opening and closing of the right hand. The specific process was as follows: rest 4s, real opening and closing of the right hand 4s, rest 4s, real opening and closing of the left hand 4s, rest 4s, real opening and closing of the left hand 4s, rest 4s, real opening and closing of the right hand 4s, rest 4s, real opening and closing of the right hand 4s, rest 4s, real opening and closing of the right hand 4s, rest 4s, real Open and close the left hand for 4s, rest for 4s, actually open and close the left hand for 4s, rest for 4s, actually open and close the right hand for 4s, rest for 4s, actually open and close the left hand for 4s, rest for 4s, actually open and close the right hand for 4s, rest for 4s, actually open and close the right hand for 4s, rest for 4s, actually open and close the right hand for 4s, rest for 4s, actually open and close the left hand for 4s, rest for 4s, actually open and close the left hand for 4s, rest for 4s, actually open and close the right hand for 4s, rest for 4s, actually open and close the left hand for 4s; in total, there are 15 rests, 15 actual opening and closings of the right and left hands, 30 total times for a total of 120s;

[0231] The fourth experiment lasted two minutes, in which rest (T0) and motor imagery tasks (T1 or T2) were performed alternately for a total of 2 minutes. The order in which T1 and T2 alternated with T0 was uncertain. This was the fourth experiment, so T1 represented imagining opening and closing the left hand, and T2 represented imagining opening and closing the right hand. The specific process was as follows: rest for 4 seconds, imagine opening and closing the right hand for 4 seconds, rest for 4 seconds, imagine opening and closing the left hand for 4 seconds, rest for 4 seconds, imagine opening and closing the left hand for 4 seconds, rest for 4 seconds, imagine opening and closing the right hand for 4 seconds, rest for 4 seconds, imagine opening and closing the right hand for 4 seconds, rest for 4 seconds, imagine opening and closing the right hand for 4 seconds, rest for 4 seconds, imagine Open and close left hand for 4 seconds, rest for 4 seconds, imagine opening and closing right hand for 4 seconds, rest for 4 seconds, imagine opening and closing left hand for 4 seconds, rest for 4 seconds, imagine opening and closing right hand for 4 seconds, rest for 4 seconds, imagine opening and closing left hand for 4 seconds, rest for 4 seconds, imagine opening and closing left hand for 4 seconds, rest for 4 seconds, imagine opening and closing right hand for 4 seconds, rest for 4 seconds, imagine opening and closing left hand for 4 seconds; 15 times in total, rest, imagine opening and closing right and left hands for 15 times in total, 30 times in total, 120 seconds in total;

[0232] The fifth experiment lasted two minutes, in which rest (T0) and motor imagery task (T1 or T2) were performed alternately for a total of 2 minutes. The order in which T1 and T2 alternated with T0 was uncertain. This was the fifth experiment, so T1 represented the real opening and closing of the hands, and T2 represented the real opening and closing of the feet. The specific process was as follows: rest 4s, real opening and closing of hands, rest 4s, real opening and closing of feet 4s, rest 4s, real opening and closing of hands 4s, rest 4s, real opening and closing of feet 4s, rest 4s, real opening and closing of hands 4s, rest 4s, real opening and closing of feet 4s, Open and close the legs for 4 seconds, rest for 4 seconds, actually open and close the legs for 4 seconds, rest for 4 seconds, actually open and close the hands for 4 seconds, rest for 4 seconds, actually open and close the hands for 4 seconds, rest for 4 seconds, actually open and close the legs for 4 seconds, rest for 4 seconds, actually open and close the legs for 4 seconds, rest for 4 seconds, actually open and close the legs for 4 seconds, rest for 4 seconds, actually open and close the legs for 4 seconds, rest for 4 seconds, actually open and close the legs for 4 seconds; in total, 15 rests, 15 actual opening and closings of the legs and hands, 30 reps for a total of 120 seconds;

[0233] The sixth experiment lasted two minutes, in which rest (T0) and motor imagery tasks (T1 or T2) were performed alternately for a total of 2 minutes. The order in which T1 and T2 alternated with T0 was uncertain. This was the sixth experiment, so T1 represented imagining opening and closing the hands, and T2 represented imagining opening and closing the feet. The specific process was as follows: rest for 4 seconds, imagine opening and closing the hands for 4 seconds, rest for 4 seconds, imagine opening and closing the feet for 4 seconds, rest for 4 seconds, imagine opening and closing the feet for 4 seconds, rest for 4 seconds, imagine opening and closing the hands for 4 seconds, rest for 4 seconds, imagine opening and closing the feet for 4 seconds, rest for 4 seconds, imagine opening and closing the hands for 4 seconds, rest for 4 seconds, imagine opening and closing the feet for 4 seconds, rest for 4 seconds, imagine Open and close your hands for 4 seconds, rest for 4 seconds, imagine opening and closing your hands for 4 seconds, rest for 4 seconds, imagine opening and closing your feet for 4 seconds, rest for 4 seconds, imagine opening and closing your feet for 4 seconds, rest for 4 seconds, imagine opening and closing your hands for 4 seconds, rest for 4 seconds, imagine opening and closing your hands for 4 seconds, rest for 4 seconds, imagine opening and closing your feet for 4 seconds, rest for 4 seconds, imagine opening and closing your feet for 4 seconds, rest for 4 seconds, imagine opening and closing your hands for 4 seconds, rest for 4 seconds, imagine opening and closing your hands for 4 seconds; a total of 15 rests, imagine opening and closing your hands and feet for a total of 15 times, a total of 30 times for a total of 120 seconds;

[0234] Experiments 7-14 were similar;

[0235] The 14 experiments for other subjects followed a similar procedure;

[0236] Step 1.2: 10 subjects were randomly selected from the subjects, and a total of N (280,000) labeled EEG signals were selected from the 10 subjects (28,000 samples per subject); each EEG signal sample is a vector of 64 elements, each element corresponding to a channel of the EEG data;

[0237] The EEG signal data is represented as a one-dimensional vector; the signal feature dimension is [1,64], where 64 is the number of electrode channels in the original EEG signal and 1 is a time point;

[0238] Next, input is given to the RNN structure and CNN structure for feature learning;

[0239] Step 2: Since EEG signal acquisition is difficult and the size of the dataset is a key factor affecting the accuracy of the deep learning network, the original dataset is expanded;

[0240] The EMD decomposition algorithm is used to expand the one-dimensional original EEG signal obtained in step 1 to obtain a new EEG signal data sample;

[0241] The EEG signals in step 1 and step 2 are labeled EEG signals;

[0242] Verify the improved effect of data augmentation in the improved deep learning model proposed in steps 3 and 4;

[0243] The specific process is:

[0244] Step 2.1: Perform EMD decomposition on the one-dimensional original EEG signal data of the five types of motor imagery obtained in step 1 to obtain the IMF components of different channels of each type of motor imagery EEG signal sample;

[0245] Step 2.2: Randomly select the IMF components of different channels of each motor imagery EEG signal sample belonging to the same category, and perform the same processing on the training set and the test set;

[0246] Step 2.3: Superimpose the IMF components of the randomly selected corresponding channels (the same channel). The sum of the components can obtain the properties of the original EEG signal and obtain a new EEG signal data sample;

[0247] After the data is expanded, the classification effects on the improved convolutional neural network and the improved recurrent neural network are compared before and after data enhancement.

[0248] Step 3: Construct an improved CNN convolutional neural network and use it to extract EEG spatial features from the new EEG signal data samples obtained in step 2 to obtain a one-dimensional EEG spatial feature X sIn the basic model, we adjusted the number of convolutional layers, pooling layers, and learning rate, and used classification accuracy, loss rate, ROC curve, and confusion matrix to identify the optimal network structure, and analyzed and compared the experimental results.

[0249] The specific process is:

[0250] Step 3.1: Build an improved CNN convolutional neural network:

[0251] The improved CNN convolutional neural network includes: input layer, first convolution layer, first pooling layer, second convolution layer, second pooling layer, first fully connected layer, second fully connected layer and output layer;

[0252] Improve the output data of the i′th layer of the CNN convolutional neural network with X i′ Indicates that i′=1, 2, …, 8, the first layer is the input layer, the second layer is the first convolutional layer, the third layer is the first pooling layer, the fourth layer is the second convolutional layer, the fifth layer is the second pooling layer, the sixth layer is the first fully connected layer, the seventh layer is the second fully connected layer, and the eighth layer is the output layer;

[0253] Step 3.2: Input the new EEG signal data sample obtained in step 2 into the input layer of the improved CNN convolutional neural network. The input layer outputs the EEG signal in the shape of [1,64].

[0254] 1 is the dimension, 64 is the number of electrode channels;

[0255] Step 3.3: First convolutional layer operation:

[0256] Set the convolution filter with kernel size [1,1] and stride size [1,1], where stride represents the x- and y-movement distances of the convolution kernel.

[0257] The padding method is chosen to be zero padding of the same shape, which keeps the sample shape unchanged in the convolution calculation;

[0258] After the first convolutional layer operation, the shape of X2 is [1, 64, 2], where 1 is the height, 64 is the width, and 2 is the number of channels;

[0259] The Relu activation function is then used to process the convolution result. It has sparseness and can effectively alleviate the vanishing gradient. The mathematical expression of the activation function Relu is as follows:

[0260]

[0261] Step 3.4: First pooling layer operation:

[0262] Maximize pooling to reduce the spatial size of input EEG features and reduce overfitting;

[0263] Set the pooling window size to [1, 2] and the stride size to [1, 2], without changing the depth. The stride represents the x- and y-movement distances of the pooling window.

[0264] After the first pooling layer operation, the shape of X3 is [1,32,2]; 1 is the height, 32 is the width, and 2 is the number of channels;

[0265] Step 3.5: Second convolutional layer operation: Set the convolution filter with kernel size [1, 2] and stride size [1, 1];

[0266] After the second convolutional layer operation, the shape is [1,32,4]; 1 is the height, 32 is the width, and 4 is the number of channels;

[0267] Then use the Relu activation function;

[0268] Step 3.6: Second pooling layer operation: set the pooling window size to [1, 2] and the stride size to [1, 2];

[0269] After the second pooling layer operation, the shape is [1,16,4]; 1 is the height, 16 is the width, and 4 is the number of channels;

[0270] Step 3.7: First fully connected layer operation: Calculate the output data according to the conventional neural network operation;

[0271] Step 3.6: The output data of shape [1, 16, 4] is flattened into a vector of shape [1, 64].

[0272] Step 3.8: Second fully connected layer operation:

[0273] The first fully connected layer operation outputs data of shape [1,64] which is flattened into a vector of shape [1,120];

[0274] Finally, the output is the result of the improved CNN convolutional neural network with a shape of [1,5];

[0275] 5 represents the five types of motor imagery tasks;

[0276] Step 3.9: Train the improved CNN convolutional neural network. The specific process is as follows:

[0277] Use cross entropy loss function; optimize the loss function through AdamOptimizer algorithm;

[0278] The weight matrix and bias matrix parameter update process of the convolutional neural network and the following recurrent neural network model is as follows:

[0279] In the first iteration, the weight matrix and bias matrix parameters are randomly initialized. As the number of iterations increases and the number of data samples fed into the network increases, the weight matrix is ​​updated.

[0280] Update the weights and biases of the network, where W represents the weight matrix and b represents the bias matrix;

[0281] The calculation formula for the weight and bias of the network in back-propagation learning is as follows:

[0282]

[0283]

[0284] in, To obtain partial derivatives, l represents the learning rate of the improved CNN convolutional neural network; W t+1 is the weight matrix of the t+1th iteration, W t is the weight matrix of the tth iteration, b t+1 is the bias matrix of the t+1th iteration, b t is the bias matrix of the t-th iteration, Find the partial derivative of E with respect to W, Find the partial derivative of E with respect to b; * is the multiplication sign;

[0285] Step 3.10: Based on the basic network designed above, design network models with different numbers of convolutional and pooling layers to try to find the optimal network structure. At the same time, adjust the network learning rate and various parameters to achieve the desired classification and feature extraction results.

[0286] The second fully connected layer has a direct relationship with the output layer and the prediction results; therefore, the output of the second fully connected layer in the optimal network structure obtained in step 3.10 is used as the spatial feature extracted by the improved CNN convolutional neural network, which is called the one-dimensional EEG spatial feature X s ; This method can automatically learn the spatial features of one-dimensional EEG signals through CNN convolutional neural network.

[0287] Step 4: Construct an improved LSTM recurrent neural network RNN ​​and use it to extract the EEG time feature of the new EEG signal in step 2 to obtain a one-dimensional EEG time feature X t Based on the basic network model structure, we adjusted the learning rate and the number of LSTM neuron layers to design four network models with different structures. We used classification accuracy, loss rate, ROC curve, and confusion matrix to find the optimal network model for extracting time features.

[0288] The step 4 is specifically as follows:

[0289] Step 4.1: Build an improved LSTM recurrent neural network RNN:

[0290] The improved LSTM recurrent neural network RNN ​​includes an input layer, a first fully connected layer, a second fully connected layer, a third fully connected layer, a first LSTM, a second LSTM, and an output layer in sequence;

[0291] Improve the output data of the i-th layer of the LSTM recurrent neural network RNN ​​structure with X i Indicates that i = 1, 2, ..., 7, the first layer is the input layer, the second layer is the first fully connected layer, the third layer is the second fully connected layer, the fourth layer is the third fully connected layer, the fifth layer is the first LSTM, the sixth layer is the second LSTM, and the seventh layer is the output layer;

[0292] Step 4.2: Input the new EEG signal data sample obtained in step 2 into the input layer of the improved LSTM recurrent neural network RNN. The input layer outputs the EEG signal in the shape of [1, 1, 64].

[0293] 1 is the height, 1 is the width, and 64 is the number of channels;

[0294] Step 4.3: The EEG signal output by the input layer is sequentially input into the first fully connected layer, the second fully connected layer, and the third fully connected layer;

[0295] Use W i,i+1 represents the weight between the i-th layer and the i+1-th layer, b i Represents the deviation of the i-th layer, the data of the i-th layer is X i , then the calculation between the three fully connected layers can be expressed as:

[0296] X i+1 =X i *W i,i+1 +b i

[0297] Step 4.4: The EEG signal output by the third fully connected layer is input into the first LSTM and the second LSTM in sequence;

[0298] The second LSTM outputs the result of LSTM with shape [1,1,5];

[0299] 5 represents the five types of motor imagery tasks;

[0300] Step 4.5: Train the improved LSTM recurrent neural network RNN ​​to obtain the trained improved LSTM recurrent neural network RNN; the process is:

[0301] Use cross entropy as the loss function;

[0302] Optimize the loss function through the AdamOptimizer algorithm;

[0303] Get the trained improved LSTM recurrent neural network RNN;

[0304] Step 4.6: Based on the basic network designed above, design network models with different learning rates and numbers of LSTM layers to try to find an optimal network structure to achieve the expected classification and feature extraction results.

[0305] Step 4.7: The data X6 output by the second LSTM layer is directly related to the output layer and the prediction result; therefore, the data X6 output by the second LSTM in the trained improved LSTM recurrent neural network RNN ​​obtained in step 4.6 is used as the time feature extracted by the improved LSTM recurrent neural network RNN, which is called the one-dimensional EEG time feature X t ; This method can automatically learn the temporal features of one-dimensional EEG signals through the LSTM structure.

[0306] Step 5: Concatenate the one-dimensional EEG spatial features and the one-dimensional EEG temporal features to obtain the concatenated spatiotemporal features;

[0307] An automatic encoder (AE) is constructed to reconstruct the spliced ​​spatiotemporal features. The reconstructed features are then input into the classifier to complete the classification and recognition of the motor imagery task.

[0308] The abstract reconstruction representation of the input features is achieved through two steps: encoding and decoding;

[0309] Set different learning rates to compare the reconstruction effect of the AE layer on the input features;

[0310] The specific process is:

[0311] Step 5.1: The one-dimensional EEG spatial feature X obtained in step 3 s And the one-dimensional EEG time feature X obtained in step 4 t Perform splicing to obtain sample X;

[0312] Step 5.2: Design the autoencoder AE; the specific process is:

[0313] Let sample X = R d*n , d is the dimension, n is the dimension;

[0314] The encoding process of sample X is expressed as follows:

[0315] H=g(W m *X+b m )

[0316] Where W mis the coding layer weight matrix, b m is the bias matrix of the coding layer, g is the activation function;

[0317] X is encoded and input into the hidden layer;

[0318] After the code is obtained, it is decoded by the decoder, and the hidden layer output sample H is decoded and output

[0319] The decoding process of sample H is expressed as follows:

[0320]

[0321] Where T represents transpose; W d represents the decoding layer weight matrix, b d represents the bias matrix of the decoding layer, and g represents the activation function;

[0322] Step 5.3: Train the autoencoder AE; the specific process is:

[0323] The training goal of the autoencoder AE is to minimize the loss function. The formula is as follows:

[0324] argminJ(W,b)

[0325] Where J(W,b) represents the loss function; W represents the weight matrix of the autoencoder AE, and b represents the bias matrix of the autoencoder AE;

[0326] In the autoencoder AE, the loss function adopts the cross entropy loss function;

[0327] Assume that the learning rate of the autoencoder AE is L. Like the deep learning network model, the gradient descent and back propagation algorithms are used to update the autoencoder AE parameters.

[0328] Step 5.4: Verify the effect of learning rate on the learning effect of autoencoder AE by setting two relatively high and low learning rates, and try to find an optimal autoencoder AE and corresponding classifier to achieve a classification effect and feature extraction effect that meets the expectations.

[0329] Step 6: Compare the classification and recognition effects of the motor imagery task using the XGBoost classifier and the lightGBM classifier;

[0330] Example 2:

[0331] Because EEG signal acquisition is difficult and dataset size is a key factor affecting the accuracy of deep learning networks, the original dataset was augmented. The EMD decomposition algorithm was used to perform data transformations based on the original training data to augment the dataset. The improved deep learning model proposed in steps 2 and 3 was used to verify the effectiveness of data augmentation.

[0332] After the data is expanded, the classification effects on the improved convolutional neural network and the improved recurrent neural network are compared before and after data augmentation;

[0333] 1: Compare the classification effects of the improved convolutional neural network before and after data enhancement through the accuracy and loss rate curves; the experimental results are as follows Figure 5 , you can see Figure 5 The data augmentation method (dashed line) is used to enhance the data set input to the network, which has a significant effect on improving the CNN network.

[0334] 2: Compare the classification effects of the improved recurrent neural network before and after data augmentation through the accuracy and loss rate curves; the experimental results are as follows Figure 6 As shown in the figure, the improved RNN network performance is improved, but the effect is not very obvious;

[0335] 3: Comparison of the classification effects of the improved convolutional neural network and the improved recurrent neural network before and after data augmentation through confusion matrix;

[0336] The confusion matrix results before and after the final autoencoder classification are as follows Figure 7a 、 7b As shown, Figure 7a Before the enhancement, Figure 7b After the data enhancement algorithm is used for classification and recognition, the confusion matrix can be used to visually show the improvement in data classification. The classification results of the first, second, third, and fourth categories have all been slightly improved compared to those before the data enhancement algorithm was used.

[0337] Example 3:

[0338] Step 3: Use the improved CNN convolutional neural network to extract data features from EEG data. In the basic model Figure 1 In the experiment, we adjusted the number of convolutional layers, pooling layers, and learning rate, and used classification accuracy, loss rate, ROC curve, and confusion matrix to identify the optimal network structure, and then analyzed and compared the experimental results.

[0339] Step 3.1: In the spatial feature extraction of EEG signals, the present invention designs network models with different numbers of convolution and pooling layers, compares the classification accuracy and loss rate, and attempts to find an optimal network structure;

[0340] Among the various network structures, the one with relatively few convolutional and pooling layers performed best. The network with four convolutional and pooling layers achieved the lowest classification accuracy, reaching approximately 58%. This suggests that the network does not need to over-learn features; excessive convolutional and pooling layers only affect the extraction of spatial features in EEG signals. The network with only one convolutional and one pooling layer achieved the highest classification accuracy, fluctuating around 71%. Based on this, parameter adjustments were made to this optimal network structure.

[0341] Step 3..2: Compare the ROC curves of the network models with different numbers of convolution and pooling layers in step 2.9.1 and verify again;

[0342] According to the ROC curve comparison of the designed network model, it can be seen that the convolutional neural network structure designed by the present invention can effectively classify EEG signals. The network structure with one layer of convolution and one layer of pooling achieves the highest classification accuracy and the largest area under the curve. At the same time, due to the high classification accuracy, it can be determined that the proposed network model can effectively extract the spatial features of EEG signals.

[0343] Step 3.3: Adjust the network learning rate under the above optimal network structure to achieve a classification effect and feature extraction effect that meets the expectations;

[0344] We compared the network's performance by setting different learning rate values. When the learning rate was higher than 0.01, the classification accuracy was suboptimal. Therefore, we tested the network's classification performance using four learning rate values: 0.001, 0.003, 0.005, and 0.007. Experiments showed that a learning rate of 0.003 achieved the best classification performance.

[0345] Step 3.4: Combining the experiments in the previous steps, the optimal model is obtained. The confusion matrix classified by this model on the validation set is as follows:

[0346]

[0347] In the confusion matrix for the five categories, we can see that the classification accuracy for the first category is 84.64%, the second category (left hand) is 68.43%, the third category (right hand) is 67.07%, the fourth category (hands) is 71.35%, and the fifth category (feet) is 76.02%. The accuracy and precision for each category are relatively high, indicating that the extracted spatial features are stable and effective.

[0348] Step 4: Use the improved LSTM recurrent neural network to extract data features from EEG data, based on the basic network model structure Figure 2 We designed four network models with different structures by adjusting the learning rate and the number of LSTM neuron layers, and used classification accuracy, loss rate, ROC curve, and confusion matrix to find the optimal network model for extracting temporal features.

[0349] Step 4.1: In the extraction of temporal features of EEG signals, the present invention designs network models with different learning rates and numbers of LSTM layers, compares the classification accuracy and loss rate, and attempts to find an optimal network structure;

[0350] The experimental results are as follows Figure 3 Among all the network structures, the one with two layers of LSTM neurons at a learning rate of 0.07 has the best performance.

[0351] Using a standard RNN network structure without LSTM neurons, classification accuracy hovered around 60%. By appropriately increasing the network's learning rate, classification accuracy improved to around 70%. To address this low accuracy, two layers of LSTM neurons were added to the original architecture to address long-term dependencies, bringing the accuracy to around 90%. Further adjustments to the network's learning rate resulted in a 4% improvement in accuracy.

[0352] Step 4.2: Compare the ROC curves of the network models with different learning rates and numbers of LSTM layers in step 4.1 and verify again;

[0353] The ROC curves of the designed network models show that the recurrent neural network structure designed in the present invention can effectively classify EEG signals. When the ROC curve is 0.07, the network structure with two layers of LSTM neurons achieves the highest classification accuracy and the largest area under the curve. At the same time, due to the high classification accuracy, it can be determined that the proposed network model can effectively extract the temporal characteristics of EEG signals.

[0354] Step 4.3: Combining the experiments in the previous steps, we get the optimal model. The confusion matrix classified by this model on the validation set is as follows:

[0355]

[0356] In the confusion matrix for the five categories, we can see that the classification accuracy for the first category is 97.03%, the second category (left hand) is 93.4%, the third category (right hand) is 92.67%, the fourth category (hands) is 95.0%, and the fifth category (feet) is 95.2%. The accuracy and precision for each category are relatively high, indicating that the extracted spatial features are stable and effective.

[0357] Step 5: In the improved convolutional neural network model designed in step 3, the extracted EEG spatial features can be obtained and set as X s In the improved recurrent neural network RNN ​​designed in step 4, the extracted EEG time feature can be obtained and set as X t Directly concatenate the one-dimensional spatiotemporal features and design an autoencoder to reconstruct the superposition of spatiotemporal features. Through the two steps of encoding and decoding, the abstract reconstruction representation of the input features is realized. Different learning rates are set to compare the reconstruction effect of AE on the input features.

[0358] Assume that the learning rate L1 = 0.05 and L2 = 0.09. The effect of the two on AE superposition is compared as follows: Figure 4 As shown in the figure, it can be seen that when the learning rate is set to 0.05 (dashed line), AE has a better reconstruction effect on the input features.

[0359] Step 6: Compare the classification and recognition effects of the motor imagery task using the XGBoost classifier and the lightGBM classifier;

[0360] By comparing the ROC curves, when the lightGBM classifier is selected to solve the problem of XGBoost's high overhead and prone to overfitting, lightGBM is closer to the upper left corner on the ROC curve than XGBoost, so it is concluded that the lightGBM method has better classification performance;

[0361] like Figure 9 The final confusion matrix results show that the lightGBM classification method adopted by the present invention has a high classification accuracy. Thanks to the memory optimization of the lightGBM classifier, the training time has also been greatly reduced.

[0362] A login interface of an EEG signal classification and recognition system based on deep learning Figure 9 As shown, the CNN network model parameter debugging interface is as follows Figure 10 As shown, the final EEG signal classification interface Figure 11 shown.

[0363] The present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for classifying and recognizing EEG signals based on deep learning, characterized by: The specific process of the method is: Step 1: Obtain the original EEG signal dataset, that is, the public dataset: PhysioNet eegmidb database; Preprocessing the original EEG signal data into a one-dimensional original EEG signal feature vector; Step 2: Use the EMD decomposition algorithm to expand the one-dimensional original EEG signal obtained in step 1 to obtain a new EEG signal data sample; Step 3: Construct an improved CNN convolutional neural network and use it to extract EEG spatial features from the new EEG signal data samples obtained in step 2 to obtain a one-dimensional EEG spatial feature X s ; Step 4: Construct an improved LSTM recurrent neural network RNN ​​and use it to extract the EEG time feature of the new EEG signal in step 2 to obtain a one-dimensional EEG time feature X t ; Step 5: Concatenate the one-dimensional EEG spatial features and the one-dimensional EEG temporal features to obtain the concatenated spatiotemporal features; An automatic encoder (AE) is constructed to reconstruct the spliced ​​spatiotemporal features. The reconstructed features are then input into the classifier to complete the classification and recognition of the motor imagery task. Step 6: The subject wears the BCI2000 EEG device to collect the subject's EEG signal to be measured, and the EEG signal to be measured is represented as a one-dimensional vector; The one-dimensional vector is input into the trained improved CNN convolutional neural network, and the output of the second fully connected layer of the improved CNN convolutional neural network is used as the spatial feature extracted by the improved CNN convolutional neural network, which is called the one-dimensional EEG spatial feature; The one-dimensional vector is input into the trained improved LSTM recurrent neural network RNN, and the data output by the second LSTM of the improved LSTM recurrent neural network RNN ​​is used as the time feature extracted by the improved LSTM recurrent neural network RNN, which is called the one-dimensional EEG time feature; The obtained one-dimensional EEG spatial features and the obtained one-dimensional EEG temporal features are spliced ​​together to obtain a sample; The obtained samples are input into the trained autoencoder AE, and the output features of the autoencoder AE are input into the classifier to complete the classification and recognition of the motor imagery task.

2. The method for classifying and recognizing EEG signals based on deep learning according to claim 1, characterized in that: In step 1, the original EEG signal dataset is obtained, i.e., the public dataset: PhysioNet eegmidb database; Preprocessing the original EEG signal data into a one-dimensional original EEG signal feature vector; The specific process is: Step 1.1: Each subject wore a BCI2000 EEG device, sat in front of a computer screen, and performed certain actions or imagined motor tasks according to the prompts on the screen. The BCI2000 EEG device used 64 channels to record each subject's EEG signals at a sampling rate of 160 Hz. The EEG signals recorded by the BCI2000 EEG device correspond to the type of motor imagery task undertaken by the subjects; The motor imagery task types are divided into five categories: The first category is rest; The second category is motor imagery of the left hand; The third category is motor imagery of the right hand; The fourth category is motor imagery of both hands; The fifth category is motor imagery of the feet; Each subject underwent 14 experiments, and the 14 experiments were as follows: 1 time, baseline, eyes open 2 times, baseline, eyes closed 3 times, Task 1: Open and close the left or right hand; 4 times, Task 2: Imagine opening and closing your left or right hand; 5 times, Task 3: Open and close your hands or feet; 6 times, Task 4: Imagine opening and closing your hands or feet; 7 times, Task 1: Open and close the left or right hand; 8 times, Task 2: Imagine opening and closing your left or right hand; 9 times, Task 3: Open and close your hands or feet; 10 times, Task 4: Imagine opening and closing your hands or feet; 11 times, Task 1: Open and close the left or right hand; 12 times, Task 2: Imagine opening and closing your left or right hand; 13 times, Task 3: Open and close your hands or feet; 14 times, Task 4: Imagine opening and closing your hands or feet; The EEG signal dataset includes three label annotation codes T0, T1 or T2; T0 represents rest, which corresponds to the rest during the entire 1st and 2nd experiments and experiments 3–14 for each subject; T1 represents the subjects’ imagined or actual opening and closing of their left hand, corresponding to the 3rd, 4th, 7th, 8th, 11th, and 12th trials for each subject; T1 represents the subjects’ imagined or actual opening and closing of both hands, corresponding to the 5th, 6th, 9th, 10th, 13th, and 14th trials for each subject; T2 represents the subjects’ imagined or actual opening and closing of their right hand, corresponding to the 3rd, 4th, 7th, 8th, 11th, and 12th trials for each subject; T2 represents the subjects’ imagined or actual opening and closing of their feet, corresponding to the 5th, 6th, 9th, 10th, 13th, and 14th trials for each subject; Step 1.2: 10 subjects were randomly selected from the subjects, and a total of N labeled EEG signals were selected from the 10 subjects; each EEG signal sample is a vector of 64 elements, and each element corresponds to a channel of the EEG data; Represent EEG signal data as a one-dimensional vector; The signal feature dimension is [1,64], where 64 is the number of electrode channels in the original EEG signal and 1 is a time point.

3. The method for classifying and recognizing EEG signals based on deep learning according to claim 2, characterized in that: In step 2, the EMD decomposition algorithm is used to expand the one-dimensional original EEG signal obtained in step 1 to obtain a new EEG signal data sample. The specific process is as follows: Step 2.1: Perform EMD decomposition on the one-dimensional original EEG signal data of the five types of motor imagery obtained in step 1 to obtain the IMF components of different channels of each type of motor imagery EEG signal sample; Step 2.2: Randomly select the IMF components of different channels of each motor imagery EEG signal sample belonging to the same category; Step 2.3: Superimpose the IMF components of the randomly selected corresponding channels. The sum of the components can obtain the properties of the original EEG signal and obtain a new EEG signal data sample.

4. The method for classifying and recognizing EEG signals based on deep learning according to claim 3, wherein: In step 3, an improved CNN convolutional neural network is constructed, and the improved CNN convolutional neural network is used to extract the EEG spatial features of the new EEG signal data sample obtained in step 2 to obtain a one-dimensional EEG spatial feature X s ; The specific process is: Step 3.1: Build an improved CNN convolutional neural network: The improved CNN convolutional neural network includes: input layer, first convolution layer, first pooling layer, first fully connected layer, second fully connected layer and output layer; Improve the output data of the i′th layer of the CNN convolutional neural network with X i′ Indicates that i′=1, 2, …, 6, the first layer is the input layer, the second layer is the first convolutional layer, the third layer is the first pooling layer, the fourth layer is the first fully connected layer, the fifth layer is the second fully connected layer, and the sixth layer is the output layer; Step 3.2: Input the new EEG signal data sample obtained in step 2 into the input layer of the improved CNN convolutional neural network. The input layer outputs the EEG signal in the shape of [1,64]. 1 is the dimension, 64 is the number of electrode channels; Step 3.3: First convolutional layer operation: Set the convolution filter with kernel size [1,1] and stride size [1,1]; After the first convolutional layer operation, the shape of X2 is [1, 64, 2], where 1 is the height, 64 is the width, and 2 is the number of channels. The convolution result is then processed using the ReLU activation function. Step 3.4: First pooling layer operation: Set the pooling window size to [1, 2] and the stride size to [1, 2]; After the first pooling layer operation, the shape of X3 is [1,32,2]; 1 is the height, 32 is the width, and 2 is the number of channels; Step 3.5: First fully connected layer operation: Step 3.4: The output data of shape [1,32,2] is flattened into a vector of shape [1,64]. Step 3.6: Second fully connected layer operation: The first fully connected layer operation outputs data of shape [1,64] which is flattened into a vector of shape [1,120]; Finally, the output layer obtains the result of the improved CNN convolutional neural network with a shape of [1,5]; 5 represents the five types of motor imagery tasks; Step 3.7: Train the improved CNN convolutional neural network. The specific process is as follows: Use cross entropy loss function; optimize the loss function through AdamOptimizer algorithm; The parameter update process of the weight matrix and bias matrix is ​​as follows: In the first iteration, the weight matrix and bias matrix parameters are randomly initialized; Update the weights and biases of the network, where W represents the weight matrix and b represents the bias matrix; The calculation formula for the weight and bias of the network in back-propagation learning is as follows: in, To obtain partial derivatives, l represents the learning rate of the improved CNN convolutional neural network; W t+1 is the weight matrix of the t+1th iteration, W t is the weight matrix of the tth iteration, b t+1 is the bias matrix of the t+1th iteration, b t is the bias matrix of the t-th iteration, Find the partial derivative of E with respect to W, Find the partial derivative of E with respect to b; * is the multiplication sign; Obtaining the trained improved CNN convolutional neural network, that is, obtaining the parameters corresponding to the trained improved CNN convolutional neural network; Step 3.8: The output of the second fully connected layer of the trained improved CNN convolutional neural network is used as the spatial feature extracted by the improved CNN convolutional neural network, which is called the one-dimensional EEG spatial feature X s .

5. The method for classifying and recognizing EEG signals based on deep learning according to claim 4, characterized in that: In step 4, an improved LSTM recurrent neural network RNN ​​is constructed, and the improved LSTM recurrent neural network RNN ​​is used to extract the EEG time feature of the new EEG signal in step 2 to obtain a one-dimensional EEG time feature X t ; The specific process is: Step 4.1: Build an improved LSTM recurrent neural network RNN: The improved LSTM recurrent neural network RNN ​​includes an input layer, a first fully connected layer, a second fully connected layer, a third fully connected layer, a first LSTM, a second LSTM, and an output layer in sequence; Improve the output data of the i-th layer of the LSTM recurrent neural network RNN ​​structure with X i Indicates that i = 1, 2, ..., 7, the first layer is the input layer, the second layer is the first fully connected layer, the third layer is the second fully connected layer, the fourth layer is the third fully connected layer, the fifth layer is the first LSTM, the sixth layer is the second LSTM, and the seventh layer is the output layer; Step 4.2: Input the new EEG signal data sample obtained in step 2 into the input layer of the improved LSTM recurrent neural network RNN. The input layer outputs the EEG signal in the shape of [1, 1, 64]. 1 is the height, 1 is the width, and 64 is the number of channels; Step 4.3: The EEG signal output by the input layer is sequentially input into the first fully connected layer, the second fully connected layer, and the third fully connected layer; Use W i,i+1 represents the weight between the i-th layer and the i+1-th layer, b i Represents the deviation of the i-th layer, the data of the i-th layer is X i , then the calculation between the three fully connected layers is expressed as: X i+1 =X i *W i,i+1 +b i Step 4.4: The EEG signal output by the third fully connected layer is input into the first LSTM and the second LSTM in sequence; The second LSTM outputs the result of LSTM with shape [1,1,5]; 5 represents the five types of motor imagery tasks; Step 4.5: Train the improved LSTM recurrent neural network RNN ​​to obtain the trained improved LSTM recurrent neural network RNN; the process is: Use cross entropy as the loss function; Optimize the loss function through the AdamOptimizer algorithm; Get the trained improved LSTM recurrent neural network RNN; Step 4.6: The data X6 output by the second LSTM in the trained improved LSTM recurrent neural network RNN ​​obtained in step 4.5 is used as the time feature extracted by the improved LSTM recurrent neural network RNN, which is called the one-dimensional EEG time feature X t .

6. The method for classifying and recognizing EEG signals based on deep learning according to claim 5, characterized in that: In step 5, the one-dimensional EEG spatial feature and the one-dimensional EEG temporal feature are spliced ​​to obtain a spliced ​​spatiotemporal feature; An automatic encoder (AE) is constructed to reconstruct the spliced ​​spatiotemporal features. The reconstructed features are then input into the classifier to complete the classification and recognition of the motor imagery task. The specific process is: Step 5.1: The one-dimensional EEG spatial feature X obtained in step 3 s And the one-dimensional EEG time feature X obtained in step 4 t Perform splicing to obtain sample X; Step 5.2: Design the autoencoder AE; the specific process is: The encoding process of sample X is expressed as follows: H=g(W m *X+b m ) Where W m is the coding layer weight matrix, b m is the bias matrix of the coding layer, g is the activation function; X is encoded and input into the hidden layer; The hidden layer output sample H is decoded and output The decoding process of sample H is expressed as follows: Where T represents transpose; W d represents the decoding layer weight matrix, b d represents the bias matrix of the decoding layer, and g represents the activation function; Step 5.3: Train the autoencoder AE; the specific process is: The training goal of the autoencoder AE is to minimize the loss function. The formula is as follows: argminJ(W,b) Where J(W,b) represents the loss function; W represents the weight matrix of the autoencoder AE, and b represents the bias matrix of the autoencoder AE; In the autoencoder AE, the loss function adopts the cross entropy loss function; Assume that the learning rate of the autoencoder AE is L, and the gradient descent and back propagation algorithms are used to update the autoencoder AE parameters; Get the trained autoencoder AE.

7. The method for classifying and recognizing EEG signals based on deep learning according to claim 6, characterized in that: The classifier is a lightGBM classifier.

8. A deep learning-based EEG signal classification and recognition system, characterized in that: The system is used to execute the deep learning-based EEG signal classification and recognition method described in any one of claims 1 to 7.

Images