A method for classifying EEG data based on deep learning and image features

By constructing the Image-EEGNet model and combining deep learning and recurrent neural networks, the problem of insufficient feature extraction from EEG data was solved, achieving better image feature extraction and classification results and enhancing the model's generalization performance.

CN113988122BActive Publication Date: 2024-11-15HANGZHOU DIANZI UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202111212519.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-19
Publication Date
2024-11-15
Estimated Expiration
2041-10-19

AI Technical Summary

Technical Problem

Existing technologies lack sufficient methods for extracting features from EEG data, making it difficult to effectively extract and classify image features from EEG data, resulting in poor classification performance when EEG data is correlated with image data.

Method used

By employing a deep learning-based image feature extractor and classifier, combined with convolutional neural networks and recurrent neural networks, the Image-EEGNet model is constructed to achieve image feature extraction and classification of EEG data.

Benefits of technology

It improves the correlation and classification effect between EEG data and image data, enhances the generalization performance of the model, and enables better extraction and classification of image features from EEG data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113988122B_ABST
    Figure CN113988122B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for classifying electroencephalogram (EEG) data based on deep learning and image features. Using EEGnet as a feature classification method and an image feature learning network as an important processing method, the features are classified by extracting and learning image features of EEG signals. Obtain EEG data and preprocess them; use pictures viewed by subjects for data preprocessing, and use EEGnet to train the image data to obtain an image feature classifier; use a feature learning method to obtain an image feature vector from the EEG data; and put the image feature vector extracted from the EEG signal into the feature classifier for classification. In the field of image feature classification of EEG data, compared with traditional machine learning and deep learning methods, the feature extraction method of the present invention has a better extraction effect, and the accuracy of image feature classification has been significantly improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to electroencephalogram (EEG) signals in the field of biometrics, and in particular to a method for extracting image features from EEG data and classifying EEG data. Background Art

[0002] Electroencephalogram (EEG) is a graph of data conversion obtained by amplifying and recording the spontaneous biopotential of the brain from the scalp through sophisticated electronic equipment. The EEG contains a large amount of information about the activity of brain neurons, which can be used to obtain a person's mental state. Usually, EEG data is used in various models to classify states, emotions, image stimuli, etc. after feature extraction. However, there is little work on extracting additional component features from EEG data. Convolutional neural network (CNN) is a commonly used feature extraction method that can effectively extract features from data.

[0003] Deep learning is an important branch of machine learning. It learns the inherent laws and representation levels of sample data. Its ultimate goal is to enable machines to surpass human performance after learning a task. It is different from previous machine learning in that it gradually transforms low-level features into high-level features through multi-layer autoencoder neural network processing, making the model have a stronger classification effect.

[0004] Based on the above technical features, the image feature extractor of the present invention is used to extract the visual image features hidden in the EEG data, and this feature is put into the visual image classifier trained by the deep learning network for effective classification. Summary of the invention

[0005] The purpose of the present invention is to provide an EEG data classification method based on deep learning and image features. The technical solution adopted to implement the present invention is:

[0006] A method for classifying EEG data based on deep learning and image features, comprising the following steps:

[0007] Step S1. Raw data collection:

[0008] The data paradigm platform was built using the EEG paradigm design platform Openvibe and the scripting language Python to design a picture stimulation paradigm. At the same time, N healthy young subjects with normal or corrected-to-normal vision were selected to watch different types of visual images, with the visual image screen switched every 500 milliseconds. The EEG data of length T were collected under the visual image stimulation.

[0009] Step S2. Data processing:

[0010] Step S2-1: Data preprocessing: preprocess the collected raw EEG data, use high-pass filtering and low-pass filtering to eliminate noise interference; use independent component analysis to eliminate electrooculographic interference; and then remove interfering pseudo-records to obtain valid EEG data;

[0011] Step S2-2: Time slice division: The EEG data obtained after preprocessing is divided into 500 millisecond time slices to obtain N time slice data of 500 milliseconds each for subsequent model training;

[0012] Step S3. Construction and training of network Image-EEGNet model:

[0013] Step S3-1: Initialization: The network Image-EEGNet model includes: a recurrent feature extractor RF and a deep learning network classifier EEGNet, and the parameters of RF and EEGNet are initialized;

[0014] Step S3-2: training the EEGNet classifier: inputting the image data as a training set, then training the input EEG data with EEGNet, updating the parameters of EEGNet, and obtaining a primary parameter model;

[0015] Step S3-3: training the cyclic feature extractor: inputting the EEG data as a training set, then training the input EEG data with the cyclic feature extractor, updating the cyclic feature extractor parameters, and obtaining a quadratic parameter model;

[0016] Step S3-4: Test and feedback results: Input the EEG data for the test, and obtain the image features through the cyclic feature extractor, and then use the image feature classifier EEGNet to classify it and obtain the classification results;

[0017] Step S4. Evaluation results: Use the image feature classifier obtained by training the visualization image with EEGNet to classify the image features extracted from the EEG data to evaluate the feature extraction effect of the model.

[0018] Preferably, the step S3 is specifically implemented as follows:

[0019] Step S3-1: Initialization:

[0020] The network Image-EEGNet model includes a cyclic feature extractor RF and a deep learning network classifier EEGNet. The parameters μ of RF are initialized respectively. RF and the parameter μ of EEGNet EEGNet Initialize;

[0021] The activation function of each layer of the cyclic feature extractor RF network is the ReLU function, where x represents the sample, and its formula is as follows:

[0022] ReLU(x)=max(0,x)

[0023] When training the cyclic feature extractor RF, the MESloss mean square error function is combined with the CrossEntropyLoss cross entropy function as the loss function; in the mean square error function, x represents the input data sample, y represents the sample generated by the training model, i represents each value in the sample matrix, and N represents the total number of elements in the sample matrix. The formula is as follows:

[0024]

[0025] In the cross entropy function, j represents the classification under the current label state, K represents the total number of classifications, z represents the value of the label, and p represents the classification probability under the current label value state. The formula is as follows:

[0026]

[0027] The activation function used in the last layer of the deep learning network classifier EEGNet network is the Sigmoid function, and its formula is as follows:

[0028]

[0029] The activation function used in other layers is the ELU function, where β is an adjustable parameter that controls when the negative part of ELU is saturated. The formula is as follows:

[0030]

[0031] Step S3-2: Train EEGNet classifier:

[0032] Read the image data in the folder and import it into EEGNet; then start the following work:

[0033] The EEGNet image feature classifier has a total of four layers of networks, which are executed in the following steps:

[0034] A1, data enters the first layer of the network, and one-dimensional linear convolution is used to perform convolution on the data. The convolution uses a kernel sliding window with a length of 32, and sets the stride to 1, the padding value to 0, and the output channel

[0035] The number is 16. After processing, the data is activated using the activation function ELU;

[0036] A2, the data enters the second network layer, and is convolved with a two-dimensional linear convolution. The convolution uses a kernel sliding window with a length of 2*32, and sets the stride to 1, the padding value to 0, and the number of output channels to 8. After the convolution, the data is pooled with a convolution kernel of 2*4. After the pooling operation, the activation function is used.

[0037] Count ELU activation data;

[0038] A3, the data enters the third network layer, and is convolved with a two-dimensional linear convolution. The convolution uses a convolution kernel sliding window with a length of 8*4, and sets the stride to 1, the padding value to 0, and the number of output channels to 4. After the convolution, the data is pooled with a convolution kernel of 2*4. After the pooling operation, the activation function is used.

[0039] Count ELU activation data;

[0040] A4, the data enters the fourth layer of the network, uses a fully connected convolution to convert the data into a tensor matrix of size 4, and uses the Sigmoid function to activate the data;

[0041] A5, the loss function used in the training process is the CrossEntropy function, and the parameters of the image feature classifier EEGNet are saved after training;

[0042] Step S3-3: Training the loop feature extractor

[0043] The constructed cyclic feature extractor is mainly composed of convolutional network and recurrent neural network RNN; the specific execution structure is as follows:

[0044] B1, data enters the first layer of the network and is convolved using Conv. The convolution parameters are input 1, output 4, and convolution kernel size 7. After the convolution operation, the data is activated using the ReLU activation function.

[0045] B2, the data enters the second layer of the network and is convolved using Conv. The convolution parameters are input 4, output 8, and the convolution kernel size is 5. After the convolution operation, the data is activated using the ReLU activation function.

[0046] B3, the data enters the third layer of the network and uses the maximum pooling MaxPool with a pooling kernel of 3; after the pooling operation, the data is activated using the ReLU activation function;

[0047] B4, the data enters the fourth layer network. The fourth layer network has three branch networks, all of which are Conv convolutional networks: the first branch network of the fourth layer network has a parameter input of 8, an output of 16, a kernel size of 3, and a ReLU activation function is used to activate the data after the convolution operation; the second branch network of the fourth layer network has a parameter input of 8, an output of 16, a kernel size of 5, and a ReLU activation function is used to activate the data after the convolution operation; the third branch network of the fourth layer network has a parameter input of 8,

[0048] The output is 16, the kernel size is 7; after the convolution operation, the ReLU activation function is used to activate the data;

[0049] B5, the data enters the fifth layer network, and the concat function is used to merge the data of the three branch networks of the fourth layer network.

[0050] The data with 48 channels is obtained; then it enters the Conv convolution layer with parameter input of 48, output of 1, and kernel size of 1;

[0051] B6, data enters the sixth layer network. The sixth layer network has two branch networks, both of which are fully connected networks: the first branch network of the sixth layer has an input parameter of 32 and an output of 32. After the convolution operation, the ReLU activation function is used to activate the data; the second branch network of the sixth layer has an input parameter of 64 and an output of 64. After the convolution operation, the ReLU activation function is used to activate the data.

[0052] The activation function activates the data; then the vectors obtained from the two branch networks are accumulated;

[0053] B7, the data enters the seventh layer of the network and is processed by the recurrent neural network LSTM. The processing layer parameter of LSTM is set to 1 layer. The activation functions Sigmoid and Tanh are used inside LSTM. The formula of Tanh is as follows:

[0054]

[0055] Then, the result vector obtained by the activation function is added to the result vector obtained by the sixth layer network to obtain the final image feature data.

[0056] B8, read the EEG data corresponding to the image viewed by the subject in the folder and import it into the cyclic feature extractor;

[0057] Enter the feature extractor and train the EEG data. The loss function used in training is the loss function of the cyclic feature extractor, where a is the influence factor, and its value is between 0 and 1. The formula is as follows:

[0058]

[0059] After training, save the parameters of the cyclic feature extractor RF;

[0060] Step S3-4: Feedback result: Get the trained model; input the EEG test data set into the trained cyclic feature extraction model to get the image feature vector in the computer data, and then put the image feature vector into the image feature vector classifier EEGNet to get the final classification result.

[0061] The beneficial effects of the present invention are as follows: the Image-EEGNet of the present invention extracts image features from EEG data for classification. The single-task problem that conventional machine learning and deep learning methods are only used for EEG feature classification is solved, and EEG data is associated with image data. In addition, the cyclic feature extractor of the present invention can be used on most image feature classification models and has good generalization performance.

[0062] The present invention has achieved good results in EEG image feature extraction and classification. It is expected that the model of the present invention can be used to restore EEG data into visual graphics seen by subjects in the future. It is also expected to have broader application prospects in actual brain-computer interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a flow chart of the present invention;

[0064] Figure 2 It is a network framework diagram adopted by the present invention;

[0065] Figure 3 It is an algorithm structure diagram of the cyclic feature extractor adopted by the present invention;

[0066] Figure 4 It is a comparison diagram of the inventive effects of the method of the present invention. DETAILED DESCRIPTION

[0067] The present invention is further described below in conjunction with the accompanying drawings and embodiments, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more explicit definition of the protection scope of the present invention.

[0068] like Figure 1 As shown, the present invention uses the deep learning model EEGNet as the image feature classification method, and obtains the image feature classifier by training the image data; then, the image features of the EEG data are extracted using the cyclic feature extractor designed and invented by us; finally, the image features extracted from the EEG are classified by the image feature classifier.

[0069] Step S1. Raw data collection:

[0070] The simulation platform was built using the EEG paradigm design platform Openvibe and the scripting language python to design the simulated image stimulation paradigm. The data of this simulation system was recorded by the data receiving software EASYCAP corresponding to the acquisition device and forwarded to Openvibe, which then saved the data in a CSV format file with a sampling frequency of 128Hz. All 32 EEG channels were referenced to the two earlobes of the FCz channel and kept their impedance below 10KQ.

[0071] Based on this system, N healthy young people with normal or corrected normal vision were selected as subjects to collect EEG signals. The task is to sit on a chair in an absolutely quiet environment without external interference, then wear the EEG data acquisition device and watch the paradigm image on the computer screen. The paradigm image is switched every 500 milliseconds, so as to achieve the effect of event-related potential stimulation to the subjects during the image switching process. When the image switching interface disappears, the task ends. The experiment includes multiple sets of experimental collection (no less than 15 times).

[0072] Step S2. Data processing:

[0073] The acquired raw data were processed using the EEGlab script.

[0074] S2-1: Data preprocessing

[0075] The data was imported into the EEGlab script, and the 50 Hz mains power and high-frequency noise were filtered using high-pass and low-pass filtering algorithms. Then, ICA was used to remove electrooculographic interference, and finally, artifacts were removed.

[0076] S2-2: Time Slice Division

[0077] The data is divided into several sequences of equal length T and stored in a file in the MAT format for subsequent model training.

[0078] like Figure 2 and Figure 3 As shown, step S3. Based on the EEGNet network and the idea of ​​extracting image features from computer data, a corresponding network model named Image-EEGNet is designed and trained, such as Figure 2 The specific implementation is as follows:

[0079] S3-1: Initialization

[0080] The network Image-EEGNet model includes a recurrent feature extractor (RF) and a deep learning network classifier (EEGNet). The parameters μ of the RF are initialized respectively. RF and the parameter μ of EEGNet EEGNet Initialize;

[0081] The activation function of each layer of the cyclic feature extractor RF network is the ReLU function, where x represents the sample, and its formula is as follows:

[0082] ReLU(x)=max(0,x)

[0083] When training the cyclic feature extractor (RF), the MESloss mean square error function is combined with the CrossEntropyLoss cross entropy function as the loss function. In the mean square error function, x represents the input data sample, y represents the sample generated by the training model, and i represents each value in the sample matrix. The formula is as follows:

[0084] MSE(x i ,y i )=(x i -y) 2 #(2)

[0085] In the cross entropy function, i represents the classification under the current label state, K represents the total number of classifications, y represents the value of the label, and p represents the classification probability under the current label value state. The formula is as follows:

[0086]

[0087] The activation function used in the last layer of the deep learning network classifier EEGNet network is the Sigmoid function, and its formula is as follows:

[0088]

[0089] The activation function used in other layers is the ELU function, where β is an adjustable parameter that controls when the negative part of ELU is saturated. The formula is as follows:

[0090]

[0091] S3-2: Training the EEGNet classifier

[0092] Read the image data in the folder and import it into EEGNet. Then start the following work:

[0093] The EEGNet image feature classifier has a total of four layers of networks:

[0094] 1. The data first enters the first layer of the network, and is first convolved with a one-dimensional linear convolution. The convolution uses a convolution kernel sliding window with a length of 32, and sets the stride to 1, the padding value to 0, and the number of output channels to 16. After processing, the data is activated using the activation function ELU;

[0095] 2. The data enters the second network layer and is convolved with a two-dimensional linear convolution. The convolution uses a convolution kernel sliding window with a length of 2*32, and sets the stride to 1, the padding value to 0, and the number of output channels to 8. After the convolution, the data is pooled with a convolution kernel of 2*4. After the pooling operation, the activation function ELU is used to activate the data.

[0096] 3. The data enters the third network layer and is convolved with a two-dimensional linear convolution. The convolution uses a convolution kernel sliding window with a length of 8*4, and sets the stride to 1, the padding value to 0, and the number of output channels to 4. After the convolution, the data is pooled with a convolution kernel of 2*4. After the pooling operation, the activation function ELU is used to activate the data.

[0097] 4. The data enters the fourth layer of the network, and the data is converted into a tensor matrix of size 4 using a fully connected convolution, and the data is activated using the Sigmoid function.

[0098] The loss function used in the training process is the CrossEntropy function, and the parameters of the image feature classifier EEGNet are saved after training.

[0099] S3-3: Training Cyclic Feature Extractor

[0100] The cyclic feature extractor constructed by the present invention is mainly composed of a convolutional network and a recurrent neural network RNN. The specific structure is as follows:

[0101] 1. The data enters the first layer of the network and is convolved using Conv. The convolution parameters are input 1, output 4, and the convolution kernel size is 7. After the convolution operation, the data is activated using the ReLU activation function;

[0102] 2. The data enters the second layer of the network and is convolved using Conv. The convolution parameters are input 4, output 8, and the convolution kernel size is 5. After the convolution operation, the data is activated using the ReLU activation function;

[0103] 3. The data enters the third layer of the network and uses the maximum pooling MaxPool with a pooling kernel of 3. After the pooling operation, the data is activated using the ReLU activation function;

[0104] 4. The data enters the fourth layer network. The fourth layer network has three branch networks, all of which are Conv convolutional networks: the first branch network has a parameter input of 8, an output of 16, a kernel size of 3, and a ReLU activation function to activate the data after the convolution operation; the second branch network has a parameter input of 8, an output of 16, a kernel size of 5, and a ReLU activation function to activate the data after the convolution operation; the third branch network has a parameter input of 8, an output of 16, and a kernel size of 7; the ReLU activation function is used to activate the data after the convolution operation;

[0105] 5. The data enters the fifth layer of the network, and the concat function is used to merge the data of the three branch networks to obtain data with 48 channels; then it enters the Conv convolution layer, with parameter input of 48, output of 1, and kernel size of 1;

[0106] 6. The data enters the sixth layer of the network. The sixth layer has two branch networks, both of which are fully connected networks: the first branch network has an input parameter of 32 and an output of 32. After the convolution operation, the ReLU activation function is used to activate the data; the first branch network has an input parameter of 64 and an output of 64. After the convolution operation, the ReLU activation function is used to activate the data; then the vectors obtained from the two branch networks are accumulated.

[0107] 7. The data enters the seventh layer of the network and is processed by the recurrent neural network LSTM. The processing layer parameter of LSTM is set to 1 layer. The activation functions Sigmoid and Tanh are used inside LSTM. The formula of Tanh is as follows:

[0108]

[0109] Then, the result vector obtained by the activation function is added to the result vector obtained by the sixth layer network to obtain the final image feature data.

[0110] Read the EEG data corresponding to the image viewed by the subject in the folder and import it into the cyclic feature extractor. Then start the following work: first enter the feature extractor, then train the EEG data. The loss function loss used in training is the loss function established for the cyclic feature extractor in the present invention, where a is the influence factor, and its value is between 0 and 1. The formula is as follows:

[0111]

[0112] After training, save the parameters of the cyclic feature extractor RF.

[0113] S3-4: Testing and Feedback Results

[0114] After the above steps, the trained model is obtained; the EEG test data set is input into the trained cyclic feature extraction model to obtain the image feature vector in the computer data, and then the image feature vector is put into the image feature vector classifier EEGNet to obtain the final classification result.

[0115] Step S4. Evaluation results:

[0116] In order to verify the effectiveness of the present invention, 15 subjects were selected and the image feature classifier obtained by training the visualized image with EEGNet was used to classify the image features extracted from the EEG data to evaluate the feature extraction effect of the model. Figure 4 As shown, compared with other methods, the method of the present invention has a better and more generalized effect on image feature extraction in EEG data, and the accuracy of the image feature classifier is also significantly higher than other methods.

Claims

1. A method for classifying EEG data based on deep learning and image features, characterized in that: The steps include: Step S1. Raw data collection: The data paradigm platform was built using the EEG paradigm design platform Openvibe and the scripting language Python to design a picture stimulation paradigm. At the same time, N healthy young subjects with normal or corrected-to-normal vision were selected to watch different types of visual images, with the visual image screen switched every 500 milliseconds. The EEG data of length T were collected under the visual image stimulation. Step S2. Data processing: Step S2-1: Data preprocessing: preprocess the collected raw EEG data, use high-pass filtering and low-pass filtering to eliminate noise interference; use independent component analysis to eliminate electrooculographic interference; Then remove the interfering pseudo-records to obtain valid EEG data; Step S2-2: Time slice division: The EEG data obtained after preprocessing is divided into 500 millisecond time slices to obtain N time slice data of 500 milliseconds each for subsequent model training; Step S3. Construction and training of network Image-EEGNet model: Step S3-1: Initialization: The network Image-EEGNet model includes: a recurrent feature extractor RF and a deep learning network classifier EEGNet, and the parameters of RF and EEGNet are initialized; Step S3-2: training the EEGNet classifier: inputting the image data as a training set, then training the input EEG data with EEGNet, updating the parameters of EEGNet, and obtaining a primary parameter model; Step S3-3: training the cyclic feature extractor: inputting the EEG data as a training set, then training the input EEG data with the cyclic feature extractor, updating the cyclic feature extractor parameters, and obtaining a quadratic parameter model; Step S3-4: Test and feedback results: Input the EEG data for the test, and obtain the image features through the cyclic feature extractor, and then use the image feature classifier EEGNet to classify it and obtain the classification results; Step S4. Evaluation results: Use the image feature classifier obtained by training the visualization image with EEGNet to classify the image features extracted from the EEG data to evaluate the feature extraction effect of the model; The step S3 is specifically implemented as follows: Step S3-1: Initialization: The network Image-EEGNet model includes a cyclic feature extractor RF and a deep learning network classifier EEGNet. The parameters μ of RF are initialized respectively. RF and the parameter μ of EEGNet EEGNet Initialize; The activation function of each layer of the cyclic feature extractor RF network is the ReLU function, where x represents the data sample, and its formula is as follows: ReLU(x)=max(0,x) When training the cyclic feature extractor RF, the MESloss mean square error function is combined with the CrossEntropyLoss cross entropy function as the loss function; in the mean square error function, x represents the input data sample, y represents the sample generated by the training model, i represents each value in the sample matrix, and N represents the total number of elements in the sample matrix. The formula is as follows: In the cross entropy function, j represents the classification under the current label state, K represents the total number of classifications, z represents the value of the label, and p represents the classification probability under the current label value state. The formula is as follows: The activation function used in the last layer of the deep learning network classifier EEGNet network is the Sigmoid function, and its formula is as follows: The activation function used in other layers is the ELU function, where β is an adjustable parameter that controls when the negative part of ELU is saturated. The formula is as follows: Step S3-2: Train EEGNet classifier: Read the image data in the folder and import it into EEGNet; then start the following work: The EEGNet image feature classifier has a total of four layers of networks, which are executed in the following steps: A1, data enters the first layer of the network, and one-dimensional linear convolution is used to perform convolution on the data. The convolution uses a kernel sliding window with a length of 32, and sets the stride to 1, the padding value to 0, the number of output channels to 16, and the activation function ELU is used to activate the data after processing; A2, the data enters the second network layer, and is convolved with a two-dimensional linear convolution. The convolution uses a kernel sliding window with a length of 2*32, and sets the stride to 1, the padding value to 0, and the number of output channels to 8. After the convolution, the data is pooled with a convolution kernel of 2*4. After the pooling operation, the activation function ELU is used to activate the data. A3, the data enters the third network layer, and is convolved with a two-dimensional linear convolution. The convolution uses a kernel sliding window with a length of 8*4, and sets the stride to 1, the padding value to 0, and the number of output channels to 4. After the convolution, the data is pooled with a 2*4 convolution kernel. After the pooling operation, the data is activated with the activation function ELU. A4, the data enters the fourth layer of the network, uses a fully connected convolution to convert the data into a tensor matrix of size 4, and uses the Sigmoid function to activate the data; A5, the loss function used in the training process is the CrossEntropy function, and the parameters of the image feature classifier EEGNet are saved after training; Step S3-3: Training the loop feature extractor The constructed cyclic feature extractor is mainly composed of convolutional network and recurrent neural network RNN; the specific execution structure is as follows: B1, data enters the first layer of the network and is convolved using Conv. The convolution parameters are input 1, output 4, and convolution kernel size 7. After the convolution operation, the data is activated using the ReLU activation function. B2, the data enters the second layer of the network and is convolved using Conv. The convolution parameters are input 4, output 8, and the convolution kernel size is 5. After the convolution operation, the data is activated using the ReLU activation function. B3, the data enters the third layer of the network and uses the maximum pooling MaxPool with a pooling kernel of 3; after the pooling operation, the data is activated using the ReLU activation function; B4, data enters the fourth layer network. The fourth layer network has three branch networks, all of which are Conv convolutional networks: the first branch network of the fourth layer network has a parameter input of 8, an output of 16, a kernel size of 3, and a ReLU activation function is used to activate the data after the convolution operation is completed; the second branch network of the fourth layer network has a parameter input of 8, an output of 16, a kernel size of 5, and a ReLU activation function is used to activate the data after the convolution operation is completed; the third branch network of the fourth layer network has a parameter input of 8, an output of 16, and a kernel size of 7; after the convolution operation is completed, the ReLU activation function is used to activate the data; B5, the data enters the fifth layer of the network, and the concat function is used to merge the data of the three branch networks of the fourth layer to obtain data with 48 channels; then it enters the Conv convolution layer, with parameter input of 48, output of 1, and kernel size of 1; B6, data enters the sixth layer network. The sixth layer network has two branch networks, both of which are fully connected networks: the first branch network of the sixth layer network has an input parameter of 32 and an output of 32. After the convolution operation, the ReLU activation function is used to activate the data; the second branch network of the sixth layer network has an input parameter of 64 and an output of 64. After the convolution operation, the ReLU activation function is used to activate the data; then the vectors obtained by the two branch networks are accumulated; B7, the data enters the seventh layer of the network and is processed by the recurrent neural network LSTM. The processing layer parameter of LSTM is set to 1 layer. The activation functions Sigmoid and Tanh are used inside LSTM. The formula of Tanh is as follows: Then, the result vector obtained by the activation function is added to the result vector obtained by the sixth layer network to obtain the final image feature data. B8, read the EEG data corresponding to the image viewed by the subject in the folder and import it into the cyclic feature extractor; Enter the feature extractor and train the EEG data. The loss function used in training is the loss function of the cyclic feature extractor, where a is the influence factor, and its value is between 0 and 1. The formula is as follows: After training, save the parameters of the cyclic feature extractor RF; Step S3-4: Feedback result: Get the trained model; input the EEG test data set into the trained cyclic feature extraction model to get the image feature vector in the computer data, and then put the image feature vector into the image feature vector classifier EEGNet to get the final classification result.

Citation Information

Patent Citations

  • Electroencephalogram signal detection method based on convolutional neural network

    CN112022153A

  • Method, device and program for determining at least one cerebral network involved in carrying out a given process

    WO2018153762A1