EEG signal classification model, method and computer system

Abstract

The present invention discloses an EEG signal classification model, method and computer system, including: extracting power spectrum features of original EEG signals; extracting differential entropy features of original EEG signals; fusing power spectrum features with original EEG signals to obtain fused EEG signals; performing multiple convolution pooling operations on the fused EEG signals, extracting features of the fused EEG signals, obtaining high-dimensional feature vectors, and then mapping the high-dimensional feature vectors to low-dimensional space through linear transformation to obtain low-dimensional feature vectors; performing dimensionality reduction and splicing of differential entropy features of original EEG signals of different frequency bands, and splicing them with the EEG signals after dimensionality reduction to obtain EEG data fused with multiple features; using a self-attention mechanism to capture key features from the EEG data fused with multiple features to obtain a final output vector; using a classifier to classify the final output vector to obtain a classified EEG signal. The present invention can capture key information in EEG signals more comprehensively.

Description

Technical Field

[0001] The present invention relates to the technical field of electroencephalogram (EEG) signal processing, and in particular to an EEG signal classification model, method and computer system based on a multi-feature fusion convolutional network with an attention mechanism. Background Art

[0002] Brain-computer interface technology (BCI) is an emerging technology field. It analyzes an individual's thoughts, emotions, and intentions by directly reading the brain's neural signals, thereby achieving control of external devices or human-computer interaction. In terms of emotion analysis, brain-computer interface technology provides a new means to study the generation, expression, and recognition of emotions and feelings. Through the feature extraction and classification of EEG signals, the recognition and classification of different emotional states can be achieved; by analyzing the frequency and amplitude of EEG signals, the emotional state and emotional changes of individuals can be inferred; by analyzing the time-frequency characteristics of EEG signals, the detection and classification of emotion-induced stimuli can be achieved. In terms of motor imagery classification, brain-computer interface technology has opened up an innovative path for exploring the generation, realization, and recognition of motor intentions. By analyzing the various characteristic parameters of EEG signals, the content and changes of individual motor imagery can be inferred; by extracting and classifying the features of EEG signals, the accurate recognition and classification of different motor imagery states can be achieved; by analyzing the time-frequency characteristics of EEG signals, the effective detection and classification of motor imagery-induced stimuli can be achieved.

[0003] Nowadays, more and more attention is paid to the emotion recognition and motor imagery recognition tasks of electroencephalogram (EEG) analysis, because EEG signals can subjectively reflect real emotions and motor imagery, and are a reliable way to recognize emotions and motor imagery. In the early days, emotion and motor imagery recognition mainly used traditional machine learning methods (such as KNN, SVM) for feature extraction and classification learning. The limitation of traditional machine learning is that feature selection is very difficult, which requires expert knowledge. The emergence of deep learning has overcome this difficulty, and deep learning methods have begun to be applied to EEG emotion and motor imagery recognition. Since the generation and expression of EEG signals involve complex neural mechanisms, it is key to pursue higher accuracy and recognition precision when analyzing individual EEG states. The optimization of feature extraction and feature fusion is particularly important, and there is an urgent need for faster and more accurate methods to analyze EEG data. Summary of the invention

[0004] The purpose of the present invention is to provide an EEG signal classification model, method and computer system based on a multi-feature fusion convolutional network with an attention mechanism, which effectively improves the recognition ability of the model and the classification accuracy of EEG signals by extracting power spectrum features and differential entropy features and using convolutional neural networks and attention mechanisms for feature fusion and classification.

[0005] In the first aspect, a brain wave signal classification model is proposed, comprising: a data acquisition module, which is configured to obtain the original brain wave signal; a power spectrum feature extraction module, which is configured to extract the power spectrum feature of the original brain wave signal; a differential entropy feature extraction module, which is configured to extract the differential entropy feature of the original brain wave signal in different frequency bands; a first feature fusion module, which is configured to fuse the extracted power spectrum feature with the original brain wave signal to obtain a fused brain wave signal; a multi-feature fusion convolutional network, which is configured to perform multiple convolution pooling operations on the fused brain wave signal to extract the fused brain wave signal features to obtain a high-dimensional feature vector, and then map the high-dimensional feature vector to a low-dimensional space through a linear transformation to obtain a low-dimensional feature vector; a second feature fusion module, which is configured to reduce the dimension of the differential entropy features of the original EEG signals of different frequency bands and splice them with the reduced-dimensional EEG signals output by the multi-feature fusion convolutional network to obtain EEG data fused with multiple features; a self-attention module, which captures key features from the EEG data fused with multiple features to obtain a final output vector; and a classifier, which is configured to classify the final output vector of the self-attention module to obtain a classified EEG signal.

[0006] In some examples, the power spectrum feature extraction module obtains the power spectrum feature by adding a Hamming window to each original EEG signal, calculating a fast Fourier transform on each original EEG signal processed by the Hamming window, squaring the result of the fast Fourier transform, and then dividing it by half the number of sampling points.

[0007] In some examples, a multi-feature fusion convolutional network includes: a two-dimensional convolutional layer, which applies multiple convolution kernels to the EEG signal after the power spectrum feature and the original EEG signal are fused to generate a feature map; a batch normalization layer, which performs batch normalization on the feature map generated by the two-dimensional convolutional layer; an activation function, which provides a non-zero gradient for negative values ​​in the feature map after processing by the batch normalization layer; an average pooling layer, which performs an average pooling operation on the feature map after processing by the activation function to obtain a high-dimensional feature vector; a fully connected layer, which maps the high-dimensional feature vector of the feature map after the average pooling operation to a low-dimensional space through a linear transformation to obtain a reduced-dimensional EEG signal.

[0008] In some examples, the differential entropy feature extraction module extracts Differential entropy features of original EEG signals in 5 frequency bands.

[0009] In some examples, the second feature fusion module The differential entropy features of the five frequency bands are concatenated by dimensionality reduction and then concatenated with the low-dimensional feature vector output by the fully connected layer to obtain EEG data that integrates multiple features.

[0010] In some examples, the self-attention module is configured to: use linear transformation to respectively calculate the query, key, and value matrices of the EEG data fused with multiple features; obtain the attention weight by calculating the dot product similarity between the query and the key; normalize the weight to obtain the attention weight matrix; and use the attention weight matrix to perform weighted summation on the value matrix to obtain the final output vector.

[0011] In the second aspect, a method for classifying EEG signals is proposed, comprising: collecting EEG signals of a test subject, inputting the EEG signals into the EEG signal classification model, and the EEG signal classification model outputting classified EEG signals. The EEG state reflecting the emotional / motor imagery content of the test subject is identified based on the classified EEG signals.

[0012] In a third aspect, a computer system is proposed, comprising: a processor; a memory, comprising one or more computer program modules; wherein the one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules include instructions for implementing the EEG signal classification method.

[0013] In a fourth aspect, a computer-readable storage medium is proposed for storing non-temporary computer-readable instructions, which can implement the EEG signal classification method when executed by a computer.

[0014] The present invention introduces the attention mechanism and combines it with a multi-feature fusion strategy to enable the model to capture the key information in the EEG signal more comprehensively. Compared with the traditional single-feature classification method, the present invention has significantly improved the feature extraction and classification accuracy, especially when processing complex EEG signal data, showing higher accuracy and robustness. It can be used in fields such as medical diagnosis assistance, such as assisting the diagnosis of psychological diseases such as depression. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flowchart of an EEG signal classification method based on a multi-feature fusion convolutional network with an attention mechanism according to an embodiment of the present invention.

[0016] Figure 2 4 is a structural block diagram of an EEG attention classification model according to an embodiment of the present invention.

[0017] Figure 3 4 is an electrode distribution diagram for collecting EEG signals according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] Figure 1 This is an overall flow chart of the EEG signal classification method based on the multi-feature fusion convolutional network of the attention mechanism. Figure 1The method shown is described in detail.

[0019] S1: Obtain the EEG signals (EEG signals) of the subjects in different emotional states for subsequent classification of EEG signals.

[0020] S11. Use an EEG acquisition device to collect EEG signals from the subject. For example, the EEG signals of the subject in different emotional states are obtained by letting the subject watch an emotion-stimulating video, and the obtained EEG signals are emotionally classified by watching the content of the video at different time points to obtain the classified EEG signals. In one embodiment, multiple subjects are asked to watch a video that has been classified to stimulate 5 emotional states. The video is about 10 minutes long, and the EEG signals of each participant are repeatedly collected. In order to obtain the best signal quality, the following parameters are set: 32 channels (electrodes) are used for data acquisition ( Figure 3 shows the electrode distribution), one electrode was set as the reference electrode, and the sampling rate was set to 1000 Hz.

[0021] S12. Use a 0-75 Hz bandpass filter to remove artifacts from the EEG signal, and the processed EEG signal is used as the original EEG signal.

[0022] S2: Use the power spectrum feature extraction module and the differential entropy feature extraction module to extract the power spectrum feature (also known as: power spectrum density feature) and differential entropy feature of the original EEG signal respectively.

[0023] S21. Extract power spectrum features based on the original EEG signal.

[0024] First, the present invention needs to define a length of T The Hamming window is applied to each raw EEG signal (i.e., the data of each electrode / channel) to reduce spectrum leakage. In one embodiment, T is 1000.

[0025] Then, the fast Fourier transform is calculated for each raw EEG signal processed by the Hamming window. The Fourier transform is used to convert the time domain signal into the frequency domain signal. The calculation formula is as follows:

[0026]

[0027] in It is the original EEG signal DFT (Fourier transform) is the representation of the original EEG signal in the frequency domain. is the original EEG signal sequence, t is the index of the time domain, ranging from 0 to . T It is the original EEG signal Length. It is the index of the frequency domain, and the range is also from 0 to . is an imaginary unit, satisfying . It is the core of Fourier transform, which is a complex exponential function that represents the rotation factor. This formula decomposes the time signal into frequencies to obtain the frequency components of the original EEG signal, which helps to extract the power spectrum features.

[0028] The result of the fast Fourier transform is squared and then divided by half the number of sampling points (due to the symmetry of the fast Fourier transform, only half the frequency needs to be calculated) to obtain the power spectrum characteristics.

[0029] S22. Extract differential entropy features based on the original EEG signal.

[0030] The differential entropy features of the original EEG signals in five frequency bands (δ: 0.5-3.5Hz, θ: 4-7Hz, α: 8-13Hz, β: 14-30Hz and γ: 31-50Hz) of each EEG channel can be calculated. The differential entropy feature calculation formula of the original EEG signals in these five frequency bands is as follows:

[0031]

[0032] in is the energy value of different frequency bands, is the desired differential entropy characteristic.

[0033] S3: The extracted power spectrum features are combined with the original EEG signal to obtain the fused EEG signal, and the fused EEG signal is input into the convolutional neural network (CNN) for feature extraction and dimensionality reduction. The dimensionality-reduced EEG signal is then combined with the differential entropy features to obtain EEG data with multiple features. Convolutional neural network is a deep learning model that can effectively extract features from complex data. Figure 2 Describe this process in detail.

[0034] S31, fusing the extracted power spectrum features and the original EEG signal to obtain a fused EEG signal.

[0035] S32, the fused EEG signal is first passed through a two-dimensional convolution layer to extract the features of the fused EEG signal. This layer generates a feature map by applying multiple convolution kernels to the input data (fused EEG signal). In one embodiment, the size of the convolution kernel is 5x5, the step size is 1, the padding is 2, and 2 pixels of zero padding are added around the border of the input data to maintain the size of the feature map.

[0036] S33. The feature map generated by the two-dimensional convolutional layer is then passed through a batch normalization layer to accelerate the training process of the neural convolutional network and improve the training stability by reducing the internal covariate shift.

[0037] S34. Then, the feature map processed by the normalization layer is activated by the LeakyReLU function to provide a non-zero gradient for the negative input value, thereby avoiding the appearance of "dead neurons" during the training process. The calculation formula is as follows:

[0038]

[0039] Where x is the input to the activation function. is a small positive number, called the leaky coefficient, which determines the slope of the function when the input is less than 0. In the standard LeakyReLU implementation, A common value for is 0.01, but this value can be adjusted depending on the specific application.

[0040] S35, the feature map processed by the activation function is then passed through an average pooling layer to obtain a high-dimensional feature vector. In one embodiment, the size of the pooling window is 2x2, and the step length of the window moving on the input feature map is 2. By applying a fixed-size window to the input feature map, the average value of all values ​​in the window is calculated, thereby reducing the size of the feature map while retaining the most important information.

[0041] S36. Perform two more convolution and pooling operations (S32-S35). Then, the high-dimensional feature vector finally output after three convolution and pooling passes through a fully connected layer, and the high-dimensional feature vector is mapped to a low-dimensional space through a linear transformation to obtain a low-dimensional feature vector.

[0042] S37, performing dimension reduction and splicing on the differential entropy features of the five different frequency bands obtained in step S22, and splicing them with the low-dimensional feature vector output by the fully connected layer to obtain EEG data integrating multiple features.

[0043] S4: The EEG data with multiple features is passed through the self-attention module to effectively capture features. The attention mechanism is a technology that improves feature selectivity in neural networks. It captures key features by assigning different weights, thereby improving the classification accuracy of the model. The self-attention module can automatically focus on the important parts of the EEG data with multiple features and reduce the interference of redundant information. This helps to improve the efficiency and accuracy of feature extraction and provide more valuable information for subsequent analysis and processing. The following is a detailed description of this process.

[0044] S41. Pass the EEG data fused with multiple features through the self-attention module, and use linear transformation to calculate the query, key and value vectors respectively. The calculation formula is as follows:

[0045]

[0046]

[0047]

[0048] in It is an embedded representation of EEG data that integrates multiple features. It contains various features extracted from the original EEG signal (such as time domain features, frequency domain features, spatial features, etc.). are query, key and value vector matrices respectively, WQ , WK , WV are the corresponding weight matrices respectively.

[0049] S42, the attention mechanism obtains the attention weight by calculating the dot product similarity between the query and the key. In the EEG signal classification task, the present invention uses the dot product to calculate the similarity weight between the query and the key. The calculation formula is as follows:

[0050]

[0051] in Indicates query vector and Key vector The attention weights between is the feature dimension of the key and query.

[0052] S43. After calculating the attention weight, use the softmax function to normalize the weight to obtain the attention weight matrix , the calculation formula is as follows:

[0053]

[0054] The attention weight matrix Each row of represents the importance of the key vector for a given query vector. Indicates query vector and Key vector In the EEG signal classification task, these weights will be used for weighted summation to affect the final classification result.

[0055] S44. Using the attention weight matrix Pair-valued vector matrix The weighted sum is performed to obtain the final output vector. In the EEG signal classification task, this final output vector will be used as the input of the classifier to predict the category of the EEG signal. Through the self-attention mechanism, the model can automatically focus on the important parts of the input features, thereby improving the accuracy and robustness of the classification.

[0056] S5: The final output vector obtained after the self-attention module is classified by the classifier. The classifier contains a fully connected layer and Function. Classifiers are conventional techniques for those skilled in the art, and will not be described in detail in the present invention. The classification result is that EEG signals with the same emotion / motor imagery content are in the same category.

[0057] The present invention also provides an embodiment of an EEG signal classification model, which includes a data acquisition module, a power spectrum feature extraction module, a differential entropy feature extraction module, a first feature fusion module, a multi-feature fusion convolutional network, a self-attention module and a classifier.

[0058] The data acquisition module is configured to obtain raw EEG signals. The specific method of obtaining raw EEG signals is shown in S11-S13.

[0059] The power spectrum feature extraction module is configured to extract the power spectrum feature of the original EEG signal. The power spectrum feature extraction module performs Hamming window processing on each original EEG signal, calculates the fast Fourier transform on each original EEG signal processed by the Hamming window, squares the result of the fast Fourier transform, and then divides it by half of the number of sampling points to obtain the power spectrum feature. For a detailed method of extracting the power spectrum feature, refer to S21.

[0060] The differential entropy feature extraction module is configured to extract differential entropy features of the original EEG signal, specifically to extract differential entropy features of the original EEG signal in different frequency bands, which has been described in detail in S22.

[0061] The first feature fusion module is configured to fuse the extracted power spectrum feature and the original EEG signal to obtain a fused EEG signal, which is also explained in S31.

[0062] The multi-feature fusion convolutional network is configured to perform multiple convolution and pooling operations on the fused EEG signal, extract the features of the fused EEG signal, obtain a high-dimensional feature vector, and then map the high-dimensional feature vector to a low-dimensional space through a linear transformation to obtain a low-dimensional feature vector. The multi-feature fusion convolutional network includes a two-dimensional convolutional layer, a batch normalization layer, an activation function, an average pooling layer, and a fully connected layer. S32-S36 provide detailed descriptions of the various parts of the multi-feature fusion convolutional network.

[0063] The second feature fusion module is configured to reduce the dimension of the differential entropy features of the original EEG signals of different frequency bands and splice them with the EEG signals after dimension reduction output by the fully connected layer to obtain EEG data fused with multiple features, which is also explained in S37.

[0064] The self-attention module is configured to capture features from EEG data fused with multiple features, see S41-S45 for details.

[0065] The classifier is configured to classify the final output vector of the self-attention module to obtain a classified EEG signal.

[0066] Finally, the model can be evaluated by calculating the loss value and accuracy of the binary cross loss entropy function. The smaller the loss value of the binary cross loss entropy function, the better the model. The calculation formula is as follows:

[0067]

[0068] Where n is the number of classifications, if the classification result is correct, y is 0, otherwise it is 1. is the probability of being classified into the i-th category, and Loss is the binary cross entropy loss value;

[0069]

[0070] Where TP is the number of samples correctly classified as positive samples, TN is the number of samples correctly classified as negative samples, FP is the number of samples misclassified as positive samples, FN is the number of samples misclassified as negative samples, and acc is the accuracy rate.

[0071] The EEG signal classification model trained by the present invention can be used to detect the EEG signal of the tester and identify the current EEG state of the tester, that is, to identify the emotion / motor imagery content through the EEG signal. The present invention can be applied in the field of brain-computer interface, and the identified EEG state can be used to control external devices.

[0072] The present invention also provides an embodiment of a computer system. The computer system is a broad concept, including but not limited to computers and external devices of brain-computer interfaces. The computer system includes a processor and a memory. The memory is used to store non-temporary computer-readable instructions (such as one or more computer program modules). The processor is used to run non-temporary computer-readable instructions, and when the non-temporary computer-readable instructions are run by the processor, one or more steps in the EEG signal classification method described above can be executed. The memory and the processor can be interconnected through a bus system and / or other forms of connection mechanisms.

[0073] For example, the processor may be a central processing unit (CPU), a graphics processing unit (GPU), or other forms of processing units with data processing capabilities and / or program execution capabilities. For example, the central processing unit (CPU) may be an X86 or ARM architecture, etc. The processor may be a general-purpose processor or a special-purpose processor, and may control other components in the computer to perform desired functions.

[0074] For example, the memory may include any combination of one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and the processor may run one or more computer program modules to implement various functions of the computer.

[0075] The present invention also provides a computer-readable storage medium for storing non-transitory computer-readable instructions. When the non-transitory computer-readable instructions are executed by a computer, one or more steps in the above-mentioned EEG signal classification method can be implemented. When the EEG signal classification model and method provided by the embodiment of the present invention are implemented in the form of software and sold or used as an independent product, they can be stored in a computer-readable storage medium. For relevant descriptions of the storage medium, please refer to the corresponding description of the memory in the computer system above, which will not be repeated here.

Claims

1. An EEG signal classification model, characterized in that: include: A data acquisition module configured to acquire raw EEG signals; A power spectrum feature extraction module, which is configured to extract power spectrum features of the original EEG signal; A differential entropy feature extraction module, which is configured to extract differential entropy features of original EEG signals in different frequency bands; A first feature fusion module is configured to fuse the extracted power spectrum feature with the original EEG signal to obtain a fused EEG signal; A multi-feature fusion convolutional network is configured to perform multiple convolution pooling operations on the fused EEG signal, extract the features of the fused EEG signal, obtain a high-dimensional feature vector, and then map the high-dimensional feature vector to a low-dimensional space through a linear transformation to obtain a low-dimensional feature vector; The second feature fusion module is configured to reduce the dimension of the differential entropy features of the original EEG signals of different frequency bands and splice them with the low-dimensional feature vectors output by the multi-feature fusion convolutional network to obtain EEG data fused with multiple features; The self-attention module captures key features from EEG data that fuse multiple features to obtain the final output vector; as well as The classifier is configured to classify the final output vector of the self-attention module to obtain a classified EEG signal.

2. The EEG signal classification model according to claim 1, characterized in that: The power spectrum feature extraction module performs Hamming window processing on each original EEG signal, calculates the fast Fourier transform of each original EEG signal processed by the Hamming window, squares the result of the fast Fourier transform, and then divides it by half of the number of sampling points to obtain the power spectrum feature.

3. The EEG signal classification model according to claim 1, characterized in that: The multi-feature fusion convolutional network includes: A two-dimensional convolutional layer that applies multiple convolution kernels to the fused EEG signal to generate a feature map; Batch normalization layer, which performs batch normalization on the feature maps generated by the two-dimensional convolutional layer; An activation function that provides a non-zero gradient for negative values ​​in the feature map after the batch normalization layer; The average pooling layer performs an average pooling operation on the feature map processed by the activation function to obtain a high-dimensional feature vector; The fully connected layer maps the high-dimensional feature vector of the feature map after the average pooling operation to a low-dimensional space through linear transformation to obtain the EEG signal after dimensionality reduction.

4. The EEG signal classification model according to claim 3, characterized in that: The differential entropy feature extraction module extracts the differential entropy features of the original EEG signals in 5 frequency bands.

5. The EEG signal classification model according to claim 4, characterized in that: The second feature fusion module reduces the dimension of the differential entropy features of the five frequency bands and concatenates them with the low-dimensional feature vectors output by the fully connected layer to obtain EEG data that integrates multiple features.

6. The EEG signal classification model according to claim 3, characterized in that: The self-attention module is configured to: use linear transformation to respectively calculate the query, key, and value matrices of the EEG data fused with multiple features; obtain the attention weight by calculating the dot product similarity between the query and the key; Normalize the weights to get the attention weight matrix; The value matrix is ​​weighted and summed using the attention weight matrix to obtain the final output vector.

7. A method for classifying electroencephalogram signals, characterized in that: include: The EEG signals of the tester are collected and input into the EEG signal classification model described in any one of claims 1 to 6, and the EEG signal classification model outputs the classified EEG signals.

8. The method for classifying EEG signals according to claim 7, characterized in that: The EEG state reflecting the test subject's emotional / motor imagery content is identified based on the classified EEG signals.

9. A computer system, characterized in that: include: processor; a memory including one or more computer program modules; The one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules include instructions for implementing the EEG signal classification method according to claim 8.

10. A computer-readable storage medium for storing non-transitory computer-readable instructions, characterized in that: When the non-transitory computer-readable instructions are executed by a computer, the EEG signal classification method of claim 8 can be implemented.

Images