Channel attention residual network emotion electroencephalogram recognition method

Abstract

The present application relates to a kind of passage attention residual network emotional electroencephalogram identification method, belong to brain information decoding technical field.The present application uses Butterworth band-pass filter to obtain the target frequency band of original electroencephalogram, combines differential entropy algorithm, respectively to target frequency band obtains feature and it is converted into 4D representation according to electrode channel position.Extracted multi-band differential entropy feature is used as the input of model, using passage attention residual network to learn feature, channel attention mechanism can be well simulated electrode channel information and enhance convolution feature learning, and can significantly improve emotional classification accuracy.The present application can be used as a kind of combination of time, frequency and spatial information emotional electroencephalogram decoding strategy, the proposed passage attention residual network model provides new ideas for building better emotional recognition framework in the future.

Description

Technical Field

[0001] This invention relates to a channel attention residual network-based emotion EEG recognition method, belonging to the field of brain information decoding technology. Background Technology

[0002] In recent years, EEG-based emotion recognition has shown great potential in the field of human-computer interaction, and building efficient emotion recognition models is crucial for improving machine intelligence. EEG signals are non-stationary and highly irregular; even a subtle change in stimulus requires extensive analysis for accurate identification. Decoding requires effective extraction of signal features, and with proper design, efficient decoding methods can be developed. Emotion recognition has wide applications in human-computer interaction, psychology, and neuroscience. Among these, EEG-based emotion recognition is currently a hot research topic. Emotion is an individual's attitude and related behavioral response to objective things. It is a complex psychological and physiological state, generated by the interaction of emotions, thoughts, and actions. Changes in emotion often cause alterations in physiological and non-physiological signals, including tone of voice, body posture, facial expressions, and EEG. Patterns used to detect emotional states mainly fall into two categories: the first is external behavioral signals, such as speech, text, and facial expressions; the second is internal physiological signals, including EEG and skin conductance responses. Both possess prominent human emotional characteristics. However, external behavioral signals are easily influenced by the participant's subjectivity; that is, participants can deliberately hide their emotions. Therefore, most existing literature is based on electroencephalography (EEG) to study emotion classification. How to accurately identify human emotional states remains an unresolved problem. Researchers have conducted extensive research on this issue, initially extracting single features for identification, such as only time-domain or frequency-domain features. However, the identification results were poor. Later, researchers discovered that fusing multiple features from multiple frequency bands could increase the identification accuracy, such as fusing frequency-domain features from four frequency bands. Previously, most identification models used machine learning methods for classification, such as support vector machines. Later, researchers abandoned machine learning, which required manual feature extraction, and instead used deep learning, such as convolutional neural networks and recurrent neural networks, which can automatically extract features end-to-end. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a channel attention residual network emotion EEG recognition method to solve the above-mentioned problem.

[0004] The technical solution of this invention is: a channel attention residual network emotion EEG recognition method. First, the original signal is divided into n segments, and each segment is processed by a 0.5-second non-overlapping sliding window. Then, a Butterworth bandpass filter is used to divide the original signal into several required sub-bands. Differential entropy features are calculated for each band, and the features are mapped onto a two-dimensional map according to the channel position. The features of the four bands are superimposed and converted into a 4D representation. The extracted features are input into a residual network containing a channel attention mechanism to learn feature information. Finally, a softmax function is used to classify emotions.

[0005] The specific steps are as follows:

[0006] Step 1: First, divide the raw EEG signal into n non-overlapping segments. Each segment is assigned the same label as the raw EEG signal. Use a 0.5-second non-overlapping sliding window to increase the data volume for each of the n segments. Then, use a Butterworth bandpass filter to divide the raw EEG signal into four frequency bands.

[0007] Step 2: Calculate the differential entropy feature for each frequency band signal in Step 1, map the feature onto a two-dimensional map according to the channel position, and transform it into a 4D representation.

[0008] Step 3: Input the differential entropy features extracted in Step 2 into a residual network with a channel attention mechanism to learn feature information, and finally use the softmax function to classify and predict emotions.

[0009] Step 1 specifically involves: before extracting differential entropy features, the original EEG signal is divided into n non-overlapping segments, each segment being T seconds long, and each segment being assigned the same label as the original EEG signal. A 0.5-second non-overlapping sliding window is used, doubling the amount of data. A Butterworth bandpass filter is then used to divide the original EEG signal into four frequency bands: θ (4-7Hz), α (8-13Hz), β (14-30Hz), and γ (31-45Hz).

[0010] Step 2 specifically includes:

[0011] The differential entropy characteristics of the four frequency bands in Step 1 are calculated according to Equation (1). Differential entropy is a generalization of Shannon information entropy in information theory to continuous variables. It can be used to measure the complexity of EEG signals and reflect the differences in activity in a certain region. Specifically:

[0012] DE=H(X=-∫ X p(X)log(p(x))dx (1)

[0013] In the formula, X is a free variable, and p(x) represents the probability density function of X, which approximately follows a Gaussian distribution N(μ,σ) over a certain length. 2 For the EEG signal of ), the differential entropy calculation can be expressed as:

[0014]

[0015] In the formula, σ 2 μ and μ represent the variance and mean of X, respectively.

[0016] Step 3 specifically refers to:

[0017] After feature extraction, a residual network with channel attention mechanism is used to classify and recognize the emotion features extracted by the differential entropy algorithm. ResNet's advantage lies in solving the vanishing and exploding gradient problems. For a stacked layer structure (composed of several layers), when the input is x, its learned features are denoted as H(x). We hope it can learn the residual F(x) = H(x) - x, so its original learned features are F(x) + x. When the residual is 0, the stacked layers only perform an identity mapping, and the network performance does not decrease. In reality, the residual is not 0, which allows the stacked layers to learn new features based on the input features, thus achieving better performance.

[0018] Therefore, the residual unit in the residual network is specifically:

[0019] y=F(x,{W i})+x (3)

[0020] In the formula, x and y are the input and output vectors of the layer under consideration, and the function F(x,{W i}) represents the residual mapping to be learned.

[0021] The channel attention mechanism uses a squeeze-excitation module to calibrate the filter, which includes two main computational units to calibrate the filter. That is, the squeeze-excitation module includes a squeeze module and an excitation module.

[0022] Before being input into the squeezing module, the input feature map X∈R H′×W′×C′ After pre-sorted convolution operation F tr We obtain tensor U∈R H×W×C .

[0023] When input into the compression module, the channel dimensions of the feature map are kept constant. The channel information is aggregated through a global average pooling layer, which essentially reduces the spatial dimension H×W×C to 1×1×C. Then, global information is embedded to generate the channel statistical information z∈R. c Then the k-th element of z is defined as:

[0024]

[0025] The activation module uses the information aggregated in the squeezing module to find the correlation between electrode channels and learns the activation of specific samples through a self-selection gate mechanism based on electrode channel dependence. This allows each electrode channel to learn to use global information to selectively emphasize useful information features and suppress features with lower weights.

[0026] The activation module includes two fully connected layers. The first fully connected layer is used to reduce dimensionality, and the second fully connected layer is used to restore the original dimensionality. Its output is limited to between 0 and 1. Specifically:

[0027]

[0028] Where z is the input of the fully connected layer, W1 and W2 are the weights of the two fully connected layers, τ(·) represents the ReLU activation function, and σ(·) represents the sigmoid activation function.

[0029] Finally, the sentiment is classified and predicted using the softmax function, specifically as follows:

[0030]

[0031] In the formula, x i Let Z be the output of the i-th node, and Z be the number of output nodes, which is also the number of classification categories. The softmax function can be used to convert the output values ​​of multi-class classification into a probability distribution in the range [0,1].

[0032] The beneficial effects of this invention are as follows: The residual network model with channel attention mechanism constructed in this invention achieved excellent performance in the final test, providing a new approach for building a better recognition framework for subsequent emotion classification and other tasks. It also provides a new and alternative method for feature extraction of emotional EEG signals. Furthermore, this work can contribute to further research and practical applications of EEG emotion recognition. Attached Figure Description

[0033] Figure 1 This is a flowchart of the feature extraction process of the present invention;

[0034] Figure 2 This is a mapping diagram of the 64 electrode positions of the present invention;

[0035] Figure 3 This is a framework diagram of the channel attention residual network EEG emotion recognition method of the present invention;

[0036] Figure 4 This is a diagram illustrating the weighted process of the channel attention mechanism of the present invention;

[0037] Figure 5 This is the confusion matrix obtained from the experiment on subject 1 in this invention. Detailed Implementation

[0038] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0039] Example 1:

[0040] 1. Emotion feature extraction strategy;

[0041] Extracting effective features can increase the model's recognition accuracy. Emotion recognition is a very promising research direction. It can be divided into two categories based on the recognition signal: non-physiological signal recognition and physiological signal recognition. Because non-physiological signals are very deceptive, although they are easy to extract, most researchers choose to use physiological signals for recognition in order to accurately identify emotion types.

[0042] Figure 1 This invention employs a feature extraction method. Experiments have shown that when the human brain performs emotional tasks, the best classification results are achieved by superimposing four frequency bands of collected EEG signals: θ (4-7Hz), α (8-13Hz), β (14-30Hz), and γ (31-45Hz). Other experiments have compared differential entropy, power spectral density, differential asymmetry, rationality asymmetry, and differential causality features, concluding that differential entropy is the most stable feature for emotion recognition tasks. Therefore, this invention selects to extract and fuse the differential entropy features of four frequency bands as input to the proposed model. First, the original signal is processed using a 0.5-second non-overlapping sliding window. Then, a Butterworth bandpass filter is used to divide the original signal into four frequency bands. Differential entropy features are then calculated for each of the four frequency bands and superimposed, finally converting the signal into a 4D representation. To facilitate visualization of the extracted features, we map the features onto a two-dimensional map based on the channel location. That is, the feature information carried by each channel is mapped onto an 8×9 two-dimensional map, where 0 indicates no electrode at that location. The channel mapping is as follows: Figure 2 As shown.

[0043] Differential entropy is a generalization of Shannon information entropy in information theory to continuous variables. It can be used to measure the complexity of electroencephalogram (EEG) signals and reflect the differences in activity within a specific region. It is defined as follows:

[0044] DE=H(X=-∫ X p9X)log(p(x))dx(1)

[0045] In the formula, X is a free variable, and p(x) represents the probability density function of X, which approximately follows a Gaussian distribution N(μ,σ) over a certain length. 2 For the EEG signal of ), the differential entropy calculation can be expressed as:

[0046]

[0047] In the formula, σ 2 μ and μ represent the variance and mean of X, respectively.

[0048] 2. Deep learning-based model building strategies;

[0049] Deep learning has gained popularity among researchers in many fields due to its end-to-end model architecture and automatic feature extraction capabilities. However, the framework for emotion recognition using convolutional neural networks is still less mature than in computer vision, image classification, and visual tracking, requiring further exploration of recognition frameworks. Generally, the lower layers of a convolutional neural network learn general feature representations of the data, while the higher layers learn specific feature representations. ResNet is also a type of convolutional neural network, with its internal structure composed of many residual blocks.

[0050] like Figure 3 As shown, we constructed a residual network with an attention mechanism to extract signal features from different domains for emotion EEG decoding. The model proposed in this patent is based on ResNet with some structural adjustments. The advantage of ResNet lies in solving the vanishing and exploding gradient problems. For a stacked layer structure (composed of several layers), the features it learns when the input is x are denoted as H(x). We hope that it can learn residuals.

[0051] F(x) = H(x) - x, so its original learned feature is F(x) + x. When the residual is 0, the stacked layer only performs an identity mapping, and the network performance will not decrease. In fact, the residual will not be 0, which will also allow the stacked layer to learn new features based on the input features, thus having better performance. A residual unit is as shown in formula (3):

[0052] y=F(x,{W i})+x(3)

[0053] Here, x and y are the input and output vectors of the layer under consideration. The function F(x, {W i}) represents the residual mapping to be learned.

[0054] For the channel attention mechanism, a squeeze-excitation module (SENet) is used, such as... Figure 4 As shown, it includes two main computational units for calibrating the filter: a squeezing module and an excitation module. Before the squeezing module, the input feature map X∈R... H ′×W′×C′ After a series of pre-sorted convolution operations F tr We obtain tensor U∈R H×W×CIn the compression module, the channel dimensions of the feature map are kept constant. A global average pooling layer aggregates the channel information to generate channel statistics, essentially reducing the spatial dimension H×W×C to 1×1×C. Then, global information is embedded. The statistical information z∈R c Then the definition of the k-th element of z is as shown in equation (4):

[0055]

[0056] The activation module uses the aggregated information in the squeeze block to find the correlation between electrode channels and learns the activation of specific samples through a self-gating mechanism based on electrode channel dependencies. This allows each electrode channel to learn to selectively emphasize useful information features and suppress features with lower weights using global information. The module includes two fully connected layers. The first fully connected layer reduces dimensionality, and the second fully connected layer restores the original dimensionality. Its output is constrained between 0 and 1, as defined in formula (5).

[0057]

[0058] Where z is the input of the fully connected layer, W1 and W2 are the weights of the two fully connected layers, τ(·) represents the ReLU activation function, and σ(·) represents the sigmoid activation function.

[0059] Finally, softmax is used to classify and predict sentiment, as defined below:

[0060]

[0061] In the formula, x i Let Z be the output of the i-th node, and Z be the number of output nodes, which is also the number of classification categories. The softmax function can be used to convert the output values ​​of multi-class classification into a probability distribution in the range [0,1].

[0062] 3. Experiment

[0063] 3.1 Obtaining the experimental dataset:

[0064] In this embodiment, the SEED public emotion dataset was used to evaluate the experimental results, without involving ethical issues. The SEED dataset is a video-evoked multichannel emotion EEG dataset collected based on the international 10-20 system. It records EEG data from 15 participants (7 men and 8 women) from China watching videos, and also includes eye-tracking data from participants 1-5 and 8-14 (12 participants). Each participant underwent three experiments, with a total of 15 trials per experiment, spaced approximately one week apart. Self-assessment was performed after each experiment. The data was downsampled to 200 Hz and filtered using a bandpass filter from 0-75 Hz. Emotions in the dataset were categorized into three classes: -1 for negative, 0 for neutral, and 1 for positive. A total of 20,268 samples from each participant's three experiments were used for the emotion classification task.

[0065] 3.2 To verify the decoding performance of the proposed method on emotional EEG data, some parameters in the model training were analyzed and adjusted. The entire experiment was based on Python 3.9, Tensorflow 2.6 deep learning framework and EEG analysis toolkit MNE.

[0066] The model was trained with a learning rate of 0.001, the optimizer used the efficient Adam algorithm, each iteration was set to 64 batches, and 5x cross-validation was used. Finally, the softmax function was used for sentiment classification. Figure 5 This is the preliminary experimental confusion matrix of this scheme on subject 1 in the SEED dataset. It can be seen that the recognition accuracy is 91% for negative emotions, 100% for neutral emotions, and 99% for positive emotions.

[0067] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A method for emotion EEG recognition using a channel attention residual network, characterized in that, The specific steps are as follows: Step 1: First, divide the raw EEG signal into n non-overlapping segments. Each segment is assigned the same label as the raw EEG signal. Use a 0.5-second non-overlapping sliding window to increase the data volume for each of the n segments. Then, use a Butterworth bandpass filter to divide the raw EEG signal into four frequency bands. Step 2: Calculate the differential entropy features for each frequency band signal in Step 1, map the features onto a two-dimensional map based on the channel location, and convert it into a 4D representation; Step 3: Input the differential entropy features extracted in Step 2 into a residual network with a channel attention mechanism to learn feature information, and finally use the softmax function to classify and predict emotions. Step 3 specifically refers to: The residual units in the residual network are specifically: （3）； In the formula, and These are the input and output vectors of the layer under consideration, and the function. This represents the residual mapping to be learned; The channel attention mechanism uses a squeeze-excitation module to calibrate the filter, and the squeeze-excitation module includes a squeeze module and an excitation module. The input feature map is input before being fed into the extrusion module. After pre-sorted convolution operation To obtain the tensor ; When inputting into the squeezing module, the channel dimension of the feature map is kept unchanged. The channel information is then aggregated through a global average pooling layer to generate channel statistics. Then the k-th element of z is defined as: （4）； The activation module includes two fully connected layers. The first fully connected layer is used to reduce dimensionality, and the second fully connected layer is used to restore the original dimensionality. Its output is limited to between 0 and 1. Specifically: （5）； Where z is the input of the fully connected layer, and These are the weights of the two fully connected layers, where τ(∙) represents the ReLU activation function and σ(∙) represents the sigmoid activation function. Finally, the sentiment is classified and predicted using the softmax function, specifically as follows: （6）； In the formula, Let Z be the output of the i-th node, and Z be the number of output nodes, which is also the number of classification categories. The softmax function is used to convert the output values ​​of the multi-class classification into a probability distribution in the range [0, 1].

2. The channel attention residual network emotion EEG recognition method according to claim 1, characterized in that, Step 1 specifically involves: dividing the original EEG signal into n non-overlapping segments, each segment being T seconds long, and assigning the same label to each segment as the original EEG signal. A 0.5-second non-overlapping sliding window is used, doubling the data volume. A Butterworth bandpass filter is then used to divide the original EEG signal into four frequency bands, namely... (4-7Hz) (8-13Hz), (14-30Hz) (31-45Hz).

3. The channel attention residual network emotion EEG recognition method according to claim 1, characterized in that, Step 2 specifically includes: The differential entropy characteristics of the four frequency bands in Step 1 are calculated according to equation (1): （1）； In the formula, It is a free variable. express The probability density function of the expression approximately follows a Gaussian distribution over a certain length. The differential entropy calculation for the EEG signal is expressed as: （2）； In the formula, and Represent The variance and mean.

Images