EEG Emotion Recognition Methods, Systems, Devices, and Media Based on Adjacent Channel Mutual Information Attention

Abstract

A method, system, device, and medium for EEG emotion recognition based on mutual information attention of adjacent channels are disclosed. The method includes: acquiring EEG data; preprocessing the EEG data; mapping the EEG data into a channel matrix to obtain channel position information and channel data vectors; calculating the average mutual information between each channel data vector and its adjacent channel data vectors; using the average mutual information as the attention weight for that channel data vector; and feeding the EEG data with the attention weight into a CapsNet to complete EEG emotion recognition. The system, device, and medium are used to implement an EEG emotion recognition method based on mutual information attention of adjacent channels. This invention can reduce the influence of noise information caused by electrode placement deviations and improve the accuracy of emotion recognition.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, specifically relating to a brainwave emotion recognition method, system, device, and medium based on mutual information attention between adjacent channels. Background Technology

[0002] Emotions are generated by the brain, and no matter how we try to fake them through speech, facial expressions, or body language, we cannot change the signals generated by the brain. Therefore, directly judging a person's emotional state through electroencephalogram (EEG) signals is the most accurate method. EEG emotion recognition refers to the method of using specific equipment to collect the EEG signals generated by different people watching emotionally stimulating videos, analyzing and processing them, and establishing a mapping relationship between EEG signals and emotional states. Due to the unforgeability of EEG signals, using EEG signals for emotion recognition is of great significance for the study and monitoring of human emotions. In many situations, obtaining the true emotional state of a target can have a significant impact on immediate decision-making. For example, when doctors diagnose or prescribe medication, obtaining the patient's true emotional state can better assist doctors in diagnosis or medication adjustment. When using multi-electrode devices to collect EEG signals, there may be inaccurate electrode placement and differences in individual head shapes, leading to deviations in electrode acquisition positions compared to the standard model. Specifically, this means that a signal from one electrode position in the standard model may be acquired by electrodes in other positions. The information collected in this way, combined with the powerful fitting ability of neural networks, can cause the network to fit a lot of noisy information or fit to an inaccurate feature space, which in turn leads to a decrease in the accuracy of EEG emotion recognition. Summary of the Invention

[0003] To address the problems existing in the prior art, this invention provides an EEG emotion recognition method, system, device, and medium based on mutual information attention of adjacent channels. By calculating the mutual information of adjacent channels and using this mutual information as the channel attention weight, that is, by incorporating mutual information attention of adjacent channels, the influence of noise information caused by electrode placement deviation is weakened, thereby improving the accuracy of emotion recognition.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0005] The EEG emotion recognition method based on mutual information attention between adjacent channels includes the following steps:

[0006] S1, acquire EEG data, preprocess the EEG data, perform channel matrix mapping on the EEG data, and obtain channel location information and channel data vector;

[0007] S2, calculate the average mutual information between each channel data vector and its neighboring channel data vectors, and use the average mutual information as the attention weight of that channel data vector;

[0008] S3 feeds EEG data with attention weights into a CapsNet to complete EEG emotion recognition.

[0009] Specifically, S1 is:

[0010] S1.1, the preprocessing of EEG data specifically involves dividing the EEG emotion data into baseline data and experimental data, and performing a baseline removal operation on the experimental data.

[0011] S1.2, Window the EEG data, with a window duration of t seconds and an EEG signal sampling rate of fHz;

[0012] S1.3, the EEG data is combined with the 10-20 international standard lead system and mapped to a 9×9 channel mapping matrix. The mapped data format is 9×9×(f×t), where the 9×9 matrix represents the channel position information. Each channel is composed of an f×t-dimensional vector, which represents the f×t data collected by the channel in one second.

[0013] Specifically, S2 is:

[0014] S2.1 For each channel vector in the 9×9 matrix, denoted as vector a, create an m×m rectangular window centered on that channel, and obtain all other channel vectors within the m×m window except for the channel vector itself, denoted as the set of vector b.

[0015] S2.2, calculate the mutual information between vector a and each vector in the set of vectors b, and take the average value of the mutual information;

[0016] S2.3, use the average mutual information calculated in S2.2 as the weight of vector a;

[0017] c j =c j ×NMIA j (1)

[0018]

[0019] Among them, c j NMIA represents a channel vector 'a' in a 9×9 matrix. j The mutual information attention weights of adjacent channels of vector a are represented by c; n represents the attention weights of adjacent channels of vector a. jThe number of all other vectors within an m×m window centered on the channel vector, MI(a,b) represents the calculation of the mutual information between vector a and vector b, and window(a) represents all vectors within an m×m window centered on vector a, i.e., the set of vector b.

[0020] In S3, the first convolution kernel size of the CapsNet is 5×5, the second convolution kernel size is 2×2, and the number of iterations for dynamic routing is 3.

[0021] In S1.3, only the channels contained in the EEG data need to be mapped. For channels that are not used when collecting EEG data, and for positions in the matrix where there is no channel information, the value is 0.

[0022] In S2.1, the range of m is: 9 > m > 1, where m is an integer.

[0023] EEG emotion recognition system based on mutual information attention between adjacent channels, including,

[0024] The EEG data mapping module acquires EEG data, preprocesses the EEG data, and performs channel matrix mapping on the EEG data to obtain channel location information and channel data vectors.

[0025] The attention weight acquisition module calculates the average mutual information between each channel data vector and its adjacent channel data vectors, and uses the average mutual information as the attention weight of that channel data vector.

[0026] The emotion recognition module feeds EEG data with attention weights into the capsule grid to complete EEG emotion recognition.

[0027] EEG emotion recognition devices based on mutual information attention between adjacent channels include,

[0028] Memory: A computer program for storing the EEG emotion recognition method based on mutual information attention of adjacent channels;

[0029] Processor: Used to implement an EEG emotion recognition method based on mutual information attention between adjacent channels when executing the computer program.

[0030] A computer-readable storage medium storing a computer program that, when executed by a processor, enables an EEG emotion recognition method based on mutual information attention between adjacent channels.

[0031] Compared with the prior art, the beneficial effects of the present invention are:

[0032] By introducing mutual information attention between adjacent channels and utilizing the potential information correlation between channels, this correlation is mined and deployed into the channel attention mechanism. This reduces the noise information caused by the deviation of electrode placement during the signal acquisition stage, improves the robustness of channel information, and enhances the accuracy of EEG emotion recognition. Attached Figure Description

[0033] Figure 1 This is a flowchart of the EEG emotion recognition method based on mutual information attention between adjacent channels provided in an embodiment of the present invention.

[0034] Figure 2 This is a schematic diagram of the 10-20 international standard lead system used in an embodiment of the present invention.

[0035] Figure 3 This is a schematic diagram of the matrix mapping rules when performing 9×9 matrix mapping in conjunction with the 10-20 international standard lead system, as provided in an embodiment of the present invention.

[0036] Figure 4 This is a schematic diagram of the channel composition information in the instance data group provided in an embodiment of the present invention.

[0037] Figure 5 This is a schematic diagram of the matrix mapping relationship on the instance data group provided in the embodiment of the present invention.

[0038] Figure 6 This is a schematic diagram of the classification experiment results on the instance data group provided in the embodiment of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0040] The application principle of the present invention will be described in detail below with reference to the accompanying drawings.

[0041] like Figure 1 As shown, the EEG emotion recognition method based on mutual information attention between adjacent channels provided in this embodiment of the invention includes the following steps:

[0042] S1, acquire EEG data, preprocess the EEG data, perform channel matrix mapping on the EEG data, and obtain channel location information and channel data vector;

[0043] S2, calculate the average mutual information between each channel data vector and its neighboring channel data vectors, and use the average mutual information as the attention weight of that channel data vector;

[0044] S3 feeds EEG data with attention weights into a CapsNet to complete EEG emotion recognition.

[0045] The EEG emotion recognition method based on mutual information attention between adjacent channels provided in this embodiment of the invention includes the following steps:

[0046] S1.1, The preprocessing of EEG data specifically involves removing the baseline signal from the EEG data, calculating the average value of the baseline signal with a length of one second, and subtracting the baseline signal of that one second length from each second of the experimental signal.

[0047] S1.2, the EEG data is windowed, which splits a complete EEG emotional data into several EEG emotional data segments with a length of 1 second, and the corresponding label of the complete EEG data is also assigned to each of the split data segments. The sampling rate of the EEG signal is 128Hz.

[0048] S1.3, using as follows Figure 3 The 9×9 channel matrix shown maps EEG data to the 10-20 international standard lead system onto the 9×9 channel mapping matrix, assigning spatial location information to the channels that are centrally and sequentially distributed in the data. The mapped data format is 9×9×128, where the 9×9 matrix represents the channel location information, and each channel consists of a 128-dimensional vector, which represents the 128 data points collected by that channel in one second.

[0049] Specifically, S2 is:

[0050] S2.1 For each channel vector in the 9×9 matrix, denoted as vector a, create a 3×3 rectangular window centered on that channel, and obtain all other channel vectors within the 3×3 window except for the channel vector itself, denoted as the set of vector b.

[0051] S2.2, calculate the mutual information between vector a and each vector in the set of vectors b, and take the average value of the mutual information;

[0052] S2.3, use the average mutual information calculated in S2.2 as the weight of vector a;

[0053] c j =c j ×NMIA j (1)

[0054]

[0055] Among them, c j NMIA represents a channel vector 'a' in a 9×9 matrix. j The mutual information attention weights of adjacent channels of vector a are represented by c; n represents the attention weights of adjacent channels of vector a.j The number of all other vectors within a 3×3 window centered on the channel vector, MI(a,b) represents the calculation of the mutual information between vector a and vector b, and window(a) represents all vectors within a 3×3 window centered on vector a, i.e., the set of vector b.

[0056] Mutual information calculation does not take into account the mutual information between vectors themselves.

[0057] In S3, the first convolution kernel size of the CapsNet is 5×5, the second convolution kernel size is 2×2, and the number of iterations for dynamic routing is 3.

[0058] In S1.3, only the channels contained in the EEG data need to be mapped. For channels that are not used when collecting EEG data, and for positions in the matrix where there is no channel information, the value is 0.

[0059] Example

[0060] The EEG data used was the officially preprocessed DEAP dataset, which contains EEG data from 32 subjects. Each subject had 40 EEG segments collected while watching videos. Each video segment contained 63 seconds of EEG data from 32 channels, with a sampling rate of 128 Hz. For each 63-second segment, the first three seconds were the baseline signal, and the last 60 seconds were the experimental signal. For each video, four subjects provided scores: Valence, Arousal, Dominance, and Liking, with scores ranging from 1 to 10. The specific implementation steps on this dataset are as follows:

[0061] 1. Baseline Removal of Initial Data: The official pre-processed data format for a single video is 32×8064, meaning 32 channels × 8064 data points. The 8064 represents 63×128, signifying 63 seconds × 128 sampling. The first three seconds of each video serve as the baseline signal. This baseline signal is divided into three consecutive one-second baseline signals, and their average is calculated to obtain a one-second baseline signal. Next, for the 60-second experimental signal, it is divided into 60 consecutive one-second experimental signal segments. The baseline signal is then subtracted from each segment to obtain the baseline-removed signal. After this process, the effective data format for a video is 32×7680.

[0062] 2. Windowing the Data: Divide the data obtained in the previous step into a window, that is, split 7680 into 60×128, representing 60 consecutive one-second signal segments, each segment being 128 samples long. At this point, the data format of a video is 32×60×128. Next, standardize the 32 channel information obtained from one sampling and transpose the data dimensions. The transposed data format is 60×32×128, representing 60 seconds × 32 channels × 128 samples. Now, the data of one channel within a one-second data segment can be regarded as a vector of length 128.

[0063] DEAP's built-in tags correspond to an entire video, so when windowing, the tags need to be copied to each data segment. For a video, DEAP contains four tags: Valence, Arousal, Dominance, and Liking. After copying the tags, for a 60×32×128 data segment, the corresponding tags are 60×4, representing 60 seconds × 4 tags. Each tag is distributed from 1 to 10. Further processing is needed: tags greater than 5 are set to 1, and tags less than or equal to 5 are set to 0.

[0064] 3. Perform matrix mapping on the channels: The 32 EEG channel names in the DEAP dataset are as follows: Figure 4 As shown. The 10-20 international standard lead system is as follows. Figure 2 As shown. Next, the distribution of the 32 EEG channels in the DEAP dataset, combined with the 10-20 international standard lead system, is mapped into a 9×9 matrix, as shown in the following figure. Figure 5 As shown. After mapping, the data format of a video is 60×9×9×128, representing 60 seconds × (9×9) matrix × 128 samples. The 9×9 matrix records the positions of the 32 channels in the 10-20 international standard lead system.

[0065] 4. Calculate the mutual information and attention of adjacent channels: Next, we begin calculating the mutual information of adjacent channels, taking channel AF3 as an example. The data used for mutual information calculation is data from the same second, specifically the 9×9×128 data format mentioned above. Each 9×9 matrix records the channel's position information, and the 128 represents a 128-dimensional vector for that channel. A 3×3 window is drawn centered on channel AF3. The channels contained within this window are Fp1, F3, and Fz, each with a 128-dimensional vector. Next, the mutual information between the AF3 channel vector and the vectors of these three channels is calculated, and the average is taken. This yields the mutual information of the adjacent channels for this channel.

[0066] The calculated average mutual information is multiplied as a weight onto the 128-dimensional vector of the AF3 channel to complete the implementation of the attention mechanism.

[0067] 5. Training and classifying the model using CapsNet: The CapsNet structure used here is the original structure of the model, with only slight differences in parameter settings, as follows:

[0068] 1) The input image size is 9×9×128, representing a 9×9 image with 128 channels.

[0069] 2) The kernel size of the input image to the convolutional layer is 5×5.

[0070] 3) The kernel size of the convolutional layer to the original capsule layer is 2×2.

[0071] 4) The number of routes set when performing dynamic routing is 3.

[0072] like Figure 6 The figure shows the experimental results of applying this invention to the DEAP dataset. Dotted data represents the accuracy of the model on the training set, and curved data represents the accuracy of the model on the test set. The experiment was conducted for 60 epochs. After the 20th epoch, the accuracy on the test set remained above 95%. The highest accuracy on the test set reached 96.2% across the 60 epochs. These accuracy results on the test set demonstrate the effectiveness of the EEG emotion recognition method based on mutual information attention between adjacent channels proposed in this invention.

[0073] EEG emotion recognition system based on mutual information attention between adjacent channels, including,

[0074] The EEG data mapping module acquires EEG data, preprocesses the EEG data, and performs channel matrix mapping on the EEG data to obtain channel location information and channel data vectors.

[0075] The attention weight acquisition module calculates the average mutual information between each channel data vector and its adjacent channel data vectors, and uses the average mutual information as the attention weight of that channel data vector.

[0076] The emotion recognition module feeds EEG data with attention weights into the capsule grid to complete EEG emotion recognition.

[0077] EEG emotion recognition devices based on mutual information attention between adjacent channels include,

[0078] Memory: A computer program for storing the EEG emotion recognition method based on mutual information attention of adjacent channels;

[0079] Processor: Used to implement an EEG emotion recognition method based on mutual information attention between adjacent channels when executing the computer program.

[0080] A computer-readable storage medium storing a computer program that, when executed by a processor, enables an EEG emotion recognition method based on mutual information attention between adjacent channels.

[0081] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A brainwave emotion recognition method based on mutual information attention between adjacent channels, characterized in that, Includes the following steps: S1, acquire EEG data, preprocess the EEG data, perform channel matrix mapping on the EEG data, and obtain channel location information and channel data vector; S2, calculate the average mutual information between each channel data vector and its neighboring channel data vectors, and use the average mutual information as the attention weight of that channel data vector; S3 feeds EEG data with attention weights into the capsule grid to complete EEG emotion recognition.

2. The EEG emotion recognition method based on mutual information attention between adjacent channels according to claim 1, characterized in that, Specifically, S1 is: S1.1, the preprocessing of EEG data specifically involves dividing the EEG emotion data into baseline data and experimental data, and performing a baseline removal operation on the experimental data. S1.2, Window the EEG data, with a window duration of t seconds and an EEG signal sampling rate of fHz; S1.3, the EEG data is combined with the 10-20 international standard lead system and mapped to a 9×9 channel mapping matrix. The mapped data format is 9×9×(f×t), where the 9×9 matrix represents the channel position information, and each channel is composed of an f×t-dimensional vector, which represents the f×t data collected by the channel in one second.

3. The EEG emotion recognition method based on mutual information attention between adjacent channels according to claim 2, characterized in that, Specifically, S2 is: S2.1 For each channel vector in the 9×9 matrix, denoted as vector a, create an m×m rectangular window centered on that channel, and obtain all other channel vectors within the m×m window except for the channel vector itself, denoted as the set of vector b. S2.2, calculate the mutual information between vector a and each vector in the set of vectors b, and take the average value of the mutual information; S2.3, use the average mutual information calculated in S2.2 as the weight of vector a; (2) in, This represents a channel vector 'a' in a 9×9 matrix. The mutual information attention weights of adjacent channels in vector a represent the values ​​of the two channels; n represents the values ​​of the two channels in vector a. The number of all other vectors within an m×m window centered on the channel vector, MI(a, b) represents the calculation of the mutual information between vector a and vector b, and window(a) represents the set of all other vectors within an m×m window centered on vector a, i.e., vector b.

4. The EEG emotion recognition method based on mutual information attention between adjacent channels according to claim 3, characterized in that, In S3, the first convolution kernel size of the capsule mesh is 5×5, the second convolution kernel size is 2×2, and the number of iterations for dynamic routing is 3.

5. The EEG emotion recognition method based on mutual information attention between adjacent channels according to claim 2, characterized in that, In S1.3, only the channels contained in the EEG data need to be mapped. For channels that are not used when collecting EEG data, and for positions in the matrix where there is no channel information, the value is 0.

6. The EEG emotion recognition method based on mutual information attention between adjacent channels according to claim 3, characterized in that, In S2.1, the range of m is: 9 > m > 1, where m is an integer.

7. A brainwave emotion recognition system based on mutual information attention between adjacent channels, characterized in that, include, The EEG data mapping module acquires EEG data, preprocesses the EEG data, and performs channel matrix mapping on the EEG data to obtain channel location information and channel data vectors. The attention weight acquisition module calculates the average mutual information between each channel data vector and its adjacent channel data vectors, and uses the average mutual information as the attention weight of that channel data vector. The emotion recognition module feeds EEG data with attention weights into the capsule grid to complete EEG emotion recognition.

8. A brainwave emotion recognition device based on mutual information attention between adjacent channels, characterized in that, include, Memory: for storing the computer program of the EEG emotion recognition method based on mutual information attention of adjacent channels as described in any one of claims 1 to 6; Processor: Used to implement the above-mentioned EEG emotion recognition method based on mutual information attention between adjacent channels when executing the computer program.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the implementation of any one of the EEG emotion recognition methods based on mutual information attention between adjacent channels, as claimed in claims 1 to 6.

Images