An EEG emotion recognition method based on brain network and matrix learning

Abstract

The present invention relates to an electroencephalogram (EEG) emotion recognition method based on brain network and matrix learning, comprising: obtaining an EEG data set of a subject, reconstructing the EEG data by using wavelet packet transform and dividing it into N sub-bands, calculating the phase-locking value between the reconstructed EEG data of different brain regions in each sub-band according to the reconstructed EEG data of different brain regions in each sub-band during emotion fluctuation by using Hilbert transform, and creating a brain functional connectivity matrix corresponding to each sub-band according to the phase-locking value between the reconstructed EEG data of different brain regions in each sub-band; constructing an ensemble learning classification model by using N classifiers and training the ensemble learning classification model; using the trained ensemble learning classification model to identify the corresponding feature sets of the brain functional connectivity matrices corresponding to the N sub-bands, obtaining classification results, and obtaining the EEG emotion classification results. This method can extract rich brain network feature information and further improve the accuracy of EEG emotion recognition.

Description

Technical Field

[0001] The present invention belongs to the technical fields of brain-computer interface and emotion classification, and particularly relates to an electroencephalogram emotion recognition method based on brain network and matrix learning. Background Art

[0002] An electroencephalogram (EEG) is a graph obtained by amplifying and recording the spontaneous bioelectric potential of the cerebral cortex of the brain from the scalp using precise instruments. It is the spontaneous and rhythmic electrical activity of brain cell groups recorded by electrodes. EEG is an electrical signal generated by the activities between neurons, which provides rich information for decoding brain emotions and has high temporal resolution and economy. Therefore, emotion recognition based on EEG is a research hotspot in emotion analysis.

[0003] Research in cognitive science and neurophysiology shows that the emotional activities of the brain reflect the integration of brain functions. By studying the global and local topological maps of brain functional connections, the ability of different brain regions to transmit information in different emotional states can be represented. Nodes in the EEG brain network are certain brain regions (mostly regions where scalp electrodes are located), and edges are certain relationships between brain functional signals such as correlation or synchrony; each brain network can be represented in the form of an adjacency matrix. The horizontal and vertical axes of the adjacency matrix are brain region numbers, and the value at the intersection of the horizontal and vertical axes is the connection value between the corresponding brain regions, that is, the correlation value or synchrony value (such as Pearson correlation); by setting a threshold, when the connection value is greater than or equal to the threshold, the corresponding place in the adjacency matrix is regarded as 1, otherwise it is regarded as 0 to obtain a brain functional connection matrix, that is, a binary brain network that only distinguishes between connected and unconnected.

[0004] In traditional methods, a threshold is usually used to retain important connections in the brain network, but the selection of the threshold is too subjective and depends solely on the experience of the experimenter without a fixed criterion, and the experimental results often cannot guarantee the optimal emotion recognition effect. To achieve the optimal emotion classification effect, cross-validation is used to select the threshold, but the time complexity of cross-validation is very high, and it is difficult to fully represent the attribute characteristics of the brain network by the local or global attribute characteristics of the brain network after selecting the appropriate threshold. To fully extract the attribute characteristics of the brain network, researchers use the CSP algorithm to extract the attribute characteristics of the brain network. When the spatial filters trained by the CSP feature extraction method extract the attribute characteristics, only a few columns of features with the largest differences in the brain network matrix are utilized, and the intermediate information is not fully utilized, which greatly affects the final emotion recognition effect. Summary of the Invention

[0005] To solve the problems in the background art, the present invention provides an electroencephalogram emotion recognition method based on brain network and matrix learning, including:

[0006] S1: Obtain the EEG dataset of the subject, and use wavelet packet transform to reconstruct the EEG data and divide it into N sub-bands. Among them, the EEG dataset contains the EEG data generated by different brain regions of the subject during emotional fluctuations;

[0007] S2: According to the EEG reconstruction data of different brain regions in each sub-band during emotional fluctuations, use Hilbert transform to calculate the phase-locking value between the EEG reconstruction data of different brain regions in each sub-band;

[0008] S3: Create a brain functional connectivity matrix corresponding to each sub-band according to the phase-locking value between the EEG reconstruction data of different brain regions in each sub-band; wherein, the horizontal axis and the vertical axis in the brain functional connectivity matrix are the brain region numbers, and the place where the horizontal and vertical intersect is the phase-locking value between the EEG reconstruction data of different brain regions;

[0009] S4: Construct an ensemble learning classification model using N classifiers and train the ensemble learning classification model; use the trained ensemble learning classification model to identify the corresponding feature set of the brain functional connectivity matrix of N sub-bands to obtain a classification result and get the EEG emotion classification result. Among them, the classifier uses a linear regression model.

[0010] The present invention has at least the following beneficial effects:

[0011] In the present invention, the brain functional connectivity matrix is directly used as the input of the model through a linear regression model, which avoids the overly subjective selection of thresholds in traditional recognition. At the same time, it completely retains the attribute characteristics of the brain network represented by the brain functional connectivity matrix, enabling the classification model to fully learn the attribute characteristics of the brain network and improving the accuracy of emotion recognition. The different EEG data are divided into multiple frequency bands to make full use of the information contained in different frequency bands of the EEG data. The ensemble learning method is used to learn multiple classifiers to identify the EEG data of different frequency bands, and the recognition results of different frequency bands are integrated to obtain the final recognition result, making the emotion recognition more accurate. Description of the Drawings

[0012] Drawings of the specification

[0013] Figure 1 It is a flowchart of the method of the present invention;

[0014] Figure 2 It is a schematic diagram of the ensemble learning classification model of the present invention;

[0015] Figure 3 It is a diagram of the effect simulation result of the present invention. Detailed Embodiment

[0016] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0017] Among them, the drawings are only for illustrative purposes, showing only schematic diagrams, not physical diagrams, and should not be construed as a limitation on the present invention; in order to better illustrate the embodiments of the present invention, some components in the drawings will be omitted, enlarged or reduced, which do not represent the dimensions of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0018] In the drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "rear", etc. indicating the orientation or positional relationship, they are based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the positional relationship in the drawings are only for illustrative purposes and should not be construed as a limitation on the present invention. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0019] Please refer to Figure 1 , the present invention provides an electroencephalogram emotion recognition method based on brain network and matrix learning, including:

[0020] S1: Obtain the EEG dataset of the subject, and use wavelet packet transform to reconstruct the EEG data and divide it into N sub-bands. Among them, the EEG dataset contains the EEG data generated by different brain regions of the subject during emotional fluctuations;

[0021] To avoid the overestimation of the accuracy of the test set caused by data leakage, the "homologous sample bundling" method is used to divide the data of a single subject into a training set and a test set. Since the beginning of each stimulus video may show the emotional fluctuations of the previous video, this method only uses the data intercepted from the middle segment of the relevant dataset and uses a non-overlapping sliding window of size 10 seconds to segment the data. Then, the segmented EEG data segments are filtered using wavelet packet transform, reconstructed, and divided into 20 sub-bands (4 - 8 hz, 6 - 10 hz,..., 42 - 45 hz);

[0022] Preferably, the dividing the EEG data reconstructed by using wavelet packet transform into N sub-bands includes:

[0023] Creating a binary decomposition tree by using a wavelet packet decomposition algorithm, where the binary decomposition tree includes i layers, and each layer contains 2 i -1 nodes, and the value corresponding to a certain frequency in the EEG data is corresponding to each node. Reconstructing the EEG data by using the values corresponding to all the nodes in the i-th layer is as follows;

[0024]

[0025] where s(t) represents the reconstructed EEG data, and f ij (t j ) represents the value corresponding to the frequency t j in the EEG data corresponding to the j-th node in the i-th layer;

[0026] According to the frequency range of each divided sub-band, dividing the reconstructed EEG data into L sub-bands, that is, the EEG reconstructed data divided into each sub-band is as follows:

[0027] s(t) d =f i,k (t k )+f i,k+1 (t k+1 )+…+f i,k+n (t k+n ), {t k , t k+1 , …, t k+n}∈(a d , b d

[0028] where (a d , b d represents the frequency range of the d-th sub-band, s(t) d represents the EEG reconstructed data divided into the d-th sub-band, and f i,k+1 (t k+1 ) represents the value corresponding to the frequency t k+1 in the EEG data corresponding to the (k + 1)-th node in the i-th layer.

[0029] ​For example, if the sampling frequency of EEG data is 128 Hz, the Nyquist frequency is 64 Hz. The established binary decomposition tree has 6 layers, and the frequency between two adjacent nodes in the 6th layer is 1 Hz. Then, the EEG reconstruction data divided into the first sub-band is the value corresponding to the 4th to 7th nodes in the 6th layer. That is, assuming x(t) represents the original EEG data, the EEG data in the first sub-band after reconstruction is {x(4), x(5), x(6), x(7)}. By analogy, the original EEG data is reconstructed and divided into 20 sub-bands.

[0030] S2: According to the EEG reconstruction data of different brain regions in each sub-band during emotional fluctuations, use Hilbert transform to calculate the phase-locking value between the EEG reconstruction data of different brain regions in each sub-band;

[0031] Preferably, the phase-locking value between the EEG reconstruction data of different brain regions is calculated using Hilbert transform:

[0032] S21: Use Hilbert transform to calculate the analytic signal (the imaginary part in complex numbers) of the EEG reconstruction data of different brain regions:

[0033]

[0034] where P.V represents the Cauchy principal value, x(t) is the EEG reconstruction data, and HT x (t) represents the analytic signal of x(t).

[0035] S22: Determine the instantaneous phase according to the analytic signals of the EEG reconstruction data of different brain regions;

[0036]

[0037] where, represents the instantaneous phase of x(t).

[0038] S23: Calculate the phase-locking value between the EEG reconstruction data of different brain regions according to the instantaneous phases of the EEG reconstruction data of different brain regions:

[0039]

[0040] where W plv represents the phase-locking value between the EEG reconstruction data x(t) and y(t), represents a signal with a modulus of 1 on the complex plane, represents the instantaneous phase of x(t), represents the instantaneous phase of y(t).

[0041] S3: Create a brain functional connectivity matrix corresponding to each sub - band according to the phase - locking values between the EEG reconstruction data of different brain regions in each sub - band; wherein, the horizontal and vertical axes in the brain functional connectivity matrix are the brain region numbers, and the value at the intersection of the horizontal and vertical is the phase - locking value between the EEG reconstruction data of different brain regions.

[0042] Please refer to Figure 2 , S4: Construct an ensemble learning classification model using N classifiers and train the ensemble learning classification model; use the trained ensemble learning classification model to identify the corresponding feature sets of the brain functional connectivity matrices corresponding to N sub - bands, obtain the classification results, and get the EEG emotion classification results. Among them, the classifier uses a linear regression model.

[0043] Preferably, the training of the ensemble learning classification model includes:

[0044] Train N classifier models corresponding to the brain functional connectivity matrices of N sub - bands respectively. Specifically, input the brain functional connectivity matrix corresponding to each sub - band into the corresponding linear regression model to predict the probability distribution of emotion categories, create a loss function of the linear regression model according to the prediction result of the linear regression model and the label information, and update the parameters of a single linear regression model through the back - propagation mechanism to generate an ensemble learning classification model; the final emotion classification result is the sum of the probability distributions output by N classifier models, and the category with the highest probability is the final classification result.

[0045] For the training set data, train individual learners independently through linear regression models of different frequency bands, and complete the learning task through the strategy of adding the scores of the model classification results, and finally form a strong learner.

[0046] Assume that the probability distribution predicted by the linear regression model is {s1, s2,... sK}, where sK represents the probability of the k - th category.

[0047] Add the prediction results of multiple sub - frequency band models, and the category with the highest value is the final classification category.

[0048] Preferably, the probability distribution of predicting emotion categories includes:

[0049]

[0050] Among them, C i is the brain functional connectivity matrix, is the predicted classification result, L is the number of sub - bands, b represents the bias parameter, T is the transpose operator, W is the weight matrix, is the eigenvector of W, is the eigenvalue of W.

[0051] Preferably, the loss function of the linear regression model includes:

[0052]

[0053] where \(W\) is the weight matrix, \(\|W\|_0\) is the nuclear norm of the weight matrix, is the predicted classification result, and \(y\) i is the label information. \(M\) represents the number of EEG samples. By using the accelerated proximal gradient algorithm, the convex optimization problems of the weight matrix and the bias parameters are solved; the rank of the weight matrix is adjusted to minimize the training error, and the band power of the latent signal is substituted into the trained linear regression model to predict the classification result.

[0054] Please refer to Figure 3 , the present invention has conducted preliminary experiments. Previous studies have shown that CSP is superior to the attribute features of brain networks in terms of effect. As a comparison, the present invention has carried out simulation experiments on the methods: EEG Emotion Classification based on Sparse EEG Latent Space Regression of Brain Networks Topology (SELSER) and common spatial pattern (CSP), and the experiments are carried out on the publicly available EEG datasets DEAP and SEED.

[0055] The results show that: whether in terms of a single sub-band or the final integrated effect, the method SELSER of the present invention is generally superior to the current mainstream methods in emotion decoding.

[0056] The introduction of the relevant datasets is as follows:

[0057] DEAP is a publicly available emotion database established by Sander Koelstra et al., including the physiological signals of 32 participants (16 males and 16 females) watching a 40-minute music video and the psychological scales of the subjects for Valence, Arousal, Dominance, and Liking of the video. Among them, the EEG signals are recorded by 32 active electrodes according to the international 10-20 system.

[0058] SEED (SJTU Emotion EEG Dataset) was released by Lv Baoliang et al. of Shanghai Jiao Tong University, including 62-channel EEG recordings of 15 subjects (7 males and 8 females, 23.27 ± 2.37 years old). This dataset mainly measures three types of emotions (positive, neutral, negative), and each type of emotion is related to five video shots. The officially preprocessed EEG data is used in the experiment.

[0059] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the present technical solution, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A method for electroencephalogram emotion recognition based on brain network and matrix learning, characterized in that, Including: S1: Obtain the EEG dataset of the subject, and use wavelet packet transform to reconstruct the EEG data and divide it into N sub-bands. Among them, the EEG dataset contains EEG data generated by different brain regions of the subject during emotional fluctuations; S2: According to the reconstructed EEG data of different brain regions in each sub-band during emotional fluctuations, use Hilbert transform to calculate the phase locking value between the reconstructed EEG data of different brain regions in each sub-band; S3: Create a brain functional connectivity matrix corresponding to each sub-band according to the phase locking value between the reconstructed EEG data of different brain regions in each sub-band; among them, the horizontal axis and the vertical axis in the brain functional connectivity matrix are the brain region numbers, and the place where the horizontal and vertical intersect is the phase locking value between the reconstructed EEG data of different brain regions; S4: Construct an ensemble learning classification model using N classifiers and train the ensemble learning classification model; use the trained ensemble learning classification model to identify the corresponding feature set of the brain functional connectivity matrix corresponding to N sub-bands, obtain the classification result, and obtain the EEG emotion classification result. Among them, the classifier uses a linear regression model.

2. The EEG emotion recognition method based on brain network and matrix learning according to claim 1, wherein, The process of using wavelet packet transform to reconstruct the EEG data and divide it into N sub-bands includes: Create a binary decomposition tree using the wavelet packet decomposition algorithm. The binary decomposition tree includes i layers, and each layer contains x i -1 nodes. Each node corresponds to a value at a certain frequency in the EEG data. Reconstruct the EEG data using the values corresponding to all the nodes in the i-th layer as follows; Among them, s(t) represents the reconstructed EEG data, and f ij (t j ) represents the value corresponding to the frequency t in the EEG data of the j-th node in the i-th layer j at that time; According to the frequency range of each divided sub-band, divide the reconstructed EEG data into L sub-bands, that is, the reconstructed EEG data divided into each sub-band is as follows: s(t) d = f i，k (t k ) + f i，k+1 (t k+1 ) + … + f i，k+n (t k+n ), {t k , t k+1 , …, t k+n} ∈ (a d , b d ) Among them, (a d , b d ) represents the frequency range of the d-th sub-band, s(t) d represents the EEG reconstruction data divided into the d-th sub-band, f i，k+1 (t k+1 ) represents the value at frequency t k+1 in the EEG data corresponding to the (k + 1)-th node in the i-th layer.

3. A method for electroencephalogram emotion recognition based on brain network and matrix learning according to claim 1, characterized in that, The process of using Hilbert transform to calculate the phase locking value between the reconstructed EEG data of different brain regions includes: S21: Use Hilbert transform to calculate the analytic signal of the reconstructed EEG data of different brain regions; S22: Determine the instantaneous phase according to the analytic signal of the reconstructed EEG data of different brain regions; S23: Calculate the phase locking value between the reconstructed EEG data of different brain regions according to the instantaneous phase of the reconstructed EEG data of different brain regions.

4. A method for electroencephalogram emotion recognition based on brain network and matrix learning according to claim 3, characterized in that, The analytic signal of the reconstructed EEG data of different brain regions includes: where P.V. represents the Cauchy principal value, x(t) is the EEG reconstruction data, and HT x (t) represents the analytic signal of x(t).

5. The EEG emotion recognition method based on brain network and matrix learning according to claim 3, characterized in that The phase locking value between the reconstructed EEG data of different brain regions includes: Among them, W ply represents the phase locking value between the EEG reconstruction data x(t) and y(t), represents a signal with a modulus of 1 on the complex plane, represents the instantaneous phase of x(t), represents the instantaneous phase of y(t).

6. The EEG emotion recognition method based on brain network and matrix learning according to claim 1, wherein The process of training the ensemble learning classification model includes: Train N classifier models corresponding to the brain functional connectivity matrices corresponding to N sub-bands respectively. Specifically, it includes: input the brain functional connectivity matrix corresponding to each sub-band into the corresponding linear regression model to predict the probability distribution of the emotion category, create the loss function of the linear regression model according to the prediction result of the linear regression model and the label information, and update the parameters of the single linear regression model through the backpropagation mechanism to generate the ensemble learning classification model; the final emotion classification result is the sum of the probability distributions output by N classifier models, and the category with the highest probability is the final classification result.

7. The electroencephalogram emotion recognition method based on brain network and matrix learning according to claim 6, wherein The prediction of the probability distribution of the emotion category includes: Among them, C i is the brain functional connectivity matrix, is the predicted classification result, L is the number of sub-bands, b represents the bias parameter, T is the transpose operator, W is the weight matrix, is the eigenvector of W, is the eigenvalue of W.

8. The electroencephalogram emotion recognition method based on brain network and matrix learning according to claim 7, characterized in that, The loss function of the linear regression model includes: Among them, W is the weight matrix, and ||W||0 is the nuclear norm of the weight matrix. is the predicted classification result, and y i is the label information, and M represents the number of EEG samples.

Images