An emotion recognition method based on brain connection reserve pool and graph convolutional neural network

Abstract

An emotion recognition method based on a brain connection reserve pool and a graph convolutional neural network applies the bidirectional feature representation and valence lateralization of EEG signals to EEG emotion recognition. The bidirectional feature representation of EEG signals is performed through the brain connection reserve pool, which not only comprehensively considers the timing, correlation and nonlinearity of EEG signals, but also enhances the long-term memory capacity of the brain connection reserve pool. Valence lateralization is applied to the training of the brain connection reserve pool and the graph convolutional neural network, which improves the feature representation ability of the brain connection reserve pool and the graph convolutional neural network for EEG signals.

Description

Technical Field

[0001] The present invention belongs to the field of computer application technology, and in particular relates to an emotion recognition method based on a brain connection reserve pool and a graph convolutional neural network. Background Art

[0002] The advent of the intelligent era is unstoppable, and emotion recognition is particularly important. Emotion recognition can be used in various fields to monitor individual emotional states. For example, in the field of medical research, it is used to monitor the emotional state of patients and take treatment measures; in the field of product design, it is used to understand the emotional changes of users and improve user experience. Therefore, emotion recognition is an important part of establishing a harmonious, friendly, and intelligent human-computer interaction environment. The methods of EEG emotion recognition are mainly divided into two methods, namely feature extraction methods and classification methods. Most traditional emotion recognition technologies represent EEG signals in time domain, frequency domain, etc. to classify emotions, but most of them do not comprehensively consider the temporal, nonlinear and correlation of EEG signals and the processing method combined with the emotional lateralization of the brain. The accuracy of emotion recognition in this method is low. Summary of the invention

[0003] In view of the deficiencies in the above-mentioned background technologies, the present invention provides an emotion recognition method based on a brain connection reserve pool and a graph convolutional neural network.

[0004] An emotion recognition method based on a brain connection reserve pool and a graph convolutional neural network, characterized in that the method comprises the following steps:

[0005] Step S1: establishing a feature representation model based on a brain connection reserve pool and an emotion recognition model based on a graph convolutional neural network, the former is used to obtain a bidirectional feature representation of the original EEG signal, and the latter is used to classify emotions based on the bidirectional feature representation of the EEG signal;

[0006] Step S2: dividing the brain connection reserve pool into left and right hemisphere connection reserve pools, and training the left and right hemisphere connection reserve pools and graph convolutional neural networks through a valence lateralization-based training algorithm;

[0007] Step S3: Use the trained brain connection reserve pool-based feature representation model to perform bidirectional feature representation on the original EEG signal, and use the trained graph convolutional neural network-based emotion recognition model to perform four-category emotion classification on the bidirectional feature representation of the EEG signal.

[0008] Furthermore, in step S1, a feature representation model based on the brain connection reserve pool and an emotion recognition model based on a graph convolutional neural network are established. The specific steps are as follows:

[0009] Step S1-1: The feature representation model based on the brain connection reserve pool is a two-layer network structure. The first layer is the input layer, which is used to receive EEG signals. The weight matrix of the input layer is W in ; The second layer is the brain connection reserve pool, which is used to generate a bidirectional feature representation of the EEG signal. The weight matrix of the brain connection reserve pool is W, and the activation function of the brain connection reserve pool is the hyperbolic tangent function tanh();

[0010] Step S1-2: The EEG signal X input by the input layer is expressed as:

[0011]

[0012] Among them, X(t) represents the EEG signals of all channels at time t, X j , X k represents the EEG signals at all times in channels j and k, respectively, and x j (t) and x k (t) represents the EEG signals at time t in channels j and k respectively, D is the duration of the EEG signal, and N is the number of EEG channels;

[0013] Step S1-3: Randomly initialize the weight matrix W of the input layer in ; The structure of the brain connection reserve pool is determined by calculating the correlation between each channel to initialize the weight matrix W of the brain connection reserve pool = {ω jk |j∈[1,N],k∈[1,N]}; Specifically, the correlation between channel j and channel k is based on the weighted phase lag index α WPLI (X j ,X k ) is measured by the following formula:

[0014]

[0015] Where Δθ is X j and X k The phase difference between them, sign() is the sign function, l j and l k Respectively represent X j and X k The amplitude after Hilbert transformation, and j, k∈[1,N]; each neuron in the brain connection reserve pool corresponds to each EEG channel; determine a threshold ε, if α WPLI (X j ,X k ) is greater than ε, then it is considered that there is a connection between channels j and k, and the connection weight between channel j and channel k is ω jk =α WPLI (X j ,X k ); otherwise, there is no connection, and ωjk =0; finally, the connection weights between all channels are obtained to form the weight matrix W of the brain connection reserve pool;

[0016] Step S1-4: The emotion recognition model based on graph convolutional neural network is a three-layer network structure, and the first two layers are graph convolution layers; the first graph convolution layer is used to perform graph convolution on the bidirectional feature representation to obtain the bidirectional graph convolution feature representation of the EEG signal, and the second graph convolution layer is used to fuse the bidirectional graph convolution feature representation to obtain the fused feature representation of the EEG signal; the weight matrix of the first graph convolution layer is W (0) , the weight matrix of the second graph convolutional layer is W (1) , the weight matrices of the first two graph convolutional layers are randomly initialized, and their activation functions are ReLU(); the third layer is a fully connected layer, which is used to classify the fused feature representation to obtain the emotion category of the EEG signal. The weight matrix and fixed bias term of the fully connected layer are W c and b are randomly initialized, and the activation function of the fully connected layer is softmax().

[0017] Further, in step S2, the brain connection reserve pool and graph convolutional neural network are trained based on the valence lateralization training algorithm, and the specific steps are as follows:

[0018] Step S2-1: The input EEG signal is Calculate the cross entropy loss function Loss, the formula is as follows:

[0019]

[0020] Among them, X i is the EEG signal of the ith sample, is the EEG signal of channel j of the ith sample, m is the number of samples, y i are the emotion classification result and actual emotion category of the i-th sample respectively, and 0 represents depression, 1 represents anger, 2 represents relaxation, and 3 represents happiness. 0 and 1 are negative emotions, while 2 and 3 are positive emotions.

[0021] Step S2-2: Through back propagation, the loss function Loss is derived and W is calculated c , W (1) and W (0) The gradients of and And update W c , W (1) and W (0) ; Among them, W c , W (1) and W (0)are the weight matrices of the fully connected layer, the second graph convolutional layer, and the first graph convolutional layer respectively; the brain connection reserve pool is divided into the left and right brain connection reserve pools; specifically, the weight matrix W of the brain connection reserve pool is divided into the weight matrix W of the left brain connection reserve pool l And the right hemisphere connection reserve pool weight matrix W r ;

[0022] in,

[0023] W = {ω jk |j∈[1,N],k∈[1,N]}

[0024] W l ={ω jk |j∈[1,N / 2],k∈[1,N / 2]}

[0025] W r ={ω jk |j∈[N / 2+1,N],k∈[N / 2+1,N]}

[0026] Therefore, for the training of brain connection reserve pool, according to the valence lateralization, when y i is positive emotion, i.e. i ∈{2,3}, then calculate W l Gradient On the contrary, when y i is negative emotion, i.e. i ∈{0,1}, then calculate W r Gradient And update W l and W r , and also W l and W r Update synchronously to W; until the loss function converges and the training is completed, finally obtaining the trained feature representation model based on the brain connection reserve pool and the emotion recognition model based on the graph convolutional neural network.

[0027] Further, in step S3, the trained feature representation model based on the brain connection reserve pool is used to perform bidirectional feature representation on the original EEG signal, and the trained emotion recognition model based on the convolutional neural network is used to perform four emotion classifications on the bidirectional feature representation of the EEG signal. The specific emotion categories are relaxation, happiness, depression and anger; among them, relaxation and happiness are positive emotions, and depression and anger are negative emotions. The specific steps are as follows:

[0028] Step S3-1: Input the original EEG signal X into the feature representation model based on the brain connection reserve pool to obtain the bidirectional feature representation H(D) of the EEG signal; specifically, After inputting into the brain connection reserve pool, we get Will Input brain connection reserve pool The calculation formula is as follows:

[0029]

[0030] in, and represent the positive sequence EEG signal and the reverse sequence EEG signal at time t in channel j, respectively, and and They represent the positive sequence reserve pool state and the reverse sequence reserve pool state at time t in channel j, respectively. D is the duration of the EEG signal, and W in and W are the weight matrices of the input layer and the brain connection reserve pool after training in step S2, respectively. Iterate the above formula according to time t to obtain the reserve pool state D at the last moment. and Bidirectional feature representation of EEG signal as channel j Then, the above formula is iterated according to channel j, and finally the bidirectional feature representation of the EEG signals of all channels is obtained: in, and Respectively represent the positive sequence feature representation and the reverse sequence feature representation of the EEG signals of all channels;

[0031] Step S3-2: The bidirectional feature representation H(D) of the EEG signal and W obtained by training in step S2 form a positive sequence feature representation diagram of the EEG signal and reverse feature representation And merge into graph G = {(H(D),W)};

[0032] Step S3-3: Obtain bidirectional graph convolution feature representation of EEG signal through the first graph convolution layer and The calculation formula is as follows:

[0033]

[0034] Among them, g(.) is the convolution function, L is the Laplace matrix of graph G, U is the standard orthogonal matrix corresponding to L, Λ=diag([λ0,...,λ N-1 ]) is the diagonal matrix of W, λ0,...,λ N-1 is the eigenvalue of W, W (0) is the weight matrix of the first graph convolutional layer after training in step S2;

[0035] Step S3-4: Obtain the fusion feature representation H of the EEG signal through the second graph convolution layer (1) , the calculation formula is as follows:

[0036]

[0037] Among them, W (1) is the weight matrix of the second graph convolutional layer after training in step S2;

[0038] Step S3-5: Representing the fusion feature H of the EEG signal (1) Emotion classification is performed through the fully connected layer to obtain the emotion category of the EEG signal The calculation formula is as follows:

[0039]

[0040] in, 0 represents depression, 1 represents anger, 2 represents relaxation, and 3 represents happiness. 0 and 1 are negative emotions, 2 and 3 are positive emotions. c is the weight matrix of the fully connected layer after training in step S2, and b is the fixed bias term of the fully connected layer.

[0041] The beneficial effect of the present invention is that the bidirectional feature representation and valence lateralization of EEG signals are applied to EEG emotion recognition. The bidirectional feature representation of EEG signals through the brain connection reserve pool not only comprehensively considers the timing, correlation and nonlinearity of EEG signals, but also enhances the long-term memory capacity of the brain connection reserve pool. The valence lateralization is applied to the training of the brain connection reserve pool and the graph convolutional neural network, which improves the feature representation ability of the brain connection reserve pool and the graph convolutional neural network for EEG signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a specific flow chart of an embodiment of the present invention.

[0043] Figure 2 Schematic diagram of a feature representation model based on a brain connection reserve pool according to an embodiment of the present invention.

[0044] Figure 3 Schematic diagram of an emotion recognition model based on a graph convolutional neural network according to an embodiment of the present invention.

[0045] Figure 4 Schematic diagram of the training algorithm flow based on valence lateralization according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings.

[0047] like Figure 1 As shown, the overall process of the embodiment of the present invention includes:

[0048] Step S1: Establish a feature representation model based on the brain connection reserve pool and an emotion recognition model based on a graph convolutional neural network, such as Figure 2As shown, a schematic diagram of a feature representation model based on a brain connection reserve pool according to an embodiment of the present invention is shown, and the specific steps are as follows:

[0049] Step S1-1: The feature representation model based on the brain connection reserve pool is a two-layer network structure. The first layer is the input layer, which is used to receive EEG signals. The weight matrix of the input layer is W in The second layer is the brain connection reserve pool, which is used to generate a bidirectional feature representation of the EEG signal. The weight matrix of the brain connection reserve pool is W, and the activation function of the brain connection reserve pool is the hyperbolic tangent function tanh().

[0050] Step S1-2: The EEG signal X input by the input layer is expressed as:

[0051]

[0052] Among them, X(t) represents the EEG signals of all channels at time t, X j , X k represents the EEG signals at all times in channels j and k, respectively, and x j (t) and x k (t) represents the EEG signals at time t in channels j and k respectively, D is the duration of the EEG signal, and N is the number of EEG channels.

[0053] Step S1-3: Randomly initialize the weight matrix W of the input layer in ; The structure of the brain connection reserve pool is determined by calculating the correlation between each channel to initialize the weight matrix W of the brain connection reserve pool = {ω jk |j∈[1,N],k∈[1,N]}. Specifically, the correlation between channel j and channel k is calculated by weighting the phase lag index α WPLI (X j ,X k ) is measured by the following formula:

[0054]

[0055] Where Δθ is X j and X k The phase difference between them, sign() is the sign function, l j and l k Respectively represent X j and X k The amplitude after Hilbert transformation, and j,k∈[1,N]. Each neuron in the brain connection reserve pool corresponds to each EEG channel. Determine a threshold ε, if α WPLI (X j ,X k ) is greater than ε, then it is considered that there is a connection between channels j and k, and the connection weight between channel j and channel k is ωjk =α WPLI (X j ,X k ). Otherwise, there is no connection, and ω jk = 0. Finally, the connection weights between all channels are obtained to form the weight matrix W of the brain connection reserve pool.

[0056] Step S1-4: Figure 3 As shown, a schematic diagram of an emotion recognition model based on a graph convolutional neural network according to an embodiment of the present invention. The emotion recognition model based on a graph convolutional neural network is a three-layer network structure. The first two layers are graph convolution layers. The first graph convolution layer is used to perform graph convolution on the bidirectional feature representation to obtain a bidirectional graph convolution feature representation of the EEG signal, and the second graph convolution layer is used to fuse the bidirectional graph convolution feature representation to obtain a fused feature representation of the EEG signal. The weight matrix of the first graph convolution layer is W (0) , the weight matrix of the second graph convolutional layer is W (1) , the weight matrices of the first two graph convolutional layers are randomly initialized, and their activation functions are ReLU(). The third layer is a fully connected layer, which is used to classify the fused feature representation to obtain the emotion category of the EEG signal. The weight matrix and fixed bias term of the fully connected layer are W c and b are randomly initialized, and the activation function of the fully connected layer is softmax().

[0057] Step S2: Train the brain connection reserve pool and graph convolutional neural network based on the valence lateralization training algorithm, such as Figure 4 As shown, a flow chart of a training algorithm based on valence lateralization according to an embodiment of the present invention is shown, and the specific steps are as follows:

[0058] Step S2-1: The input EEG signal is Calculate the cross entropy loss function Loss, the formula is as follows:

[0059]

[0060] Among them, X i is the EEG signal of the ith sample, is the EEG signal of channel j of the ith sample, m is the number of samples, y i are the emotion classification result and actual emotion category of the i-th sample respectively, and 0 represents depression, 1 represents anger, 2 represents relaxation, and 3 represents happiness. 0 and 1 belong to negative emotions, while 2 and 3 belong to positive emotions.

[0061] Step S2-2: Through back propagation, the loss function Loss is derived and W is calculated c , W (1) and W(0) The gradients of and And update W c , W (1) and W (0) , where W c , W (1) and W (0) are the weight matrices of the fully connected layer, the second graph convolutional layer, and the first graph convolutional layer, respectively. The brain connection reserve pool is divided into the left and right brain connection reserve pools. Specifically, the weight matrix W of the brain connection reserve pool is divided into the weight matrix W of the left brain connection reserve pool l And the right hemisphere connection reserve pool weight matrix W r . Where W = {ω ij |i∈[1,N],j∈[1,N]},W l ={ω ij |i∈[1,N / 2],j∈[1,N / 2]},W r ={ω ij |i∈[N / 2+1,N],j∈[N / 2+1,N]}. Therefore, for the training of the brain connection reserve pool, according to the valence lateralization, when y i is positive emotion, i.e. i ∈{2,3}, then calculate W l Gradient On the contrary, when y i is negative emotion, i.e. i ∈{0,1}, then calculate W r Gradient And update W l and W r , and also W l and W r Update synchronously to W. Until the loss function converges, the training is completed, and finally the trained feature representation model based on the brain connection reserve pool and the emotion recognition model based on the graph convolutional neural network are obtained.

[0062] Step S3: Use the trained feature representation model based on the brain connection reserve pool to perform bidirectional feature representation on the original EEG signal, and use the trained emotion recognition model based on the convolutional neural network to perform four emotion classifications on the bidirectional feature representation of the EEG signal. The specific emotion categories are relaxation, happiness, depression and anger. Among them, relaxation and happiness are positive emotions, depression and anger are negative emotions. The specific steps are as follows:

[0063] Step S3-1: Input the original EEG signal X into the feature representation model based on the brain connection reserve pool to obtain the bidirectional feature representation H(D) of the EEG signal. After inputting into the brain connection reserve pool, we get Will Input brain connection reserve pool The calculation formula is as follows:

[0064]

[0065] in, and represent the positive sequence EEG signal and the reverse sequence EEG signal at time t in channel j, respectively, and and They represent the positive sequence reserve pool state and the reverse sequence reserve pool state at time t in channel j, respectively. D is the duration of the EEG signal, and W in and W are the weight matrices of the input layer and the brain connection reserve pool after training in step S2, respectively. Iterate the above formula according to time t to obtain the reserve pool state of D at the last moment and Bidirectional feature representation of EEG signal as channel j Then, the above formula is iterated according to channel j, and finally the bidirectional feature representation of the EEG signals of all channels is obtained: in, and They respectively represent the positive sequence feature representation and reverse sequence feature representation of the EEG signals of all channels.

[0066] Step S3-2: The bidirectional feature representation H(D) of the EEG signal and W obtained by training in step S2 form a positive sequence feature representation diagram of the EEG signal and reverse feature representation And merge into graph G = {(H(D),W)}.

[0067] Step S3-3: Obtain bidirectional graph convolution feature representation of EEG signal through the first graph convolution layer and The calculation formula is as follows:

[0068]

[0069] Among them, g(.) is the convolution function, L is the Laplace matrix of graph G, U is the standard orthogonal matrix corresponding to L, Λ=diag([λ0,...,λ N-1 ]) is the diagonal matrix of W, λ0,...,λ N-1 is the eigenvalue of W, W (0) is the weight matrix of the first graph convolutional layer after training in step S2.

[0070] Step S3-4: Obtain the fusion feature representation H of the EEG signal through the second graph convolution layer (1) , the calculation formula is as follows:

[0071]

[0072] Among them, W (1) is the weight matrix of the second graph convolutional layer after training in step S2.

[0073] Step S3-5: Representing the fusion feature H of the EEG signal (1) Emotion classification is performed through the fully connected layer to obtain the emotion category of the EEG signal The calculation formula is as follows:

[0074]

[0075] in, 0 represents depression, 1 represents anger, 2 represents relaxation, and 3 represents happiness. 0 and 1 are negative emotions, 2 and 3 are positive emotions. c is the weight matrix of the fully connected layer after training in step S2, and b is the fixed bias term of the fully connected layer.

[0076] The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiment. Any equivalent modifications or changes made by ordinary technicians in this field based on the contents disclosed by the present invention should be included in the protection scope recorded in the claims.

Claims

1. An emotion recognition method based on brain connection reserve pool and graph convolutional neural network, characterized in that: The method comprises the following steps: Step S1: establishing a feature representation model based on a brain connection reserve pool and an emotion recognition model based on a graph convolutional neural network, the former is used to obtain a bidirectional feature representation of the original EEG signal, and the latter is used to classify emotions based on the bidirectional feature representation of the EEG signal; Step S2: dividing the brain connection reserve pool into left and right hemisphere connection reserve pools, and training the left and right hemisphere connection reserve pools and graph convolutional neural networks through a valence lateralization-based training algorithm; Step S3: using the trained brain connection reserve pool-based feature representation model to perform bidirectional feature representation on the original EEG signal, and using the trained graph convolutional neural network-based emotion recognition model to perform four-category emotion classification on the bidirectional feature representation of the EEG signal; In step S3, the trained feature representation model based on the brain connection reserve pool is used to perform bidirectional feature representation on the original EEG signal, and the trained emotion recognition model based on the convolutional neural network is used to perform four emotion classifications on the bidirectional feature representation of the EEG signal. The specific emotion categories are relaxation, happiness, depression and anger; among them, relaxation and happiness are positive emotions, and depression and anger are negative emotions. The specific steps are as follows: Step S3-1: Input the original EEG signal X into the feature representation model based on the brain connection reserve pool to obtain the bidirectional feature representation H(D) of the EEG signal; specifically, After inputting into the brain connection reserve pool, we get Will Input brain connection reserve pool The calculation formula is as follows: in, and represent the positive sequence EEG signal and the reverse sequence EEG signal at time t in channel j, respectively, and and They represent the positive sequence reserve pool state and the reverse sequence reserve pool state at time t in channel j, respectively. D is the duration of the EEG signal, and W in and W are the weight matrices of the input layer and the brain connection reserve pool after training in step S2, respectively. Iterate the above formula according to time t to obtain the reserve pool state of D at the last moment. and Bidirectional feature representation of EEG signal as channel j Then, the above formula is iterated according to channel j, and finally the bidirectional feature representation of the EEG signals of all channels is obtained: in, and Respectively represent the positive sequence feature representation and the reverse sequence feature representation of the EEG signals of all channels; Step S3-2: The bidirectional feature representation H(D) of the EEG signal and W obtained by training in step S2 form a positive sequence feature representation diagram of the EEG signal and reverse feature representation And merge into graph G = {(H(D),W)}; Step S3-3: Obtain bidirectional graph convolution feature representation of EEG signal through the first graph convolution layer and The calculation formula is as follows: Among them, g(.) is the convolution function, L is the Laplace matrix of graph G, U is the standard orthogonal matrix corresponding to L, Λ=diag([λ0,...,λ N-1 ]) is the diagonal matrix of W, λ0,...,λ N-1 is the eigenvalue of W, W (0) is the weight matrix of the first graph convolutional layer after training in step S2; Step S3-4: Obtain the fusion feature representation H of the EEG signal through the second graph convolution layer (1) , the calculation formula is as follows: Among them, W (1) is the weight matrix of the second graph convolutional layer after training in step S2; Step S3-5: Representing the fusion feature H of the EEG signal (1) Emotion classification is performed through the fully connected layer to obtain the emotion category of the EEG signal The calculation formula is as follows: in, 0 represents depression, 1 represents anger, 2 represents relaxation, and 3 represents happiness. 0 and 1 are negative emotions, 2 and 3 are positive emotions. c is the weight matrix of the fully connected layer after training in step S2, and b is the fixed bias term of the fully connected layer.

2. The emotion recognition method based on brain connection reserve pool and graph convolutional neural network according to claim 1, characterized in that: In step S1, a feature representation model based on the brain connection reserve pool and an emotion recognition model based on a graph convolutional neural network are established. The specific steps are as follows: Step S1-1: The feature representation model based on the brain connection reserve pool is a two-layer network structure; the first layer is the input layer, which is used to receive EEG signals, and the weight matrix of the input layer is W in ; The second layer is the brain connection reserve pool, which is used to generate a bidirectional feature representation of the EEG signal. The weight matrix of the brain connection reserve pool is W, and the activation function of the brain connection reserve pool is the hyperbolic tangent function tanh(); Step S1-2: The EEG signal X input by the input layer is expressed as: Among them, X(t) represents the EEG signals of all channels at time t, X j , X k represents the EEG signals at all times in channels j and k, respectively, and x j (t) and x k (t) represents the EEG signals at time t in channels j and k respectively, D is the duration of the EEG signal, and N is the number of EEG channels; Step S1-3: Randomly initialize the weight matrix W of the input layer in ; The structure of the brain connection reserve pool is determined by calculating the correlation between each channel to initialize the weight matrix W of the brain connection reserve pool = {ω jk |j∈[1,N],k∈[1,N]}; Specifically, the correlation between channel j and channel k is based on the weighted phase lag index α WPLI (X j ,X k ) is measured by the following formula: Where Δθ is X j and X k The phase difference between them, sign() is the sign function, l j and l k Respectively represent X j and X k The amplitude after Hilbert transformation, and j, k∈[1,N]; each neuron in the brain connection reserve pool corresponds to each EEG channel; determine a threshold ε, if α WPLI (X j ,X k ) is greater than ε, then it is considered that there is a connection between channels j and k, and the connection weight between channel j and channel k is ω jk =α WPLI (X j ,X k ); otherwise, there is no connection, and ω jk =0; finally, the connection weights between all channels are obtained to form the weight matrix W of the brain connection reserve pool; Step S1-4: The emotion recognition model based on graph convolutional neural network is a three-layer network structure, and the first two layers are graph convolution layers; the first graph convolution layer is used to perform graph convolution on the bidirectional feature representation to obtain the bidirectional graph convolution feature representation of the EEG signal, and the second graph convolution layer is used to fuse the bidirectional graph convolution feature representation to obtain the fused feature representation of the EEG signal; the weight matrix of the first graph convolution layer is W (0) , the weight matrix of the second graph convolutional layer is W (1) , the weight matrices of the first two graph convolutional layers are randomly initialized, and their activation functions are ReLU(); the third layer is a fully connected layer, which is used to classify the fused feature representation to obtain the emotion category of the EEG signal. The weight matrix and fixed bias term of the fully connected layer are W c and b , are randomly initialized, and the activation function of the fully connected layer is softmax().

3. The emotion recognition method based on brain connection reserve pool and graph convolutional neural network according to claim 1, characterized in that: In step S2, the brain connection reserve pool and graph convolutional neural network are trained based on the valence lateralization training algorithm. The specific steps are as follows: Step S2-1: The input EEG signal is Calculate the cross entropy loss function Loss, the formula is as follows: Among them, X i is the EEG signal of the ith sample, is the EEG signal of channel j of the ith sample, m is the number of samples, y i are the emotion classification result and actual emotion category of the i-th sample respectively, and 0 represents depression, 1 represents anger, 2 represents relaxation, and 3 represents happiness. 0 and 1 are negative emotions, while 2 and 3 are positive emotions. Step S2-2: Through back propagation, the loss function Loss is derived and W is calculated c , W (1) and W (0) The gradients are ▽W c , ▽W (1) and ▽W (0) , and update W c , W (1) and W (0) ; Among them, W c , W (1) and W (0) are the weight matrices of the fully connected layer, the second graph convolutional layer, and the first graph convolutional layer respectively; the brain connection reserve pool is divided into the left and right brain connection reserve pools; specifically, the weight matrix W of the brain connection reserve pool is divided into the weight matrix W of the left brain connection reserve pool l And the right hemisphere connection reserve pool weight matrix W r ; in, W={ω jk |j∈[1,N],k∈[1,N]} W l ={ω jk |j∈[1,N / 2],k∈[1,N / 2]} W r ={ω jk |j∈[N / 2+1,N],k∈[N / 2+1,N]} Therefore, for the training of brain connection reserve pool, according to the valence lateralization, when y i is positive emotion, i.e. i ∈{2,3}, then calculate W l The gradient of ▽W l , on the contrary, when y i is negative emotion, i.e. i ∈{0,1}, then calculate W r The gradient of ▽W r , and update W l and W r , and also W l and W r Update synchronously to W; until the loss function converges and the training is completed, finally obtaining the trained feature representation model based on the brain connection reserve pool and the emotion recognition model based on the graph convolutional neural network.

Images