Emotion recognition SimAM-CRNN method based on electroencephalogram signals

Abstract

The invention discloses an emotion recognition SimAM-CRNN method based on electroencephalogram signals. The method comprises the following steps that a head space is constructed into an h * w matrix according to electrode arrangement; an original EEG signal is divided into N segments, and 3D features are obtained through decomposition, extraction, arrangement and superposition; inputting the processed electroencephalogram signals into a CNN (Convolutional Neural Network) based on a SimAM attention mechanism, passing through a maximum pooling layer, flattening and then fully connecting, and carrying out preliminary identification; after feature extraction of SimAM-CNN, inputting the feature to an LSTM unit for extraction of time dimension information; and a softmax layer is used as a classifier, and the extracted feature set is used as input to carry out emotion recognition. According to the method, the SimAM attention mechanism is added on the basis of the 4D-CRNN, and the SimAM can deduce the 3D attention weight for the feature map under the condition that network parameters are not increased, so that the emotion related features are better extracted. In addition, a Batch Normalization layer is added, network convergence is accelerated, the accuracy is improved, and meanwhile, data overfitting is prevented.

Description

Technical Field

[0001] The present invention relates to the technical field of electroencephalogram (EEG) signal processing, and in particular to a SimAM-CRNN method for emotion recognition based on EEG signals. Background Art

[0002] Emotion recognition is the discrimination of human emotional states, including happiness, anger, sadness, etc., based on physiological and behavioral characteristics. Emotion recognition methods can be roughly divided into shallow models and deep learning models. In these models, feature extraction and processing are important steps. In shallow models, the features of raw EEG signals are extracted and manually selected. For example, Duan et al. used the differential entropy (DE) of each EEG channel as the discriminant feature of emotion, and used shallow classifiers such as support vector machine (SVM) and K-nearest neighbor (KNN) to achieve emotion recognition. Mohammadi et al. used wavelet transform to extract time-frequency domain features, and used SVM and KNN to detect emotional states. Li et al. used 9 time-frequency domain features such as Hjorth parameter (HP) and 9 nonlinear dynamic system features such as spectral entropy (SE) to explore the importance of different EEG features in emotion recognition. Since the features in shallow models are manually designed based on neuroscience knowledge, they have a fairly high degree of discrimination in emotion recognition. However, since the classifiers are usually simple, it is difficult to further analyze and fully utilize the manually designed features in shallow models.

[0003] Deep learning methods are currently widely used in emotion recognition and have shown superior capabilities over traditional machine learning methods. Huang et al. proposed an end-to-end bi-hemispheric difference convolutional neural network (BiDCNN) to convert multi-channel EEG signals into two-dimensional frames, retain the topological structure of the channels, and learn each pre-processed data through a two-dimensional convolutional layer. Lawhern et al. designed EEGnet, combining deep convolution and separable convolution techniques. Wang et al. proposed EmotioNet, which integrates batch normalization and dense prediction mechanisms to convert two-dimensional frames into three-dimensional cube inputs. Experimental results show that its performance is better than EEGnet. Yang et al. proposed a hierarchical network for classifying differential entropy (DE) features extracted from different frequency bands. Shen et al. converted the DE features of different channels into a four-dimensional structure, integrated spectral, spatial and temporal information, and used a four-dimensional convolutional recurrent neural network (4DCRNN) to recognize emotions. Song et al. introduced an attention mechanism based on 4DCRNN to strengthen the internal relationship between frequency bands and improve recognition performance. In the process of emotion recognition, different brain regions and frequency bands contribute differently to emotions, and the temporal dependency of the four-dimensional representation needs to be considered. Summary of the invention

[0004] The design goal of the present invention is to provide an effective emotion recognition method, the purpose of which is to extract key physiological features from the original EEG signals to achieve accurate quantification of brain neural activity and improve the accuracy and reliability of emotion recognition. To solve the above problems, the present invention provides a SimAM-CRNN method for emotion recognition based on EEG signals.

[0005] To achieve the purpose of the present invention, a method for emotion recognition based on EEG signals, SimAM-CRN, is provided, comprising the following steps:

[0006] S100, constructing the head space into an h×w matrix according to the electrode arrangement;

[0007] S200, dividing the original EEG signal into N segments, and obtaining 3D features through decomposition, extraction, arrangement, and superposition;

[0008] S300, the processed EEG signal is input into the CNN based on the SimAM attention mechanism, and after passing through the maximum pooling layer, it is flattened and fully connected for preliminary recognition;

[0009] S400, after feature extraction by SimAM-CNN, is input into the LSTM unit to extract time dimension information;

[0010] S500, through the softmax layer as a classifier, the extracted feature set is used as input for emotion recognition.

[0011] In some specific embodiments, the specific method of step S100 is:

[0012] Choose to divide the overhead space into an 8×9 grid, with unoccupied positions represented by 0.

[0013] In some specific embodiments, the specific method of step S200 is:

[0014] The original EEG signal is divided into N segments, each of which is Ts long and has no overlap. Each segment can be divided into 2T continuous parts. Then, each part is decomposed into 4 frequency bands (δ[4-8Hz], α[8-14Hz], β[14-31Hz], γ[31-51Hz]). Next, DE features are extracted from each frequency band and arranged into a compact 2D graph. The differential entropy (DE) feature is used to measure the complexity of the EEG signal and is defined as:

[0015] h(Z)=∫ z f(Z)log(r(z))dz (1)

[0016] Where Z is a random variable and f(z) is the probability density function of Z;

[0017] The differential entropy (DE) feature of the Gaussian distribution is calculated as follows:

[0018]

[0019] Finally, the 2D images of the four frequency bands are superimposed to obtain the 3D features. Through the above operation, the original EEG signal is converted into N segments, represented as Where N is the total number of samples, h and w are the height and width of the 2D graph, respectively, d represents the number of frequency bands, and 2T represents twice the segment length.

[0020] In some specific embodiments, the specific method of step S300 is:

[0021] The basic structure of SimAM-CNN includes four convolutional layers, four batch normalization layers, four attention modules, a maximum pooling layer and a fully connected layer. The convolution kernel sizes are 5×5 for the first layer, 4×4 for the second layer, 4×4 for the third layer and 1×1 for the fourth layer. The maximum pooling layer uses a stride of 2×2. By applying binary labels and regular terms, the minimum energy can be obtained by the following formula:

[0022]

[0023] in, represents the mean value of all neurons, represents the variance of these neurons, t is the target neuron, y i are other neurons in the input feature channel, and β is the regularization coefficient. The number of neurons in the channel can be calculated by M = H × W. In general, the difference between a neuron and its surrounding neurons is related to energy. The importance of each neuron can be expressed by 1 / E * Measure; optimize features through a scaling operator.

[0024] In some of the specific embodiments, the overall optimization process of the SimAM module is as follows:

[0025]

[0026] Among them, E groups all e in the channel and spatial dimensions; in order to limit the size of the values ​​in E, the Sigmoid function is used;

[0027] The formula for the batch normalization BN layer is as follows:

[0028]

[0029] Among them, μ and σ represent the mean and variance of each feature map, represents the normalized data, x iIt can be seen as the output obtained by translation and scaling using β and α; β and α are translation and scaling factors respectively, and they are both learnable parameters, which means that they can change with each training batch; after the training process is completed, the total of β and γ is the overall dataset size divided by the batch size.

[0030] In some specific embodiments, the specific method of step S400 is:

[0031] The LSTM unit receives three inputs: the input Qi at the current time i, the output e at the previous time i-1 i-1 and the hidden state x at the previous time i-1 i-1 ; Then, the LSTM unit produces two outputs: the output e at the current time i i and the hidden state x of the i-th time feature i ; The LSTM unit contains three gates: input gate, forget gate and output gate, which control the data flow through sigmoid and tanh activation functions respectively; Although the training samples and test samples are not synchronized and continuous in time, the encoded samples still contain spatial information after the spatial feature extraction module; The input gate weight learns spatial information, while the forget gate weight learns temporal information, and the input gate and forget gate compete to input new information or retain the current temporal information.

[0032] In some specific embodiments, the specific method of step S400 is:

[0033] The LSTM layer output is calculated as follows:

[0034] f t =μ(w f ·[x (t-1) ,y t ]+b f ) (11)

[0035] i t =μ(w i ·[x (t-1) ,y t ]+b i ) (12)

[0036] o t =μ(w o ·[x (t-1) ,y t ]+b o ) (13)

[0037]

[0038] x t =o t ·tanh(et ) (16)

[0039] Among them, y t is the data at this moment, c (t-1) The state of the unit at the last moment retains the memory of history. is the input information at this moment, representing the information of the current data, e t is the state of the unit at this moment, x t is the output of the unit at this moment. When processing each moment, y t With x (t-1) The two matrices are directly concatenated together [x (t-1) ,y t ] as input, plus appropriate weights w and bias b.

[0040] In some specific embodiments, the specific method of step S500 is:

[0041] The formula for softmax is:

[0042] P = softmax(WL+b) (17)

[0043] Where L = {L1, L2, ..., L n}, L i (i=1, 2, ..., n) represents the predicted probability of the ith EEG sample, W and b are the weight and bias of the softmax function, respectively.

[0044] Beneficial effects of the invention: The invention discloses a SimAM-CRN method for emotion recognition based on EEG signals, comprising the following steps: constructing the head space into an h×w matrix according to the electrode arrangement; dividing the original EEG signal into N segments, decomposing, extracting, arranging, and superimposing to obtain 3D features; inputting the processed EEG signal into a CNN based on the SimAM attention mechanism, passing through the maximum pooling layer, flattening and then fully connected for preliminary recognition; after feature extraction by SimAM-CNN, inputting into the LSTM unit to extract time dimension information; using the softmax layer as a classifier, taking the extracted feature set as input for emotion recognition. The invention adds the SimAM attention mechanism on the basis of 4D-CRNN, and SimAM can infer 3D attention weights for feature maps without increasing network parameters, and better extract emotion-related features. In addition, the Batch Normalization layer is added to accelerate network convergence and improve accuracy, while preventing data overfitting. For the ECA-CRNN method, the performance of the SimAM module is better than that of the ECA module, because SimAM is more flexible and lightweight than the ECA module. The present invention also replaces the activation function and uses ELU instead of ReLU. Compared with ReLU, which only allows positive output, ELU allows negative output, which enables the network to better fit the data in some cases. The present invention performed a five-fold cross validation on the DEAP dataset and compared other emotion recognition methods. When the number of training epochs is 100, our method outperforms 4D-CRNN (94.22% valence accuracy, 95.58% arousal accuracy) in terms of 95.29% valence accuracy and 95.62% arousal accuracy, and also outperforms ECA-CRNN (93.58% valence accuracy, 94.06% arousal accuracy). BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flow chart of a SimAM-CRNN method for emotion recognition based on EEG signals of the present invention;

[0046] Figure 2 is a 2D sparse graph filled with 32 channels of the present invention;

[0047] Figure 3 It is the SimAM-CRNN emotion recognition process of the present invention;

[0048] Figure 4 It is a schematic diagram of recording the best recognition accuracy of SimAM-CRNN of the present invention;

[0049] Figure 5 It is a schematic diagram of recording the worst recognition accuracy of SimAM-CRNN of the present invention. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0051] Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar symbols throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0052] The present invention develops a SimAM-CRNN four-dimensional attention neural network for emotion recognition, which aims to improve the accuracy of emotion recognition, so as to capture the important information in EEG signals more comprehensively and accurately, and provide a more reliable processing solution for neuroscience research and other brain science research fields.

[0053] Reference Figure 1 , Figure 2 , Figure 3 , Figure 4 and Figure 5 , an emotion recognition SimAM-CRN method based on EEG signals, comprising the following steps:

[0054] S100, constructing the head top space into an h×w matrix according to the electrode arrangement.

[0055] Specifically, this method divides the space above the head into an 8×9 grid, which is similar to a compact graph rather than a sparse graph, and unoccupied positions are represented by 0. This structure helps us to effectively organize and analyze electrode data in space, thereby extracting spatial features more accurately.

[0056] S200, dividing the original EEG signal into N segments, and obtaining 3D features through decomposition, extraction, arrangement, and superposition.

[0057] S300, the processed EEG signal is input into the CNN based on the SimAM attention mechanism, and after passing through the maximum pooling layer and flattening, it is fully connected for preliminary recognition.

[0058] S400, after feature extraction by SimAM-CNN, is input into the LSTM unit to extract time dimension information.

[0059] S500, using the softmax layer as a classifier, taking the extracted feature set as input for emotion recognition. The present invention uses the softmax layer as a classifier, taking the extracted feature set L = {L1, L2, ..., L n} as input for emotion recognition.

[0060] In some specific embodiments, the specific method of step S100 is: select to divide the head space into 8×9 grids, and the unoccupied positions are represented by 0.

[0061] In some specific embodiments, the specific method of step S200 is:

[0062] The original EEG signal is divided into N segments, each of which is Ts long and has no overlap. Each segment can be divided into 2T continuous parts. Then, each part is decomposed into 4 frequency bands (δ[4-8Hz], α[8-14Hz], β[14-31Hz], γ[31-51Hz]). Next, DE features are extracted from each frequency band and arranged into a compact 2D graph. The differential entropy (DE) feature is used to measure the complexity of the EEG signal and is defined as:

[0063] h(z)=∫ z f(z)log(f(z))dz (1)

[0064] Where Z is a random variable and f(z) is the probability density function of Z;

[0065] The differential entropy (DE) feature of the Gaussian distribution is calculated as follows:

[0066]

[0067] Finally, the 2D images of the four frequency bands are superimposed to obtain the 3D features. Through the above operation, the original EEG signal is converted into N segments, represented as Where N is the total number of samples, h and w are the height and width of the 2D graph, respectively, d represents the number of frequency bands, and 2T represents twice the segment length.

[0068] In some specific embodiments, the specific method of step S300 is:

[0069] The basic structure of SimAM-CNN includes four convolutional layers, four batch normalization layers, four attention modules, a maximum pooling layer and a fully connected layer; the convolution kernel sizes are 5×5 for the first layer, 4×4 for the second layer, 4×4 for the third layer and 1×1 for the fourth layer, and the maximum pooling layer uses a stride of 2×2; the SimAM attention module infers three-dimensional attention weights for feature maps without increasing network parameters. This module is based on well-known neuroscience theories and determines the importance of each neuron by optimizing the energy function. The SimAM module can identify key neurons and construct an energy function. By applying binary labels and regular terms, the minimum energy can be obtained by the following formula:

[0070]

[0071] in, represents the mean value of all neurons, represents the variance of these neurons, t is the target neuron, y i are other neurons in the input feature channel, and β is the regularization coefficient. The number of neurons in the channel can be calculated by M = H × W. In general, the difference between a neuron and its surrounding neurons is related to energy. The importance of each neuron can be expressed by 1 / E * Measure; optimize features through a scaling operator.

[0072] In some of the specific embodiments, the overall optimization process of the SimAM module is as follows:

[0073]

[0074] Among them, E groups all e in the channel and spatial dimensions; in order to limit the size of the values ​​in E, the Sigmoid function is used;

[0075] The batch normalization (BN) method relies on two consecutive linear transformations, hoping that the transformed values ​​meet certain characteristics. It not only speeds up the convergence of the model, but also alleviates the problem of scattered feature distribution to a certain extent, making deep neural network (DNN) training faster and more stable.

[0076] The formula for the batch normalization BN layer is as follows:

[0077]

[0078]

[0079] Among them, μ and σ represent the mean and variance of each feature map, represents the normalized data, x i It can be seen as the output obtained by translation and scaling using β and α; β and α are translation and scaling factors respectively, and they are both learnable parameters, which means that they can change with each training batch; after the training process is completed, the total of β and γ is the overall dataset size divided by the batch size.

[0080] In some specific embodiments, the specific method of step S400 is:

[0081] The LSTM unit receives three inputs: the input Qi at the current time i, the output e at the previous time i-1 i-1 and the hidden state x at the previous time i-1 i-1 ; Then, the LSTM unit produces two outputs: the output e at the current time ii and the hidden state x of the i-th time feature i ; The LSTM unit contains three gates: input gate, forget gate and output gate, which control the data flow through sigmoid and tanh activation functions respectively; Although the training samples and test samples are not synchronized and continuous in time, the encoded samples still contain spatial information after the spatial feature extraction module; The input gate weight learns spatial information, while the forget gate weight learns temporal information, and the input gate and forget gate compete to input new information or retain the current temporal information.

[0082] In some specific embodiments, the specific method of step S400 is:

[0083] The LSTM layer output is calculated as follows:

[0084] f t =μ(w f ·[x (t-1) ,y t ]+b f ) (11)

[0085] i t =μ(w i ·[x (t-1) ,y t ]+b i ) (12)

[0086] o t =μ(w o ·[x (t-1) ,y t ]+b o ) (13)

[0087]

[0088] x t =o t ·tanh(e t ) (16)

[0089] Among them, y t is the data at this moment, c (t-1) The state of the unit at the last moment retains the memory of history. is the input information at this moment, representing the information of the current data, e t is the state of the unit at this moment, x t is the output of the unit at this moment. When processing each moment, y t With x (t-1) The two matrices are directly concatenated together [x (t-1) ,y t ] as input, plus appropriate weights w and bias b.

[0090] In some specific embodiments, the specific method of step S500 is:

[0091] The formula for softmax is:

[0092] P = softmax(WL+b) (17)

[0093] Where L = {L1, L2, ..., L n}, L i (i=1, 2, ..., n) represents the predicted probability of the ith EEG sample, W and b are the weight and bias of the softmax function, respectively.

[0094] Example: Reference Figure 2 , the EEG signal channel is divided into an 8×9 grid, which is similar to a compact graph rather than a sparse graph, and unoccupied positions are represented by 0. This structure helps the present invention to effectively organize and analyze electrode data in space, thereby extracting spatial features more accurately.

[0095] First, the original EEG signal is converted into a four-dimensional space-spectrum-time representation containing multiple time segments, denoted as Where n = 1, 2, ..., N. Then, the spectrum, time and space information of the EEG signal are extracted by combining CNN and LSTM with SimAM attention mechanism. The SimAM attention module infers the importance of each neuron by optimizing the energy function, avoiding excessive adjustments to the network structure, and has higher flexibility, modularity and lightness. Finally, softmax is used for classification. The SimAM-CRNN emotion recognition process is as follows Figure 3 shown.

[0096] When the pre-processed data is input into the network, the present invention finds that different segment lengths in the experiment will produce different results. Table 1 shows the influence of different segment lengths on the recognition accuracy when the epoch is set to 100.

[0097] Table 1 SimAM-CRNN time segment results

[0098]

[0099] Figure 4 and Figure 5 The recognition accuracy of the present invention on the DEAP dataset is demonstrated. In terms of valence, when the time length is 3 seconds, the emotion recognition rate is the highest, reaching 95.29%, with a standard deviation of 1.43%. In terms of arousal, when the time length is 2 seconds, the recognition rate is the best, reaching 95.62%, with a standard deviation of 1.01%;

[0100] Beneficial effects of the invention: The invention discloses a SimAM-CRN method for emotion recognition based on EEG signals, comprising the following steps: constructing the head space into an h×w matrix according to the electrode arrangement; dividing the original EEG signal into N segments, decomposing, extracting, arranging, and superimposing to obtain 3D features; inputting the processed EEG signal into a CNN based on the SimAM attention mechanism, passing through the maximum pooling layer, flattening and then fully connected for preliminary recognition; after feature extraction by SimAM-CNN, inputting into the LSTM unit to extract time dimension information; using the softmax layer as a classifier, taking the extracted feature set as input for emotion recognition. The invention adds the SimAM attention mechanism on the basis of 4D-CRNN, and SimAM can infer 3D attention weights for feature maps without increasing network parameters, and better extract emotion-related features. In addition, the Batch Normalization layer is added to accelerate network convergence and improve accuracy, while preventing data overfitting. For the ECA-CRNN method, the performance of the SimAM module is better than that of the ECA module, because SimAM is more flexible and lightweight than the ECA module. The present invention also replaces the activation function and uses ELU instead of ReLU. Compared with ReLU, which only allows positive output, ELU allows negative output, which enables the network to better fit the data in some cases. The present invention performed a five-fold cross validation on the DEAP dataset and compared other emotion recognition methods. When the number of training epochs is 100, our method outperforms 4D-CRNN (94.22% valence accuracy, 95.58% arousal accuracy) in terms of 95.29% valence accuracy and 95.62% arousal accuracy, and also outperforms ECA-CRNN (93.58% valence accuracy, 94.06% arousal accuracy).

[0101] The above are only preferred specific implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solutions and the inventive concepts of the present invention within the scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A SimAM-CRNN method for emotion recognition based on EEG signals, characterized in that it comprises the following steps: S100, constructing the head space into an h×w matrix according to the electrode arrangement; S200, dividing the original EEG signal into N segments, and obtaining 3D features through decomposition, extraction, arrangement, and superposition; S300, the processed EEG signal is input into the CNN based on the SimAM attention mechanism, and after passing through the maximum pooling layer, it is flattened and fully connected for preliminary recognition; S400, after feature extraction by SimAM-CNN, is input into the LSTM unit to extract time dimension information; S500, through the softmax layer as a classifier, the extracted feature set is used as input for emotion recognition.

2. The SimAM-CRNN method for emotion recognition based on EEG signals according to claim 1 is characterized in that the specific method of step S100 is: Choose to divide the overhead space into an 8×9 grid, with unoccupied positions represented by 0.

3. The SimAM-CRNN method for emotion recognition based on EEG signals according to claim 1 is characterized in that the specific method of step S200 is: The original EEG signal is divided into N segments, each of which is Ts long and has no overlap. Each segment can be divided into 2T continuous parts. Then, each part is decomposed into 4 frequency bands (δ[4-8Hz], α[8-14Hz], β[14-31Hz], γ[31-51Hz]). Next, DE features are extracted from each frequency band and arranged into a compact 2D graph. The differential entropy (DE) feature is used to measure the complexity of the EEG signal and is defined as: h(Z)=∫ z f(Z)log(f(z))dz (1) in, Z is a random variable, f(z) is the probability density function of Z; The differential entropy (DE) feature of the Gaussian distribution is calculated as follows: Finally, the 2D images of the four frequency bands are superimposed to obtain the 3D features. Through the above operation, the original EEG signal is converted into N segments, represented as n=1, 2, ..., N, where N is the total number of samples, h and w are the height and width of the 2D graph, respectively, d represents the number of frequency bands, and 2T represents twice the segment length.

4. The SimAM-CRNN method for emotion recognition based on EEG signals according to claim 1 is characterized in that the specific method of step S300 is: The basic structure of SimAM-CNN includes four convolutional layers, four batch normalization layers, four attention modules, a maximum pooling layer and a fully connected layer. The convolution kernel sizes are 5×5 for the first layer, 4×4 for the second layer, 4×4 for the third layer and 1×1 for the fourth layer. The maximum pooling layer uses a stride of 2×2. By applying binary labels and regular terms, the minimum energy can be obtained by the following formula: in, represents the mean value of all neurons, represents the variance of these neurons, t is the target neuron, y i are other neurons in the input feature channel, and β is the regularization coefficient. The number of neurons in the channel can be calculated by M = H × W. In general, the difference between a neuron and its surrounding neurons is related to energy. The importance of each neuron can be expressed by 1 / E * Measure; optimize features through a scaling operator.

5. The SimAM-CRNN method for emotion recognition based on EEG signals according to claim 4 is characterized in that: The overall optimization process of the SimAM module is as follows: in, E groups all e in the channel and spatial dimensions; in order to limit the size of the values ​​in E, the Sigmoid function is used; The formula for the batch normalization BN layer is as follows: Among them, μ and σ represent the mean and variance of each feature map, represents the normalized data, x i It can be seen as the output obtained by translation and scaling using β and α; β and α are translation and scaling factors respectively, and they are both learnable parameters, which means that they can change with each training batch; after the training process is completed, the total of β and γ is the overall dataset size divided by the batch size.

6. The SimAM-CRNN method for emotion recognition based on EEG signals according to claim 1 is characterized in that the specific method of step S400 is: The LSTM unit receives three inputs: the input Qi at the current time i, the output e at the previous time i-1 i-1 and the hidden state x at the previous time i-1 i-1 ; Then, the LSTM unit produces two outputs: the output e at the current time i i and the hidden state x of the i-th time feature i ; The LSTM unit contains three gates: input gate, forget gate and output gate, which control the data flow through sigmoid and tanh activation functions respectively; Although the training samples and test samples are not synchronized and continuous in time, the encoded samples still contain spatial information after the spatial feature extraction module; The input gate weight learns spatial information, while the forget gate weight learns temporal information, and the input gate and forget gate compete to input new information or retain the current temporal information.

7. The SimAM-CRNN method for emotion recognition based on EEG signals according to claim 1 is characterized in that the specific method of step S400 is: The LSTM layer output is calculated as follows: f t =μ(w f ·[x (t-1) ,y t ]+b f ) (11) i t =μ(w i ·[x (t-1) ,y t ]+b i ) (12) the t =μ(w o ·[x (t-1) ,y t ]+b o ) (13) x t =o t ·tanh(e t ) (16) in, y t is the data at this moment, c (t-1) The state of the unit at the last moment retains the memory of history. is the input information at this moment, representing the information of the current data, e t is the state of the unit at this moment, x t The output of the unit at this moment, y is processed at each moment t With x (t-1) The two matrices are directly concatenated together [x (t-1) ,y t ] as input, plus appropriate weights w and bias b.

8. The SimAM-CRNN method for emotion recognition based on EEG signals according to claim 1 is characterized in that the specific method of step S500 is: The softmax formula is: P = softmax(WL+b) (17) Where L = {L1, L2, ..., L n }, L i (i=1,2,…,n) represents the predicted probability of the ith EEG sample, W and b are the weight and bias of the softmax function, respectively.

Images