Dynamic pruning electroencephalogram emotion recognition method and system based on graph guidance

Abstract

The invention discloses a dynamic pruning electroencephalogram emotion recognition method and system based on graph guidance, relates to artificial intelligence, and provides the scheme for overcoming the defects in the prior art. Differential entropy features of the electroencephalogram signals are extracted, a deep learning model based on a Transform backbone network is constructed, the differential entropy features serve as input of the deep learning model with channels as units, and channel word boards are formed after block embedding; constructing a self-adaptive dynamic graph neural network structure by taking the differential entropy features as input, and constructing an adjacent matrix representing the connection strength between channels; integrating a pruning module into a specified Transform encoder, screening out channel word boards of which the emotion classification contribution to the electroencephalogram signals is lower than a threshold value, and discarding the channel word boards; and adding the adjacent matrix into calculation of a pruning process. The method has the advantages that the model is promoted to be transited from image processing to EEG processing. The redundancy of input features is reduced, and the calculation speed of the model is improved. And spatial feature fusion of the EEG signals is enhanced.

Description

Technical Field

[0001] The present invention relates to artificial intelligence, and in particular to a graph-guided dynamic pruning EEG emotion recognition method and system. Background Art

[0002] Emotions play adaptive, social or motivational roles in human life because they produce different human behavioral characteristics. Emotions affect decision-making, perception, interpersonal communication, and also affect people's physiological and psychological state. The understanding of emotional signals in daily life environments has become an important aspect that affects people's communication through language and non-verbal behavior. Expressing emotions through facial expressions is considered to be one of the most direct means for humans to communicate feelings and intentions. Emotion is a complex and comprehensive physiological and psychological state that can be characterized by behavioral and physiological signals. Neurological studies have shown that physiological signals are closer to the source of emotions than body signals. Among them, EEG signals are difficult to disguise their response to emotions, and their evaluation of emotions is relatively objective, which helps to obtain the most authentic emotions of humans. Therefore, studying the emotional information contained in EEG signals is of great significance for reliably evaluating human emotions and promoting research in the fields of neurology and medicine.

[0003] In the field of EEG emotion recognition, traditional machine learning methods usually extract features with high redundancy and poor generalization ability, and cannot achieve ideal results in emotion classification tasks. With the continuous development and application of deep learning, EEG-based emotion classification using various neural networks has gradually become a mainstream research direction, and Transformer, as an important breakthrough in the field of deep learning, has been widely used in the field of EEG emotion recognition. Transformer can effectively capture long-range dependencies in input data through the self-attention mechanism, which makes it more advantageous than traditional long short-term memory networks (LSTM) and recurrent neural networks (RNN) when processing EEG signals. EEG signals usually contain complex temporal information and spatial structures. Transformer can automatically extract deep features through its parallel computing ability and global context modeling ability, thereby improving the accuracy of emotion classification. In addition, Transformer has a flexible structure and can perform feature weighting through the self-attention mechanism, thereby effectively processing noise and unimportant information in EEG signals. Therefore, Transformer has significant performance advantages in EEG emotion recognition tasks and has gradually become one of the mainstream models.

[0004] Defects and shortcomings of the existing technology:

[0005] (1) In existing EEG-based emotion recognition tasks, most Transformer models usually take data from all channels as input, but do not fully consider the possible channel redundancy problem, which may cause redundant channel information to affect the classification results.

[0006] (2) As the application of Transformer models in the field of EEG emotion recognition continues to deepen, its structure has gradually become more complex. In addition to the increase in the number of encoder layers, the large number of ci plaques in the model is also an important factor that significantly increases the number of parameters and computational overhead.

[0007] (3) In the current EEG-based emotion classification Transformer model, most methods only use single-dimensional or two-dimensional features as the input of the Transformer model. There are few methods that can organically integrate multi-dimensional and multi-level features, and they cannot fully utilize the multi-dimensional feature characteristics of EGG EEG signals. Summary of the invention

[0008] The purpose of the present invention is to provide a graph-guided dynamic pruning EEG emotion recognition method and system to solve the problems existing in the above-mentioned prior art.

[0009] The graph-guided dynamic pruning EEG emotion recognition system described in the present invention performs emotion recognition according to the following steps:

[0010] S1. Input the original EEG signal, extract the differential entropy features of the EEG signal, and build a deep learning model based on the Transformer backbone network;

[0011] S2, dividing according to the dimension of the channel, taking the differential entropy feature as the input of the deep learning model in units of channels, and forming channel word cards after block embedding;

[0012] S3, differential entropy features are used as input to build an adaptive dynamic graph neural network structure, and the relationship between channels is measured by absolute value distance, and an adjacency matrix representing the connection strength between channels is constructed;

[0013] S4, integrate the pruning module into the specified Transformer encoder, filter out the channel word cards whose contribution to the EEG signal emotion classification is lower than the threshold and discard them;

[0014] S5. Add the adjacency matrix to the calculation of the pruning process, and aggregate the information between different channels to improve the accuracy of pruning.

[0015] The graph-guided dynamic pruning EEG emotion recognition method described in the present invention utilizes the dynamic pruning EEG emotion recognition system to perform cross-domain EEG emotion recognition.

[0016] The graph-guided dynamic pruning EEG emotion recognition method and system described in the present invention have the following advantages:

[0017] (1) The Transformer model, which is widely used in image processing, is used as the backbone network and combined with EEG data features. The EEG channel data is used as the feature input of the model. Through the block embedding operation, each channel becomes the input word card of the Transformer model, thus facilitating the transition of the model from image processing to EEG processing.

[0018] (2) Use the pruning module to evaluate the contribution of different channel inputs to sentiment classification, eliminate channels that contribute less to the classification results, reduce the redundancy of input features, and improve the calculation speed of the model.

[0019] (3) Using a dynamic graph structure based on input features and leveraging existing prior knowledge about the connections between different brain regions, we dynamically construct an adjacency matrix representing the relationship strength between each channel of the EEG emotion signal and incorporate it into the calculation of the pruning module to enhance the spatial feature fusion of the EEG signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the framework structure of the deep learning model described in the present invention.

[0021] Figure 2 It is a flow chart of the dynamic pruning EEG emotion recognition method described in the present invention. DETAILED DESCRIPTION

[0022] Most of the existing EEG signal emotion recognition methods do not consider the possible redundancy of channels, do not make full use of multi-dimensional and multi-level feature fusion, and lack lightweight models. Therefore, the present invention adopts EEG signal channel pruning, and integrates the spatial domain features of EEG signals during the pruning process. On the basis of combining multi-dimensional features and removing redundant channels, the model calculation amount is reduced. Figure 1 to Figure 2 As shown, the graph-guided dynamic pruning EEG emotion recognition system described in the present invention performs emotion recognition in the following steps:

[0023] S1. Input the original EEG signal, extract the differential entropy features, and cut the differential entropy features in the time domain dimension using window cutting, and then get several data formats of N×T w ×C EEG signal, where N is the number of channels, T w is the time domain window size, C is the number of frequency bands, and the training set and test set are divided according to a certain ratio to complete the data preprocessing, and the processed data is used as the input of the Transformer backbone network.

[0024] S2. After the differential entropy features in units of channels are used as the input of the deep learning model, there is no need to perform block segmentation in the block embedding part. Instead, a 1×1 convolutional layer is used to traverse the channel data directly, and the time domain and dimensional features are flattened. After feature mapping by the linear layer, the channel word brand is established through the GELU activation function.

[0025] S3, in the adaptive dynamic graph neural network module, the differential entropy feature extracted in step S1 is used as input to construct a non-negative function A mn =g(x m , x n )(m,n∈{1,2,...,N]) is used to represent the connection strength between the mth channel and the nth channel and the absolute value distance is used to measure the relationship between the channels: This A mn is the adjacency matrix to be constructed, and the values ​​of this matrix are learnable based on input changes. The learning process of the adjacency matrix is ​​constrained by the following loss function:

[0026] When x m and x n The distance between ||x m -x n ||2 When the difference is greater, A mn The smaller the connection strength between the corresponding channels.

[0027] S4. In the channel pruning module, the pruning process occurs in a specific Transformer encoder structure, subtracting redundant channel word information and reducing the amount of computation of the deep learning model. First, a binary mask D∈[0,1] is designed. N To represent the input tune x∈R N×C Whether to keep or discard, the initial mask of all ci tunes is set to 1. The feature mapping of ci tunes is performed using a multi-layer perceptron to obtain local features: Z local =MLP(x)∈R N×C′ , where C′=C / 2. The global feature is obtained by averaging the local features and aggregating the existing ci information: Z global =Agg(MLP(x),D)∈R N×C′ , The local feature represents the information of a certain channel, and the global feature represents the information of the entire input EEG signal. The two are combined to obtain the local-global embedding feature: And use another multi-layer perceptron to predict the probability of discarding or retaining channel information: I = Softmax(MLP(z))∈R N×2 , where I i，0 Indicates the probability of discarding the i-th channel feature, I i，1Represents the probability of retaining the i-th channel feature.

[0028] S5. In order to make channel pruning more accurate, the adjacency matrix A obtained in the adaptive dynamic graph neural network module is mn Added to the calculation of global features, the expression of Agg is modified as follows:

[0029] A mn Similar to the weight matrix, it represents the connection relationship between channels. Its format is N×N, and each row is normalized. global , u represents MLP(x), which has the format of N×C, so when u is surrounded by the adjacency matrix A mn When multiplying from the left, the format of u does not change. And since A mn Information can be aggregated according to weight distribution, and the accuracy of pruning is improved.

[0030] The graph-guided dynamic pruning EEG emotion recognition method described in the present invention utilizes the dynamic pruning EEG emotion recognition system to perform cross-domain EEG emotion recognition.

[0031] For those skilled in the art, various other corresponding changes and deformations can be made according to the technical solutions and concepts described above, and all of these changes and deformations should fall within the protection scope of the claims of the present invention.

Claims

1. A graph-guided dynamic pruning EEG emotion recognition system, characterized by: Follow these steps to perform emotion recognition: S1. Input the original EEG signal, extract the differential entropy features of the EEG signal, and build a deep learning model based on the Transformer backbone network; S2, dividing according to the dimension of the channel, taking the differential entropy feature as the input of the deep learning model in units of channels, and forming channel word cards after block embedding; S3, differential entropy features are used as input to build an adaptive dynamic graph neural network structure, and the relationship between channels is measured by absolute value distance, and an adjacency matrix representing the connection strength between channels is constructed; S4, integrate the pruning module into the specified Transformer encoder, filter out the channel word cards whose contribution to the EEG signal emotion classification is lower than the threshold and discard them; S5. Add the adjacency matrix to the calculation of the pruning process, and aggregate the information between different channels to improve the accuracy of pruning.

2. According to the graph-guided dynamic pruning EEG emotion recognition system of claim 1, it is characterized in that: The step S1 is specifically as follows: Input the original EEG signal, extract the differential entropy feature, and cut the differential entropy feature in the time domain dimension using the window cutting method, and then get several data formats of N×T w ×C EEG signal, where N is the number of channels, T w is the time domain window size, C is the number of frequency bands, and the data is divided into training set and test set to complete the data preprocessing; the processed data is used as the input of the Transformer backbone network.

3. According to the graph-guided dynamic pruning EEG emotion recognition system of claim 2, it is characterized in that: The step S2 is specifically as follows: After the differential entropy features in channels are used as the input of the deep learning model, the channel data is directly traversed using a 1×1 convolutional layer without performing block segmentation in the block embedding part, and the time domain and dimensional features are flattened. After feature mapping by the linear layer, the channel word brand is established through the GELU activation function.

4. The graph-guided dynamic pruning EEG emotion recognition system according to claim 3 is characterized in that: The step S3 is specifically as follows: In the adaptive dynamic graph neural network module, the differential entropy features extracted in step S1 are used as input to construct a non-negative function A mn =g(x m , x n )(m,n∈{1,2,...,N]) is used to represent the connection strength between the mth channel and the nth channel and the absolute value distance is used to measure the relationship between the channels: Among them A mn The adjacency matrix to be constructed, the values ​​of the matrix are learned and modified based on the input changes; The learning process of the adjacency matrix is ​​constrained by the following loss function: Indicates that when x m and x n The distance between ||x m -x n ||2 When the difference is greater, A mn The smaller the connection strength between the corresponding channels.

5. The graph-guided dynamic pruning EEG emotion recognition system according to claim 4 is characterized in that: The step S4 is specifically as follows: First, design a binary mask D∈[0, 1] N To represent the input ci tune x∈R N×C Whether to keep or discard, the initial mask of all ci tunes is set to 1; the feature mapping of ci tunes is performed using a multi-layer perceptron to obtain local features: Z local =MLP(x)∈R N×C′ , where C′=C / 2; the global feature is obtained by averaging the local features and aggregating the existing ci information: Z global =Agg(MLP(x),D)∈R N×C′ , The local feature represents the information corresponding to a single channel, and the global feature represents the information of the entire input EEG signal. The two are combined to obtain the local-global embedding feature: And use another multi-layer perceptron to predict the probability of discarding or retaining channel information: I = Softmax(MLP(z))∈R N×2 , where l i，0 Indicates the probability of discarding the i-th channel feature, I i，1 Represents the probability of retaining the i-th channel feature.

6. The graph-guided dynamic pruning EEG emotion recognition system according to claim 5 is characterized in that: The step S5 is specifically as follows: The adjacency matrix A obtained in the adaptive dynamic graph neural network module mn Added to the calculation of global features, the expression of Agg is modified as follows: Represents the connection relationship between channels. Its format is N×N, and each row is normalized.

7. A graph-guided dynamic pruning EEG emotion recognition method, characterized in that: Cross-domain EEG emotion recognition is performed using the dynamic pruning EEG emotion recognition system as described in any one of claims 1 to 6.

Images