Two-way electroencephalogram emotion recognition method and system combining general path learning and medical priori

Abstract

The invention relates to a deep learning and emotion recognition technology, in particular to a two-way electroencephalogram emotion recognition method and system combining general path learning and medical priori. The method comprises the following steps: preprocessing an electroencephalogram signal, and extracting a difference entropy feature; constructing an adjacent matrix AC based on a general path and an adjacent matrix AR of a brain region divided based on medical priori; taking the difference entropy features as node features, respectively taking the adjacent matrixes AC and AR as edges, and constructing a path graph structure and a prior graph structure; performing graph convolution operation on the path graph structure and the prior graph structure to update node features according to the adjacency matrixes AC and AR; fusing the two paths of updated node features to obtain high-dimensional fusion feature matrix representation; and the electroencephalogram emotion is identified and classified through a multi-layer perceptron. According to the method, general emotion recognition features and biological prior of brain region division are effectively combined, and more comprehensive and accurate feature representation is provided for emotion recognition.

Description

Technical Field

[0001] The present invention relates to deep learning and emotion recognition technologies, and particularly to a dual-channel EEG emotion recognition method and system combining general path learning and medical prior knowledge. Background Art

[0002] Emotions not only profoundly affect people's cognitive and behavioral activities but also are important influencing factors for mental health. In recent years, with the development of deep learning technologies, emotion recognition algorithms based on physiological signals have made remarkable progress. In particular, electroencephalogram (EEG) signals have received extensive attention due to their close relationship with emotions.

[0003] In the emotion recognition task based on electroencephalogram (EEG) signals, the existing methods have the following deficiencies:

[0004] 1. Limited ability to extract global features: Although some methods use graph convolutional networks (GCNs) to model the connections between EEG nodes, they usually use a fixed adjacency matrix and fail to fully explore the implicit dynamic relationships in EEG pathways, resulting in insufficient learning ability of the model for global features.

[0005] 2. Lack of effective utilization of biological priors: Although medical research has revealed the importance of different brain regions in emotion processing, many models fail to reasonably utilize this prior knowledge and simply treat EEG nodes as independent individuals, ignoring the structured information of brain region division.

[0006] 3. Simple feature fusion strategy: Current models usually fuse multi-modal or multi-path features by direct concatenation or weighted summation, lacking more refined interaction modeling, which easily leads to insufficient complementarity between information flows and thus limits the improvement of model performance. Summary of the Invention

[0007] To solve the problem that the existing technology fails to fully consider the brain region structure and the signal propagation path related to emotions, the present invention provides a dual-channel EEG emotion recognition method and system combining general path learning and medical prior knowledge, effectively combining general emotion recognition features with the biological prior knowledge of brain region division, and also optimizing the fusion of the two-channel information through a cross-attention mechanism, providing a more comprehensive and accurate feature representation for emotion recognition.

[0008] On the one hand, an embodiment of the present invention provides a dual-channel EEG emotion recognition method combining general path learning and medical prior knowledge, including the following steps:

[0009] S1. Preprocess the original electroencephalogram signal; the preprocessing includes filtering and extracting differential entropy features from the filtered electroencephalogram signal;

[0010] S2. Construct an adjacency matrix A based on the general path C , which is used to capture the general connection pattern between EEG channels;

[0011] S3. Construct an adjacency matrix A for the brain regions divided based on medical prior R , which is used to describe the connection pattern between the brain regions divided based on medical prior;

[0012] S4. Take the differential entropy feature as the node feature, and take the adjacency matrix A C as the edge to construct a path graph structure; take the differential entropy feature as the node feature, and take the adjacency matrix A R as the edge to construct a prior graph structure; based on the cross-attention mechanism, perform graph convolution operations on the path graph structure to update the node features according to the adjacency matrix A C , perform graph convolution operations on the prior graph structure to update the node features according to the adjacency matrix A R ; fuse the updated two-way node features to obtain a high-dimensional fused feature matrix representation;

[0013] S5. According to the high-dimensional fused feature matrix, use a multi-layer perceptron to identify and classify EEG emotions.

[0014] On the other hand, an embodiment of the present invention provides a dual-channel EEG emotion recognition system combining general path learning and medical prior, which is implemented based on the above recognition method. The recognition system includes the following modules:

[0015] A preprocessing module, which is used to preprocess the original EEG signal, including filtering processing, and extracting differential entropy features from the filtered EEG signal;

[0016] A first adjacency matrix construction module, which is used to construct an adjacency matrix A based on the general path C , which is used to capture the general connection pattern between EEG channels;

[0017] A second adjacency matrix construction module, which is used to construct an adjacency matrix A for the brain regions divided based on medical prior R , which is used to describe the connection pattern between the brain regions divided based on medical prior;

[0018] A graph structure construction and feature fusion module, which is used to take the differential entropy feature as the node feature, and take the adjacency matrix A C as the edge to construct a path graph structure; take the differential entropy feature as the node feature, and take the adjacency matrix A R as the edge to construct a prior graph structure; based on the cross-attention mechanism, perform graph convolution operations on the path graph structure to update the node features according to the adjacency matrix A C , perform graph convolution operations on the prior graph structure to update the node features according to the adjacency matrix A RUpdate the node features; fuse the two updated node features to obtain a high-dimensional fused feature matrix representation;

[0019] A classification module, which is used to identify and classify EEG emotions through a multi-layer perceptron based on the high-dimensional fused feature matrix.

[0020] Compared with the prior art, the beneficial effects achieved by the present invention include:

[0021] Through a dual-channel parallel model that combines EEG pathway learning and prior medical brain region division, dual-channel data is processed in parallel. That is, while automatically learning the global features related to emotion recognition in EEG data, the channel features of different brain regions are weighted and fused through an attention mechanism to calculate the correlation between different brain regions; and the data after dual-channel parallel processing is informationally fused, so as to perform multi-dimensional space emotion recognition through graph convolution and autoencoders, improving the performance of the emotion recognition task. The present invention can not only effectively combine the general emotion recognition features with the biological prior of brain region division, but also optimize the fusion of the two-channel information through a cross-attention mechanism, providing a more comprehensive and accurate feature representation for emotion recognition. Description of the Drawings

[0022] Figure 1 It is a flowchart of the dual-channel EEG emotion recognition method in an embodiment of the present invention. Detailed Embodiments

[0023] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0024] Embodiment

[0025] This embodiment provides a dual-channel EEG emotion recognition method that combines general path learning and medical prior knowledge. For the emotion recognition task based on electroencephalogram (EEG) signals, a dual-channel parallel model that combines EEG path learning and medical prior brain region division is designed. The core innovation of this method lies in processing two information flows through a graph convolutional network (GCN) respectively. The first path is the general path, which automatically learns the global features related to emotion recognition from EEG data through a learnable adjacency matrix, and captures the potential patterns in emotion classification. The second path is the medical prior path. Based on the medical prior knowledge of EEG data, the electrode nodes are divided into several brain regions, and the channel features are weighted and fused through an attention mechanism within each brain region. Then, the correlation between brain regions is calculated to construct an adjacency matrix, strengthening the biological rationality of spatial features. These two paths are processed in parallel, and finally, information fusion is performed through a cross-attention mechanism, so as to complementarily enhance the global pattern and brain region features, and improve the performance of the emotion recognition task.

[0026] As Figure 1 shown, the dual-channel EEG emotion recognition method of this embodiment specifically includes the following steps:

[0027] S1. Preprocess the original electroencephalogram signal.

[0028] During the preprocessing of the electroencephalogram (EEG) signal, a band-pass filter is first used to filter the signal to eliminate noise and artifacts. Specifically, the EEG data is processed through a band-pass filter of 1 Hz to 75 Hz to ensure that the signal is within the specified frequency range. Subsequently, in this embodiment, for the filtered electroencephalogram signal, the power spectral density (PSD) features and differential entropy (DE) features are extracted from each 1-second data segment. These features are calculated in five frequency bands: 1) δ wave: 1 - 4 Hz; 2) θ wave: 4 - 8 Hz; 3) α wave: 8 - 14 Hz; 4) β wave: 14 - 31 Hz; 5) γ wave: 31 - 50 Hz. The power spectral density (PSD) is a measure of the signal power distribution with respect to frequency, and the differential entropy (DE) is an index to measure the complexity of the signal. The calculation formula of DE is as follows:

[0029]

[0030] In this embodiment, it is assumed that the EEG signal follows a Gaussian distribution. Under this assumption, the calculation of the DE feature can be simplified. For a random variable following a Gaussian distribution, the calculation formula of its differential entropy can be further expressed as:

[0031]

[0032] where σ is the standard deviation, μ is the mean, and σ 2 represents the variance.

[0033] In this way, the complexity and information content of EEG signals in different frequency bands can be quantified, providing key feature support for subsequent emotional state analysis. The extracted differential entropy (DE) features are used as node features in the subsequent construction of the graph connection structure to effectively characterize the time-frequency information of each channel, thereby improving the accuracy of emotion recognition.

[0034] The constructed dataset is a leave-one-subject-out dataset across subjects, which includes training data and test data. When constructing the leave-one-subject-out dataset across subjects, a strategy is adopted that in each model training process, the data of one subject is used as the test set, and the data of the remaining subjects is used as the training set. This method enables each subject to be used as an independent test set for evaluation, thus ensuring that the model can be effectively verified on the data of multiple subjects, further enhancing the generalization ability of the model. This evaluation method helps to verify whether the model can adapt to new individual data, avoids overfitting, and can better reflect the prediction ability of the model for unknown individuals in practical applications.

[0035] In the specific implementation, based on the publicly available "SJTU Emotion EEG Dataset" electroencephalogram emotion dataset, which contains 15 Chinese subjects (7 males and 8 females, with an average age of 23.27 years and a standard deviation of 2.37). To protect the privacy of the subjects, the dataset anonymizes the names of the subjects and labels each subject from 1 to 15. The data collection includes EEG and eye movement data. Among them, subjects numbered 1-5 and 8-14 (a total of 12 subjects) provided EEG and eye movement data, while subjects numbered 6, 7, and 15 only provided EEG data. In the training process of the leave-one-subject-out method, one subject is selected as the test set in each round of experiment, and the data of the remaining 14 subjects is used as the training set. After each training is completed, the performance of the model on this test set is evaluated, and by repeating this process, the average performance of the model on all subjects is finally obtained. This method can make full use of each piece of information in the dataset, while ensuring that the model can adapt to the differences of different individuals, thereby improving its adaptability and robustness in practical applications.

[0036] S2. Construct an adjacency matrix based on the general path.

[0037] In the fields of emotion and cognition, the propagation of EEG signals in brain regions usually presents a fixed pathway pattern in the same task. In the emotion recognition task, the same emotion will be activated in specific brain regions and propagate in the brain along a fixed path.

[0038] This fixed pathway pattern is reflected in the connection and signal propagation paths between nodes when constructing the brain region graph connection structure from EEG signals. The adjacency matrix can be used to represent them, showing a general pattern independent of the input. Therefore, instead of trying to manually define the fixed brain region graph connection relationship, this embodiment adopts a learnable adjacency matrix to automatically mine this general graph connection structure from the training data of the dataset through the model. After the model training is completed, this adjacency matrix will be fixed and applied to all input instances.

[0039] In this embodiment, the adjacency matrix that can be fixed after the above model training can be expressed as Specifically, the adjacency matrix A C is obtained by learning through gradient optimization from EEG signals and is used to capture the general connection pattern between EEG channels.

[0040] In the implementation process, the process of obtaining the adjacency matrix for capturing the general connection pattern between EEG channels includes the following steps:

[0041] S21. Initialize the adjacency matrix A C as a matrix of size N×N (where N is the number of EEG channels). The initialization method can adopt random initialization with a normal distribution to provide sufficient flexibility to adapt to the individual differences of different subjects while ensuring the rationality of the initial structure.

[0042] S22. Under the framework of the graph convolutional network, define learnable parameters for the adjacency matrix A C so that it can be adaptively optimized through gradient descent during the training process of the graph convolutional network model, thereby dynamically adjusting the connection weights between EEG channels.

[0043]

[0044] Among them, is the adjacency matrix A obtained by the (t + 1)-th gradient descent update C ; is the adjacency matrix A obtained by the t-th gradient descent update C ; ρ is the update weight hyperparameter used to control the update amplitude of the adjacency matrix to ensure that it can maintain stability during the optimization process and can be flexibly adjusted to enhance the feature expression ability of EEG signals. represents the gradient of the loss function with respect to the adjacency matrix, measuring the impact of the current adjacency matrix on the optimization objective.

[0045] This update mechanism adjusts the adjacency matrix through gradient feedback, enabling it to adaptively learn the optimal connection relationships between nodes during training, rather than relying on a fixed structure defined a priori. Through this mechanism, the model can continuously optimize the topological structure during training to better fit the spatial correlation characteristics of EEG signals, thereby improving the expressive ability of the emotion recognition task. This dynamic update method can effectively enhance the performance of the graph neural network in feature extraction and classification tasks, making the learned graph structure more in line with the intrinsic topological characteristics of the data. S23. Perform symmetric normalization on the adjacency matrix.

[0046] To enhance the stability of the model and the robustness of numerical calculations, the adjacency matrix in this embodiment uses symmetric normalization. The normalized adjacency matrix is expressed as:

[0047]

[0048] where D is the degree matrix of the adjacency matrix A C , and the elements on the diagonal are defined as D ii = ∑ j A Cij . Among them, A cij indicates whether there is an edge between node i and node j (1 means yes, 0 means no). The degree matrix D is defined as:

[0049]

[0050] where n is the number of nodes in the graph connection structure.

[0051] S24. After the graph convolutional network model is trained, fix the adjacency matrix A C .

[0052] After the model is trained, the adjacency matrix will be solidified as a fixed topological structure for simulating the transmission of emotion information between brain regions and used in the inference stage for all input instances. In the subsequent verification stage, this adjacency matrix will be directly applied to ensure that the model maintains a consistent connection pattern during inference, thereby stably reflecting the functional interaction relationship between brain regions.

[0053] S3. Construct an adjacency matrix based on the brain regions divided according to medical prior knowledge.

[0054] To better simulate the signal transmission between brain regions, divide the brain regions based on medical prior knowledge and combine the attention mechanism to achieve adaptive feature fusion of different brain regions. The specific steps are as follows:

[0055] S31. Divide the brain regions according to medical prior knowledge and obtain the corresponding position relationships between each brain region and the electrodes.

[0056] In the emotion recognition task, channels in different brain regions may contribute differently to the overall features. In this embodiment, according to medical prior knowledge, for example, based on the 10-20 system and medical brain region classification, the EEG channels of the 62-channel ESI NeuroScan system are divided into 16 brain regions. The corresponding position relationships between each brain region and the electrodes are shown in Table 1:

[0057] Table 1

[0058]

[0059]

[0060] As can be seen from Table 1, each brain region includes 2 - 6 EEG channels.

[0061] S32. Within each brain region, the EEG channel features are weighted and fused through an attention mechanism to obtain the fused brain region features.

[0062] Specifically, first, based on the corresponding position relationship in step S31, according to the input brain region name and channel list, the EEG channels of each brain region are mapped to the index positions in the input feature tensor X:

[0063]

[0064] Among them, regions_idx[r] represents the set of channel indices corresponding to the r-th brain region, channels[i] represents the i-th EEG node channel, regions[r] represents the r-th brain region, and R is the number of brain regions.

[0065] Then, a weight vector is defined for each brain region where n r is the number of channels included in the r-th brain region. The initial weights are set to all 1, and are subsequently optimized through gradient descent. For each brain region, the corresponding channel features are extracted from the input feature tensor X, weighted calculated and normalized to obtain the aggregated features (also called fused features) of different brain regions. The fused brain region feature H region is expressed as:

[0066]

[0067] where, h i represents the feature of the i-th node channel in the brain region; e i , e j are the channel weights, initialized as learnable parameters, and adaptively updated through training. i and j represent different node channels; α i is the normalized attention weight, used to calculate the attention relationship between the two node channels i and j, and the sum of all attention weights is 1; nchannel is the number of channels in the brain region.

[0068] S33. On the basis of brain region feature fusion, construct an adjacency matrix for the brain regions divided based on medical prior knowledge to describe the connection pattern between the brain regions divided based on medical prior knowledge, so as to simulate the signal propagation between different brain regions.

[0069] This connection pattern between brain regions is realized through a dynamically learned adjacency matrix. This dynamically learned adjacency matrix can not only capture the inherent relationship between brain regions, but also adapt to the requirements of the emotion recognition task and improve the model's expression ability for relevant emotion patterns. The construction process of the connection pattern between brain regions in this step includes:

[0070] S331. Calculate the spatial similarity between brain region features. The calculation process is as follows: First, based on the input brain region features, calculate the spatial similarity between every two brain region features. Assume the input brain region features are where B is the batch size, N represents the number of brain regions (for example, 16 brain regions), and F is the feature dimension of each brain region. By calculating the spatial similarity of brain region features, an initial similarity matrix S is generated, and the element S of the similarity matrix ij is defined by the following formula:

[0071]

[0072] where x i represents the feature vector of the i-th brain region, and x j represents the feature vector of the j-th brain region.

[0073] S332. Global prior and learnable connection: In order to combine the prior knowledge of medicine and biology and at the same time enhance the model's adaptability to EEG data, a globally learnable prior matrix Aglobal is introduced to represent the fixed connection weights between brain regions. Multiply the similarity matrix by the prior matrix and process it through an activation function (such as ReLU) to obtain an unnormalized adjacency matrix Araw:

[0074]

[0075] where represents the inversion of the prior matrix.

[0076] S333. Self-loop supplement: In order to ensure that the features of each brain region can not only interact with other brain regions, but also retain its own information, add a self-loop to the unnormalized adjacency matrix Araw. Specifically, by adding the identity matrix I, an adjacency matrix Aself-loop including self-loops is obtained:

[0077] Aself-loop = Araw + I

[0078] S334. Symmetric normalization: To enhance the numerical stability of the model and make the information propagation in the graph convolution operation more efficient, the adjacency matrix Aself-loop is symmetrically normalized. Specifically, first, calculate the degree matrix D of the adjacency matrix Aself-loop. The diagonal elements of the degree matrix are defined as:

[0079]

[0080] Then, calculate the square root inverse matrix of the degree matrix and apply it to the adjacency matrix including self-loops to generate the final normalized adjacency matrix A as the adjacency matrix A R :

[0081]

[0082] By introducing learnable weights, the method of this embodiment can adaptively adjust the contribution of each channel to the brain region features, thereby enhancing the feature representation ability. On this basis, the normalization operation ensures the consistency of the scale of different brain region features during the fusion process, thereby improving the training stability and convergence speed of the model. At the same time, combined with the medical prior knowledge of the brain regions, the method of this embodiment can more accurately extract the spatial features of EEG data and reasonably model the signals from a biological perspective. Specifically, the construction of the adjacency matrix A further strengthens this modeling process. The adjacency matrix A not only automatically adjusts the connection relationship between brain regions based on the similarity of the input data, but also incorporates global prior information, enabling the model to capture potential emotion recognition pathways, and ultimately making the model more accurate and robust in the emotion classification task.

[0083] S4. Use the differential entropy feature as the node feature and the adjacency matrix A C as the edge to construct a path graph structure; use the differential entropy feature as the node feature and the adjacency matrix A R as the edge to construct a prior graph structure; perform graph convolution operations on the path graph structure to update the node features according to the adjacency matrix A C ; perform graph convolution operations on the prior graph structure to update the node features according to the adjacency matrix A R ; fuse the two updated node features to obtain a high-dimensional fused feature matrix representation.

[0084] In this embodiment, steps S2 and S3 are in a parallel relationship. The adjacency matrix of the fixed general path is constructed through step S2, and the adjacency matrix that can be learned based on brain region priors is constructed through step S3. Then, graph convolution operations are performed in step S4 to achieve the fusion of node features. That is, based on the cross-attention mechanism, after extracting node features from the prior graph structure and the pathway graph structure through graph convolution operations respectively, the cross-attention module is used to fuse the extracted dual-path node features, so as to fully integrate the prior information and the data-driven topological relationship, thereby enhancing the discriminative ability of emotion recognition.

[0085] This step adopts a parallel modeling method based on graph convolution, constructs independent graph structures based on the pathway pattern in step S2 and the medical prior brain region pattern in step S3 respectively, which can be denoted as the pathway graph structure and the prior graph structure respectively, and performs graph convolution operations through the convolutional network (GCN) respectively to fully mine the node feature information in the two patterns. These two parts define different adjacency matrices respectively: the adjacency matrix of the pathway pattern is denoted as A C , and the adjacency matrix of the brain region pattern is denoted as A R . The node features of the two patterns are integrated through a fusion mechanism after parallel computing, and finally serve the downstream emotion classification task.

[0086] Specifically described as follows: The goal of the pathway pattern is to learn the general feature path related to the emotion recognition task. For this purpose, the constructed adjacency matrix A C adopts a data-driven and learnable manner and is independent of the input features, representing the global general feature path in emotion recognition. Based on the adjacency matrix A C , the formula for updating node features through graph convolution operations is:

[0087]

[0088] where is the node feature representation of the l-th layer, and the node feature of the first layer (i.e., when l = 1) is the differential entropy feature extracted after the preprocessing in step S1; N is the number of nodes, d is the feature dimension; is the normalized adjacency matrix of the general path; D c is the degree matrix of the adjacency matrix A C ; is the learnable weight matrix of the l-th layer; σ is the non-linear activation function. Through the stacking of multiple layers of graph convolution, the model can aggregate the features from different nodes layer by layer, thereby capturing the global pattern in emotion recognition.

[0089] For the brain region pattern divided based on medical priors, the adjacency matrix A R is constructed through the node correlation of brain region division to model the connection characteristics between brain regions. Based on the adjacency matrix A R, the formula for updating node features through graph convolution operations is as follows:

[0090]

[0091] Among them, is the node feature representation of the l-th layer. The node features of the first layer (i.e., when l = 1) are the differential entropy features extracted after the preprocessing in step S1; M is the number of brain regions; is the normalized adjacency matrix based on brain region correlation, that is, the normalized adjacency matrix of the adjacency matrix A R ; is the weight matrix of the l-th layer.

[0092] The pathway pattern extracts the global features related to emotion recognition, while the brain region pattern enhances the biological rationality of the features based on medical prior knowledge and is the local feature; the parallel graph convolution operations combine the global and local characteristics, and the fusion realizes the complementary enhancement of the global and local characteristics, thereby improving the emotion classification ability of the model.

[0093] To effectively combine the feature representations of the general path and the medical prior path, this embodiment introduces a cross-attention mechanism (Cross-Attention Mechanism) to achieve complementary enhancement between the two information flows and further improve the emotion recognition performance of the model. Assume that the general path feature of the first path is The medical prior path feature of the second path is where B is the batch size, N is the number of nodes (either EEG channels or brain regions), d C and d P are the dimensions of the two-way features respectively.

[0094] In the specific implementation process, first project the two-way input features into the query Q, key K, and value V spaces respectively:

[0095]

[0096] Among them, and are both learnable linear projection matrices, and d a is the dimension of the attention space.

[0097] Next, calculate the attention weights through cross-attention between the first path and the second path respectively, and perform weighted fusion. The attention modeling of the first path feature on the second path feature is completed by calculating the similarity between the query vector of the first path (Q C ) and the key vector of the second path (K P ). Specifically, this process quantifies the dependence of the first path feature on the second path feature through attention weights. The specific formula is as follows:

[0098]

[0099] The softmax operation calculates the attention distribution along the node dimension. The attention weight is used to calculate the value V of the second path. P Perform weighted summation to obtain the fusion representation of the first path to the second path. Similarly, the attention modeling of the second path feature to the first path feature is achieved by calculating the query vector (Q P ) and the key vector (K C ). Specifically, this process measures the dependency of the second-path feature on the first-path feature through the attention weight, and its mathematical form is as follows:

[0100]

[0101] Similarly, the second path uses the attention weight to calculate the value V of the first path. C In this process, the second-path features focus on the first-path features through the cross-attention mechanism and dynamically integrate their information according to the learned attention distribution. This modeling enables the second-path features to effectively absorb the complementary information in the first-path features, further enrich the feature representation, and improve the model's expressiveness and task performance.

[0102] Finally, the result obtained by cross-attention calculation is fused with the original features to achieve information interaction:

[0103] H C =ReLU(XC+AttentionC→P)

[0104] H P =ReLU(XP+AttentionP→C)

[0105] Among them, ReLU is a nonlinear activation function used to enhance the representation ability of the model. C and H P Perform concatenation or weighted fusion to obtain the final feature matrix representation:

[0106] Hfused=concat(H C ,HP)

[0107] S5. Based on the high-dimensional fusion feature matrix, EEG emotions are identified and classified through a multi-layer perceptron.

[0108] In the emotion recognition task, after the aforementioned two-way parallel feature extraction and cross-attention fusion, a high-dimensional fusion feature representation is obtained. Where N represents the number of nodes (or brain regions), and d represents the feature dimension of each node.

[0109] To further utilize these fused features to complete the emotion classification task, a multi-layer perceptron (MLP) is introduced as the classifier. The specific steps are as follows: Since the MLP accepts input feature vectors of a fixed dimension, first flatten the high-dimensional feature matrix Z into a vector zflatten = Flatten(Z), and this operation converts the two-dimensional structure of nodes and features into a one-dimensional representation. The flattened vector passes through multiple fully connected layers (FCs). Each fully connected layer includes a linear transformation, a non-linear activation function, and batch normalization for feature mapping and extraction of high-level semantic features. The feature mapping of the l-th layer is represented as:

[0110] h (l) = σ(W (l) h (l-1) + b (l) ), l = 1, …, L,

[0111] where, h (0) = z flatten is the input feature; is the weight matrix of the l-th layer; is the bias vector; d l is the output dimension of the l-th layer; σ(·) is the activation function, such as ReLU or GELU. The last fully connected layer maps the high-dimensional features to the predicted probabilities of emotion categories. Assuming the emotion classification task has C categories, the representation of the output layer is:

[0112]

[0113] where, is the predicted probability distribution of the category; are the weights and biases of the last layer. To train the classifier, define the cross-entropy loss function to optimize the difference between the prediction result and the true label y:

[0114]

[0115] where, M is the number of training samples; y i,c ∈ {0, 1} indicates whether sample i belongs to category c (one-hot encoding); is the predicted probability that the model assigns sample i to category c. Optimize this loss function through gradient descent to update all model parameters, including the weights and biases of the MLP.

[0116] Based on the same inventive concept, this embodiment also provides a dual-channel electroencephalogram emotion recognition system that combines general path learning and medical prior knowledge, which is implemented based on the above-mentioned dual-channel electroencephalogram emotion recognition method. The recognition system includes the following modules:

[0117] A preprocessing module for preprocessing the original electroencephalogram signal, including filtering processing and extracting differential entropy features from the filtered electroencephalogram signal;

[0118] A first adjacency matrix construction module for constructing an adjacency matrix A based on the general path C for capturing the general connection pattern between EEG channels;

[0119] A second adjacency matrix construction module for constructing an adjacency matrix A of brain regions divided based on medical prior knowledge R for describing the connection pattern between brain regions divided based on medical prior knowledge;

[0120] A graph structure construction and feature fusion module for using the differential entropy feature as the node feature and the adjacency matrix A c as the edge to construct a path graph structure; using the differential entropy feature as the node feature and the adjacency matrix A R as the edge to construct a prior graph structure; based on the cross-attention mechanism, performing graph convolution operations on the path graph structure to update the node features according to the adjacency matrix A C and performing graph convolution operations on the prior graph structure to update the node features according to the adjacency matrix A R ; fusing the updated two-channel node features to obtain a high-dimensional fusion feature matrix representation;

[0121] A classification module for recognizing and classifying electroencephalogram emotions through a multi-layer perceptron based on the high-dimensional fusion feature matrix.

[0122] The above-mentioned modules are respectively used to implement steps S1 - S5 of the emotion recognition method, and the detailed implementation process refers to the specific step description.

[0123] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A dual-channel EEG emotion recognition method combining general path learning and medical prior, characterized in that It includes the following steps: S1. Preprocess the original electroencephalogram (EEG) signal; the preprocessing includes filtering processing and extracting differential entropy features from the filtered EEG signal; S2. Construct the adjacency matrix A based on the common path C , which is used to capture the common connection patterns between EEG channels; S3. Construct the adjacency matrix A of the brain regions divided based on medical prior knowledge R , which is used to describe the connection pattern between the brain regions divided based on medical prior knowledge; S4. Use the differential entropy feature as the node feature and the adjacency matrix A C as the edge to construct a path graph structure; Using the differential entropy feature as the node feature and the adjacency matrix A R as the edge to construct the prior graph structure; Based on the cross-attention mechanism, perform graph convolution operations on the pathway graph structure according to the adjacency matrix A C Update the node features, and perform graph convolution operations on the prior graph structure according to the adjacency matrix A R Update the node features; Fuse the updated two-way node features to obtain a high-dimensional fused feature matrix representation; S5. According to the high-dimensional fused feature matrix, identify and classify EEG emotions through a multi-layer perceptron.

2. The dual-channel EEG emotion recognition method according to claim 1, wherein Adjacency matrix A of step S2 C Obtained by gradient optimization learning from EEG signals; the steps for constructing the adjacency matrix A C include: S21. Initialize the adjacency matrix A C ; S22. Under the framework of the graph convolutional network, the adjacency matrix A C is defined as a learnable parameter, and is adaptively optimized through gradient descent during the training of the graph convolutional network model to dynamically adjust the connection weights between EEG channels; S23. Perform symmetric normalization on the adjacency matrix to obtain the normalized adjacency matrix. After the graph convolutional network model is trained, fix the adjacency matrix A C .

3. The dual-channel EEG emotion recognition method according to claim 1, characterized in that, Step S3 includes: S31. Divide brain regions according to medical priors and obtain the corresponding position relationship between each brain region and the electrodes; S32. In each brain region, perform weighted fusion on EEG channel features through an attention mechanism to obtain the fused brain region features; S33. On the basis of brain region feature fusion, construct an adjacency matrix A for the brain regions divided based on medical prior knowledge R .

4. The dual-channel EEG emotion recognition method according to claim 3, wherein Step S33 includes: S331. Calculate the spatial similarity between brain region features to generate an initial similarity matrix S; S332. Introduce a globally learnable prior matrix Aglobal to represent the fixed connection weights between brain regions; multiply the similarity matrix by the prior matrix and process it through an activation function to obtain an unnormalized adjacency matrix Araw; S333. Add an identity matrix I to the unnormalized adjacency matrix Araw to obtain an adjacency matrix Aself-loop including self-loops; S334. Symmetrically normalize the adjacency matrix A self-loop including self-loops to generate the final normalized adjacency matrix A as the adjacency matrix A R .

5. The dual-channel EEG emotion recognition method according to claim 3, characterized in that, Step S32 includes: First, based on the corresponding position relationship in Step S31, according to the input brain region name and channel list, map the EEG channels of each brain region to the index positions in the input feature tensor X: where regions_idx[r] represents the set of channel indices corresponding to the r-th brain region, channels[i] represents the i-th EEG node channel, regions[r] represents the r-th brain region, and R is the number of brain regions. Then, a weight vector is defined for each brain region where n r is the number of channels included in the r-th brain region; the initial weights are set to all 1 and are subsequently optimized by gradient descent; for each brain region, its corresponding channel features are extracted from the input feature tensor X, weighted and calculated, and normalized to obtain the fused features of different brain regions.

6. The dual-channel EEG emotion recognition method according to claim 5, wherein The fused brain region feature H region Expressed as: Among them, h i represents the feature of the i-th node channel in the brain region; e i and e j are channel weights; α i is the normalized attention weight, which is used to calculate the attention relationship between the i-th and j-th node channels; n channel is the number of channels in the brain region.

7. The dual-channel EEG emotion recognition method according to claim 1, wherein In step S4, based on the adjacency matrix A C , the formula for updating the node features through graph convolution operation is: Among them, is the node feature representation of the l-th layer. When l = 1, the node feature is the differential entropy feature extracted after the preprocessing in step S1; N is the number of nodes, and d is the feature dimension; is the normalized adjacency matrix of the general path, D C is the adjacency matrix A C 's degree matrix; is the learnable weight matrix of the l-th layer; σ is the non-linear activation function.

8. The dual-channel EEG emotion recognition method according to claim 1, characterized in that, In step S4, based on the adjacency matrix A R , the formula for updating node features through graph convolution operation is: Among them, is the node feature representation of the l-th layer. When l = 1, the node feature is the differential entropy feature extracted after the preprocessing in step S1; M is the number of brain regions; is the adjacency matrix A R 's normalized adjacency matrix; is the weight matrix of the l-th layer.

9. The dual-channel EEG emotion recognition method according to claim 1, wherein In Step S5, the multi-layer perceptron flattens the high-dimensional feature matrix into a vector, and the flattened vector performs feature mapping and extraction of high-level semantic features through multiple fully connected layers. The last fully connected layer maps the high-dimensional features to the prediction probabilities of emotion categories.

10. A dual-channel EEG emotion recognition system combining general path learning and medical prior, characterized in that, Implemented based on the recognition method described in any one of claims 1-9, the recognition system includes the following modules: A preprocessing module for preprocessing the original EEG signal, including filtering processing and extracting differential entropy features from the filtered EEG signal; The first adjacency matrix construction module is used to construct the adjacency matrix A based on the general path C , which is used to capture the general connection pattern between EEG channels; The second adjacency matrix construction module is used to construct an adjacency matrix A for the brain regions divided based on medical prior knowledge R , which is used to describe the connection pattern between the brain regions divided based on medical prior knowledge; The graph structure construction and feature fusion module is used to construct a path graph structure by taking the differential entropy feature as the node feature and the adjacency matrix A C as the edge Using the differential entropy feature as the node feature and the adjacency matrix A R as the edge to construct the prior graph structure; Based on the cross-attention mechanism, perform graph convolution operations on the pathway graph structure to update node features according to the adjacency matrix A C Perform graph convolution operations on the prior graph structure to update node features according to the adjacency matrix A R Update node features; Fuse the updated two-way node features to obtain a high-dimensional fused feature matrix representation; A classification module for identifying and classifying EEG emotions through a multi-layer perceptron according to the high-dimensional fused feature matrix.

Images