Cross-subject emotion recognition method, device and system based on EEG multi-branch graph convolution and storage medium

Abstract

The invention relates to the technical field of brain wave signal processing, and discloses a cross-subject emotion recognition method, device and system based on EEG multi-branch graph convolution and a storage medium, and the method comprises the steps: obtaining EEG data; filtering the EEG data to obtain a first signal; filtering the first signal in five frequency bands to obtain a second signal; in the second signal, differential entropy features are extracted according to a time window of set time; based on the differential entropy features, graph features of all frequency bands are extracted through a multi-branch graph convolutional network; inputting the image features of each frequency band into an image feature fusion module for fusion to obtain emotion features; and inputting the emotion features into a multi-layer perceptron to obtain a prediction probability of the emotion, and using a domain adaptation technology to close feature distribution between a source domain and a target domain in a training stage. According to the method, the influence of cross-domain data distribution difference on model performance can be effectively reduced, and the accuracy and robustness of emotional state recognition are improved.

Description

Technical Field

[0001] The present application relates to the technical field of brain wave signal processing, and in particular to a method, device, system and storage medium for cross-subject emotion recognition based on EEG multi-branch graph convolution. Background Art

[0002] Emotion is a complex psychological state that affects human cognition, communication and decision-making in daily life. Emotion is affected by the individual's subjective feelings, external stimuli and physiological signal arousal, and can be identified through non-physiological signals or physiological signals. In daily life, face, body posture, voice and gesture can all intuitively express emotional state. However, individuals can hide their true emotions through subjective consciousness, which makes it difficult to recognize emotions based on non-physiological signals. In recent years, people have paid more and more attention to emotion recognition based on physiological signals, such as electroencephalogram (EEG), heart rate (HR), skin charge (GSR) and electromyography (EMG). Compared with non-physiological signals, physiological signals can directly express the emotional state of individuals, and the recognition results are more reliable. In addition, the acquisition of EEG signals is non-invasive, portable and easy to operate. Therefore, emotion recognition based on EEG has received extensive attention and research.

[0003] In EEG signal processing, due to the non-stationarity of EEG signals, the collected signals will have large distribution changes due to differences in time, subjects, and equipment. This variability will affect the generalization of the model in cross-subject tasks and increase the difficulty of applying the model to different individuals. In recent years, transfer learning has gradually become a popular method to solve cross-subject problems, especially domain adaptation technology. Domain adaptation can adjust the model to adapt to the data distribution of the target domain (new subjects) so that the model can perform well in the data of new subjects. Since graph convolution shows superior performance in processing graph structure data, EEG signals can be modeled as graph structures, where each node represents an EEG channel and the edges between nodes represent physiological or functional connections between different channels. Therefore, some studies combine graph convolution with transfer learning to solve the cross-subject problem of emotions, such as using the maximum mean difference (MMD) to measure the distribution of node information in the source domain and the target domain, thereby minimizing the difference in feature data distribution between the two. However, this method ignores the common spatial patterns between subjects. Secondly, most graph convolutions combined with transfer learning use a single branch structure to capture topological features. However, this method is difficult to solve the problem of differences in different graph structures in EEG data. To this end, some studies have introduced multi-branch structures to simultaneously capture graph features at different levels or angles to cope with the diversity of graph structures.

[0004] Therefore, the existing methods of cross-subject emotion recognition based on graph convolution have the following problems: (1) Emotional states show obvious differences in inter-channel correlations in different frequency bands. For example, in low-frequency bands (such as alpha and theta waves), emotions usually involve coordinated activities in a wider range of brain regions, while in high-frequency bands (such as beta and gamma waves), active areas are more concentrated in specific functional areas. Single-branch graph convolution cannot reflect this difference, resulting in the inability to fully utilize the relationship between the channels of EEG signals.

[0005] (2) The fusion method of multi-branch graph convolution usually adopts splicing, but this method cannot effectively promote the information interaction between different graph features. Splicing simply juxtaposes the features of each branch and lacks modeling of the relationship between features, thus limiting the model's ability to learn complex graph structures.

[0006] (3) Cross-subject learning methods based on graph convolution mainly focus on the overall alignment of graph node information, ignoring the common spatial patterns of the same emotions between different subjects. Summary of the invention

[0007] In order to solve the above problems, the present application provides a cross-subject emotion recognition method, device, system and storage medium based on EEG multi-branch graph convolution, which comprehensively mines the correlation between EEG signal channels by capturing the topological features of each frequency band, and uses domain adaptation technology to constrain the learned adjacency matrix in the feature space based on the data of the source domain and the target domain, combined with the asymmetric spatial pattern of the emotional state in the brain area, so as to construct a common graph pattern in the multi-source domain data, thereby improving the generalization ability of the model in the target domain, and effectively reducing the impact of cross-domain data distribution differences on model performance, thereby improving the accuracy and robustness of emotional state recognition.

[0008] In order to achieve the above purpose, the technical solution adopted in this application is as follows: In a first aspect, the present application provides a cross-subject emotion recognition method based on EEG multi-branch graph convolution, the method comprising: Acquire EEG data; Filtering the EEG data to obtain a first signal; Filtering the first signal in five frequency bands to obtain a second signal; In the second signal, extracting differential entropy features in a time window of a set time; Based on the differential entropy features, the graph features of each frequency band are extracted through a multi-branch graph convolutional network; Input the graph features of each frequency band into the graph feature fusion module for fusion to obtain the emotion features; Inputting the emotion features into a multi-layer perceptron to obtain the predicted probability of the emotion; A multi-branch graph convolutional network, a graph feature fusion module and a multi-layer perceptron connected in sequence are used as an emotion recognition model, the emotion recognition model is trained, and emotion recognition is achieved through the trained model.

[0009] Furthermore, in the second signal, the differential entropy feature is extracted in a time window of a set time, and the extraction formula is: ; In the formula, Indicates Frequency band The differential entropy characteristics of the channels, and It is Frequency band The mean and variance of the EEG signal of each channel, is a natural constant, For the Frequency band EEG signals of 1 channel.

[0010] Furthermore, based on the differential entropy feature, the graph features of each frequency band are extracted through a multi-branch graph convolutional network, including: The differential entropy feature is used as the input signal, and its adjacency matrix is ​​calculated based on the input signal. The adjacency matrix construction formula is as follows: ; In the formula, Indicates The adjacency matrix of the frequency bands, express The differential entropy characteristics of the frequency bands, and are two learnable weight matrices, represents the matrix transpose, express ,exp means function; Take the input signal and its adjacency matrix as nodes and calculate the maximum edge of each node. The index of is calculated as: ; In the formula, Indicates The front of the node edge weight A large index, Indicates j The edge set of a node; The function is expressed as: ; In the formula, represents the index vector of the edge, indicates that there are k edges, represents the j-th largest edge index, represents the j+1th largest edge index, The ordinal number representing the edge index, represents the j-th largest edge, represents the j+1th largest edge; The mask of the adjacency matrix is ​​calculated by indexing, and the calculation formula is as follows: ; In the formula, Indicates The node and Whether there is a connection between the nodes, if there is a connection, it is 1, if there is no connection, it is 0; The mask of the adjacency matrix With The adjacency matrix of the frequency bands Perform Hadamard product operation to obtain sparse connection matrix ; According to the sparse connection matrix , aggregate and update the channel nodes to extract the topological features of the EEG signal. The calculation formula is as follows: ; In the formula, Indicates The topological characteristics of the frequency bands, represents the weight matrix; According to the topological characteristics of each frequency band, the graph characteristics of each frequency band are obtained ,in, They represent the topological characteristics of the first, second, third, fourth, and fifth frequency bands respectively.

[0011] Furthermore, the graph features of each frequency band are input into the graph feature fusion module for fusion to obtain the emotion features, including: The features of each frequency band Input to the graph feature fusion module to obtain the fused graph features : ; In the formula, express , represents the convolution operation, Indicates that a convolution operation is performed on G using a convolution kernel of size (5, 1); Fusion graph features Flattened into a one-dimensional vector, the emotional features are obtained .

[0012] Furthermore, the multi-layer perceptron is expressed as: ; In the formula, represents the emotion prediction probability output by the multi-layer perceptron, represents the weight matrix of the multilayer perceptron, represents the bias of the multi-layer perceptron, and F represents the emotional feature.

[0013] Furthermore, when the emotion recognition model is trained, a loss value of the emotion recognition model is calculated using a loss function, where the loss function includes domain adaptation loss, asymmetric loss, and cross entropy loss.

[0014] Furthermore, training the emotion recognition model includes: Construct training samples and test samples, and use the data in the training samples as source domain data , the data in the test sample is used as the target domain data , respectively input into the multi-branch graph convolution for the following operations: ; ; In the formula, represents the source domain characteristics, represents the target domain features, M represents the emotion feature extraction process; The maximum mean difference is used to calculate the source domain features and target domain features The difference between , expressed as: ; In the formula, Indicates the source domain i Sample features, Indicates the target domain j The characteristics of the samples, n represents the number of samples in the source domain, m represents the number of samples in the target domain, represents the Gaussian kernel function, represents the sample similarity in the source domain, represents the sample similarity in the target domain, Represents the differential entropy feature of the EEG signal; Represents the sample similarity between the source domain and the target domain. The k(.) formula can be expressed as: ; In the formula, represents the source domain sample, represents the target sample, represents the kernel width, express function; Calculate the connection strength of brain regions under negative emotions through degree center distribution: ; In the formula, Expressing negative emotions i The sum of the adjacent strengths of a channel with other channels, N Indicates the number of channels, The connection strength between the i-th node representing negative emotions and other nodes, The connection strength between other nodes representing negative emotions and the i-th node, The connection strength between the i-th node and itself, which represents negative emotions; Calculating the connection strength of brain regions under positive emotions through degree centrality distribution .

[0015] The difference between negative and positive emotions in brain regions is measured by the maximum mean difference, and an asymmetric loss is obtained. , expressed as: ; In the formula, Indicates i The connection strength of positive emotion samples, Expression j The connection strength of negative emotion samples, represents the number of positive samples, Indicates the number of negative samples.

[0016] The predicted emotion probability and the true emotion label obtained from the source domain are used as the input of the cross entropy to obtain the category loss , expressed as: ; In the formula, represents the number of classes, Indicates whether the sample belongs to category C. Indicates the predicted probability that the sample belongs to class C; The domain adaptation loss, asymmetric loss, and cross entropy loss are added together to calculate the total loss value of the model , expressed as: ; In the formula, and are two hyperparameters.

[0017] In a second aspect, the present application provides a cross-subject emotion recognition device based on EEG multi-branch graph convolution, the device comprising: a data acquisition module configured to acquire EEG data; A first filtering module, configured to filter the EEG data to obtain a first signal; A second filtering module is configured to filter the first signal in five frequency bands to obtain a second signal; An entropy feature extraction module is configured to extract differential entropy features in the second signal in a time window of a set time; A graph feature extraction module is configured to extract graph features of each frequency band through a multi-branch graph convolutional network based on the differential entropy feature; The feature fusion module is configured to input the graph features of each frequency band into the graph feature fusion module for fusion to obtain the emotion features; An emotion prediction module is configured to input the emotion feature into a multi-layer perceptron to obtain a predicted probability of the emotion; The emotion recognition module is configured to use a multi-branch graph convolutional network, a graph feature fusion module and a multi-layer perceptron connected in sequence as an emotion recognition model, train the emotion recognition model, and realize emotion recognition through the trained model.

[0018] In a third aspect, the present application provides a cross-subject emotion recognition system based on EEG multi-branch graph convolution, the system comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the method as described above.

[0019] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium storing instructions, which, when executed by a processor, executes the method described above.

[0020] This application has at least the following beneficial effects: 1. This application combines the asymmetric distribution of emotions in the left and right brains to construct a common graph structure model between different subjects, so that the difference in features extracted from the same type in the source domain and the target domain is reduced. This can effectively solve the needs of cross-subjects and enhance the generalization of the model.

[0021] 2. Considering the differences in connectivity of different frequency bands, the present invention extracts connectivity features of each frequency band through multiple graph convolutions, and fuses the connectivity features between frequency bands through a fusion module to learn more representative emotion features, thereby comprehensively exploring the relationship between EEG signal channels.

[0022] 3. This application proposes a graph feature fusion module that can fuse different graph features to generate more representative emotion features. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flowchart of a cross-subject emotion recognition method based on EEG multi-branch graph convolution according to an embodiment of the present application is shown.

[0024] Figure 2 A structural diagram of a multi-branch graph convolutional network according to an embodiment of the present application is shown, wherein 5 represents the number of frequency bands, 16 represents the number of channel features, and 62 represents the number of channels.

[0025] Figure 3 A structural diagram of a graph feature fusion module according to an embodiment of the present application is shown, wherein 5 represents the number of frequency bands, 16 represents the number of channel features, 62 represents the number of channels, and 32 represents the number of channels after fusion.

[0026] Figure 4 A schematic diagram of the training process of the emotion recognition model according to an embodiment of the present application is shown.

[0027] Figure 5 A structural diagram of a cross-subject emotion recognition device based on EEG multi-branch graph convolution according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0028] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0029] The specific implementation of the present application is further described in detail below in conjunction with the drawings and examples.

[0030] See also Figure 1 , which is a flow chart of a cross-subject emotion recognition method based on EEG multi-branch graph convolution. The cross-subject emotion recognition method based on EEG multi-branch graph convolution includes steps S1 to S8, which are described in detail as follows.

[0031] S1: Acquire EEG data.

[0032] In this embodiment, an EEG acquisition device is used to continuously monitor the voltage changes of brain electrical activities as EEG data.

[0033] S2: Filter the EEG data to obtain the first signal X .

[0034] S3: Filter the first signal in five frequency bands to obtain a second signal.

[0035] In this embodiment, the first signal X Five frequency bands ( ) is filtered to obtain the second signal Among them, the ranges of the five frequency bands are: (1-4 Hz), (4-7 Hz), (8-13 Hz), (13-30 Hz) , (>30 Hz). S4: In the second signal, extract the differential entropy feature in a time window of a set time.

[0036] In some embodiments, the second signal The differential entropy feature is extracted with a time window of 1s, and the feature extraction formula is: ; In the formula, Indicates Frequency band The differential entropy characteristics of the channels, and is the mean and variance of the EEG signal of the jth channel in the ith frequency band, is a natural constant, is the EEG signal of the jth channel in the i-th frequency band.

[0037] S5: Based on the differential entropy features, the graph features of each frequency band are extracted through a multi-branch graph convolutional network.

[0038] In some embodiments, the structure of the multi-branch graph convolutional network is as follows: Figure 2 As shown in FIG. 1 , the specific implementation steps of extracting the graph features of each frequency band through a multi-branch graph convolutional network include: S51. Since the connection relationship between EEG signal nodes is dynamically changing, the adjacency matrix is ​​calculated through the input signal. The adjacency matrix construction formula is as follows: ; In the formula, Indicates The adjacency matrix of the frequency bands, express The differential entropy characteristics of the frequency bands, and are two learnable weight matrices, represents the matrix transpose, express ,exp means function.

[0039] S52. Calculate the maximum edge of each node The index of the (smallest to largest)

[0040] ; In the formula, Indicates The front of the node edge weight A large index, Represents the edge set of the j-th node.

[0041] The function can be expressed as: ; In the formula, represents the index vector of the edge, represents k edges, represents the j-th largest edge index, represents the j+1th largest edge index, The ordinal number representing the edge index, represents the j-th largest edge, represents the j+1th largest edge.

[0042] S53, by index The mask of the adjacency matrix is ​​calculated as follows: ; Mask of the adjacency matrix The dimension size and is the same, among which Indicates The node and If there is a connection, the value is 1; if there is no connection, the value is 0.

[0043] S54, will and Perform Hadamard product operation to obtain sparse connection matrix , the formula can be expressed as: ; S55, according to the generated adjacency matrix , aggregate and update the channel nodes to extract the topological features of the EEG signal. The formula is as follows: ; In the formula, Indicates The topological characteristics of the frequency bands, represents the weight matrix.

[0044] Therefore, the input signal Through asymmetric graph convolution, we can get .

[0045] S6: Input the graph features of each frequency band into the graph feature fusion module for fusion to obtain the emotion features.

[0046] In some embodiments, the structure of the graph feature fusion module is as follows: Figure 3 The specific implementation steps of step S6 include: S61, input the graph features G of each frequency band into the graph feature fusion module to obtain the fused graph features : ; In the formula, express , represents the convolution operation, Indicates that a convolution operation is performed using a convolution kernel of size (5,1).

[0047] S62. Flatten the fusion graph features into a one-dimensional vector to obtain the final emotion features .

[0048] S7: Input the emotion features into the multi-layer perceptron to obtain the predicted probability of the emotion.

[0049] In step S7, the obtained emotion features are input into the multi-layer perceptron to obtain the predicted probability of emotion. The multi-layer perceptron can be expressed as: ; In the formula, represents the emotion prediction probability output by the multilayer perceptron, represents the weight matrix of the multilayer perceptron, represents the bias of the multi-layer perceptron, and F represents the emotional feature.

[0050] S8: Use the multi-branch graph convolutional network, graph feature fusion module and multi-layer perceptron connected in sequence as the emotion recognition model, train the emotion recognition model, and realize emotion recognition through the trained model.

[0051] In some embodiments, Figure 4As shown in FIG. 1 , the training process of the emotion recognition model is shown. In step S8, the loss value of the model is calculated by the loss function, and the loss function consists of three parts: domain adaptation loss, asymmetric loss, and cross entropy loss. The purpose of domain adaptation is to reduce the data differences between subjects. The asymmetric loss is used to constrain the graph convolution to maintain an asymmetric spatial pattern during the training process so that the graph convolution can construct a common graph pattern between subjects. The cross entropy is used to solve the classification problem of emotional features. Step S8 is specifically implemented by the following steps: S81. Use the data in the training sample as source domain data , the data in the test sample is used as the target domain data , respectively input into the multi-branch graph convolution: ; ; In the formula, represents the source domain characteristics, represents the target domain features, Represents the extraction process of emotional features.

[0052] S82. Calculate source domain features by using maximum mean difference (MMD) and target domain features The difference between , the formula can be expressed as: ; In the formula, Indicates the source domain i Sample features, Indicates the target domain j The characteristics of the samples, n represents the number of samples in the source domain, m represents the number of samples in the target domain, represents the Gaussian kernel function, represents the sample similarity in the source domain, represents the sample similarity in the target domain, Represents the differential entropy feature of the EEG signal.

[0053] Represents the sample similarity between the source domain and the target domain. The k(.) formula can be expressed as: ;

[0054] In the formula, represents the source domain sample, represents the target sample, represents the kernel width, express function.

[0055] S83. Calculating the connection strength of brain regions under negative emotions through degree center distribution : ;

[0056] In the formula, Expressing negative emotions i The sum of the connection strengths of a channel with other channels, N Indicates the number of channels, The connection strength between the i-th node representing negative emotions and other nodes, The connection strength between other nodes representing negative emotions and the i-th node, The connection strength between the i-th node representing negative emotions and itself, Adjacency matrix representing negative strength.

[0057] S84. Similarly, the connection strength of the brain region under positive emotions is calculated through the degree center distribution .

[0058] S85. Use MMD to measure the difference in brain regions between negative and positive emotions, thereby obtaining asymmetric loss , the formula can be expressed as: ;

[0059] In the formula, Indicates i The connection strength of positive emotion samples, Expression j The connection strength of negative emotion samples, represents the number of positive samples, Indicates the number of negative samples.

[0060] S86. Use the emotion prediction probability and the true emotion label obtained from the source domain as the input of the cross entropy to obtain the category loss , the formula can be expressed as: ;

[0061] In the formula, represents the number of classes, Indicates whether the sample belongs to category C. It represents the predicted probability that the sample belongs to class C.

[0062] S84. Calculate the total loss value of the model by adding the three loss values , the formula can be expressed as: ;

[0063] In the formula, and is a hyperparameter.

[0064] To summarize, the method provided in the embodiment of the present application is based on the original data collected by EEG. First, DE features are extracted through preprocessing, and then graph features of each frequency band are extracted through graph convolution. Then, the graph features of each frequency band are fused using a feature fusion module to learn more representative graph features, and finally, emotion prediction is performed through a classifier.

[0065] The present application also provides a cross-subject emotion recognition device based on EEG multi-branch graph convolution, such as Figure 5 As shown, the device comprises: A data acquisition module 501 is configured to acquire EEG data; A first filtering module 502, configured to filter the EEG data to obtain a first signal; A second filtering module 503 is configured to filter the first signal in five frequency bands to obtain a second signal; An entropy feature extraction module 504 is configured to extract differential entropy features in the second signal in a time window of a set time; A graph feature extraction module 505 is configured to extract graph features of each frequency band through a multi-branch graph convolutional network based on the differential entropy feature; The feature fusion module 506 is configured to input the graph features of each frequency band into the graph feature fusion module for fusion to obtain the emotion features; The emotion prediction module 507 is configured to input the emotion feature into a multi-layer perceptron to obtain a predicted probability of the emotion; The emotion recognition module 508 is configured to use a multi-branch graph convolutional network, a graph feature fusion module and a multi-layer perceptron connected in sequence as an emotion recognition model, train the emotion recognition model, and realize emotion recognition through the trained model.

[0066] It should be noted that the device described in this embodiment and the method described previously belong to the same technical concept, have the same technical principles, and can achieve the same beneficial effects, so they will not be repeated here.

[0067] An embodiment of the present application also provides a cross-subject emotion recognition system based on EEG multi-branch graph convolution, the system comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the method described in any of the above embodiments.

[0068] An embodiment of the present application further provides a non-temporary computer-readable storage medium storing instructions, and when the instructions are executed by a processor, the method described in any of the above embodiments is executed.

[0069] The above implementation modes are only used to illustrate the present application, and are not intended to limit the present application. Ordinary technicians in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present application. Therefore, all equivalent technical solutions also belong to the scope of the present application, and the scope of patent protection of the present application shall be limited by the claims.

Claims

1. A cross-subject emotion recognition method based on EEG multi-branch graph convolution, characterized in that: The method comprises: Acquire EEG data; Filtering the EEG data to obtain a first signal; Filtering the first signal in five frequency bands to obtain a second signal; In the second signal, extracting differential entropy features in a time window of a set time; Based on the differential entropy features, the graph features of each frequency band are extracted through a multi-branch graph convolutional network; Input the graph features of each frequency band into the graph feature fusion module for fusion to obtain the emotion features; Inputting the emotion features into a multi-layer perceptron to obtain the predicted probability of the emotion; A multi-branch graph convolutional network, a graph feature fusion module and a multi-layer perceptron connected in sequence are used as an emotion recognition model, the emotion recognition model is trained, and emotion recognition is achieved through the trained model.

2. The cross-subject emotion recognition method based on EEG multi-branch graph convolution as claimed in claim 1, characterized in that: In the second signal, the differential entropy feature is extracted in a time window of a set time, and the extraction formula is: ； In the formula, Indicates Frequency band The differential entropy characteristics of the channels, and It is Frequency band The mean and variance of the EEG signal of each channel, is a natural constant, For the Frequency band EEG signals of 1 channel.

3. The cross-subject emotion recognition method based on EEG multi-branch graph convolution as claimed in claim 1, characterized in that: Based on the differential entropy features, the graph features of each frequency band are extracted through a multi-branch graph convolutional network, including: The differential entropy feature is used as the input signal, and its adjacency matrix is ​​calculated based on the input signal. The adjacency matrix construction formula is as follows: ； In the formula, Indicates The adjacency matrix of the frequency bands, express The differential entropy characteristics of the frequency bands, and are two learnable weight matrices, represents the matrix transpose, express ,exp means function; Take the input signal and its adjacency matrix as nodes and calculate the maximum edge of each node. The index of is calculated as: ； In the formula, Indicates The front of the node edge weight A large index, represents the edge set of the jth node; The function is expressed as: ； In the formula, represents the index vector of the edge, indicates that there are k edges, represents the j-th largest edge index, represents the j+1th largest edge index, The ordinal number representing the edge index, represents the j-th largest edge, represents the j+1th largest edge; The mask of the adjacency matrix is ​​calculated by indexing, and the calculation formula is as follows: ； In the formula, Indicates The node and Whether there is a connection between the nodes, if there is a connection, it is 1, if there is no connection, it is 0; The mask of the adjacency matrix With The adjacency matrix of the frequency bands Perform Hadamard product operation to obtain sparse connection matrix ; According to the sparse connection matrix , aggregate and update the channel nodes to extract the topological features of the EEG signal. The calculation formula is as follows: ； In the formula, Indicates The topological characteristics of the frequency bands, represents the weight matrix; According to the topological characteristics of each frequency band, the graph characteristics of each frequency band are obtained ,in, They represent the topological characteristics of the first, second, third, fourth, and fifth frequency bands respectively.

4. The cross-subject emotion recognition method based on EEG multi-branch graph convolution as claimed in claim 1, characterized in that: The graph features of each frequency band are input into the graph feature fusion module for fusion to obtain the emotion features, including: The features of each frequency band Input to the graph feature fusion module to obtain the fused graph features : ； In the formula, express , represents the convolution operation, Indicates that a convolution operation is performed on G using a convolution kernel of size (5,1); Fusion graph features Flattened into a one-dimensional vector, the emotional features are obtained .

5. The cross-subject emotion recognition method based on EEG multi-branch graph convolution as claimed in claim 1, characterized in that: The multi-layer perceptron is represented as: ； In the formula, represents the emotion prediction probability output by the multilayer perceptron, represents the weight matrix of the multilayer perceptron, represents the bias of the multi-layer perceptron, and F represents the emotional feature.

6. The cross-subject emotion recognition method based on EEG multi-branch graph convolution as claimed in claim 1, characterized in that: When the emotion recognition model is trained, the loss value of the emotion recognition model is calculated by a loss function, where the loss function includes domain adaptation loss, asymmetric loss and cross entropy loss.

7. The cross-subject emotion recognition method based on EEG multi-branch graph convolution as claimed in claim 6, characterized in that: Training the emotion recognition model includes: Construct training samples and test samples, and use the data in the training samples as source domain data , the data in the test sample is used as the target domain data , respectively input into the multi-branch graph convolution for the following operations: ； ； In the formula, represents the source domain characteristics, represents the target domain features, Represents the extraction process of emotional features; The maximum mean difference is used to calculate the source domain features and target domain features The difference between , expressed as: ； In the formula, Indicates the source domain i Sample features, Indicates the target domain j The characteristics of the samples, n represents the number of samples in the source domain, m represents the number of samples in the target domain, represents the Gaussian kernel function, represents the sample similarity in the source domain, represents the sample similarity in the target domain, Represents the differential entropy feature of the EEG signal; Represents the sample similarity between the source domain and the target domain. The formula of k(.) can be expressed as: ； In the formula, represents the source domain sample, represents the target sample, represents the kernel width, express function; Calculate the connection strength of brain regions under negative emotions through degree center distribution: ； In the formula, Expressing negative emotions i The sum of the connection strengths of a channel with other channels, N Indicates the number of channels, The connection strength between the i-th node representing negative emotions and other nodes, The connection strength between other nodes representing negative emotions and the i-th node, The connection strength between the i-th node and itself, which represents negative emotions; Calculating the connection strength of brain regions under positive emotions through degree centrality distribution ; The difference between negative and positive emotions in brain regions is measured by the maximum mean difference, and an asymmetric loss is obtained. , expressed as: ； In the formula, Indicates i The connection strength of positive emotion samples, Expression j The connection strength of negative emotion samples, represents the number of positive samples, represents the number of negative samples; The predicted emotion probability and the true emotion label obtained from the source domain are used as the input of the cross entropy to obtain the category loss , expressed as: ； In the formula, represents the number of classes, Indicates whether the sample belongs to category C. Indicates the predicted probability that the sample belongs to class C; The domain adaptation loss, asymmetric loss, and cross entropy loss are added together to calculate the total loss value of the model , expressed as: ； In the formula, and are two hyperparameters.

8. A cross-subject emotion recognition device based on EEG multi-branch graph convolution, characterized in that: The device comprises: a data acquisition module configured to acquire EEG data; A first filtering module, configured to filter the EEG data to obtain a first signal; A second filtering module is configured to filter the first signal in five frequency bands to obtain a second signal; An entropy feature extraction module is configured to extract differential entropy features in the second signal in a time window of a set time; A graph feature extraction module is configured to extract graph features of each frequency band through a multi-branch graph convolutional network based on the differential entropy feature; The feature fusion module is configured to input the graph features of each frequency band into the graph feature fusion module for fusion to obtain the emotion features; An emotion prediction module is configured to input the emotion feature into a multi-layer perceptron to obtain a predicted probability of the emotion; The emotion recognition module is configured to use a multi-branch graph convolutional network, a graph feature fusion module and a multi-layer perceptron connected in sequence as an emotion recognition model, train the emotion recognition model, and realize emotion recognition through the trained model.

9. A cross-subject emotion recognition system based on EEG multi-branch graph convolution, characterized by: The system comprises: Memory for storing computer programs; A processor, configured to execute the computer program to implement the method according to any one of claims 1 to 7. 10 . A non-transitory computer-readable storage medium storing instructions, which, when executed by a processor, perform the method according to claim 1 .

Images