EEG emotion recognition system based on brain topology guided hierarchical hypergraph neural network

Abstract

The present application provides an EEG emotion recognition system based on a brain topology-guided hierarchical hypergraph neural network, comprising: an EEG sample processing module for representing EEG data as a sequence of N nodes, a hierarchical hypergraph construction module for constructing N hyperedges for the N nodes, and constructing a hypergraph based on the hyperedges; a brain topology-inspired hypergraph learning module for modeling a brain topology structure #imgabs0# through the associations between different brain regions, and for the first l Layer, based on the brain topology #imgabs1#, for each of the N nodes in the first l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l The node intermediate features of the layer, and the hypergraph convolution based on the association matrix and node intermediate features corresponding to the hypergraph, to obtain the node intermediate features of each node in the first l +1 layer node features, based on the Lth layer node features of each node, perform EEG emotion recognition.

Description

Technical Field

[0001] The present application relates to the field of neural network technology, and in particular to an EEG emotion recognition system based on a brain topology guided hierarchical hypergraph neural network. Background Art

[0002] Existing EEG emotion recognition methods oversimplify the relationship between EEG samples and learn the correlation between EEG channels without restriction, which affects the generalization of the model and limits its application in subject-independent scenarios. Summary of the invention

[0003] In view of this, an embodiment of the present application provides an EEG emotion recognition system based on a brain topology guided hierarchical hypergraph neural network to overcome the above problems or at least partially solve the above problems.

[0004] The first aspect of the embodiment of the present application provides an EEG emotion recognition system based on a brain topology guided hierarchical hypergraph neural network, the system comprising:

[0005] An EEG sample processing module, used for representing the EEG data as a sequence comprising N nodes, each node in the sequence corresponds to an EEG sample in the EEG data, and N is the total number of EEG samples contained in the EEG data;

[0006] A hierarchical hypergraph construction module, for constructing N hyperedges for N nodes, and constructing a hypergraph based on the hyperedges, wherein each hyperedge in the hypergraph connects one or more nodes among the N nodes, and multiple nodes connected to the same hyperedge belong to the same cluster;

[0007] A brain-topology-inspired hypergraph learning module for using , , , , Respectively represent the functional relationship between each EEG channel in the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe brain regions, using Represents the association between different brain regions and models the brain topology , for the first l Layers, based on the brain topology , for each of the N nodes in the l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l The node intermediate features of the layer, as well as the association matrix corresponding to the hypergraph and each node in the l The intermediate features of the nodes in the layer are hypergraph convolved to obtain the l+1 layer node features, based on the Lth layer node features of each node in the hypergraph learning network inspired by brain topology, for EEG emotion recognition.

[0008] Optionally, constructing N hyperedges for N nodes and constructing a hypergraph based on the hyperedges includes:

[0009] Hyperedge construction phase based on hierarchical clustering: each of the N nodes is initialized as a cluster to obtain a cluster set, and for each cluster in the cluster set, the two clusters with the closest distance are merged each time, and the cluster set is updated until the clustering termination condition is met, wherein the clustering termination condition is ρ=S1 / S2, ρ is a hyperparameter, S1 is the number of clusters contained in the current state, and S2 is the number of clusters contained in the initial state;

[0010] Heuristic hyperedge enhancement stage: when the number of clusters contained in the current state is less than N, determine the first node that is farthest from multiple centroids of the clusters contained in the current state in the current state, the farthest distance between the first node and the multiple centroids is d, initialize the first node as cluster T, traverse the N nodes based on the cluster T, merge the nodes in the N nodes that are less than d from the first node into the cluster T, until the number of clusters contained in the current state is N;

[0011] Convert the N clusters contained in the current state into N hyperedges, where each hyperedge connects one or more nodes contained in the cluster corresponding to the hyperedge;

[0012] According to the connection relationship between each hyperedge and the node, an association matrix is ​​constructed, and a hypergraph is generated based on the association matrix.

[0013] Optionally, the modeling obtains the brain topology ,include:

[0014] Given , , , , , ,in , , , , , respectively represent the weight matrices within the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe brain regions, a weight matrix representing the associations between different brain regions;

[0015] According to the connection relationship between the EEG channels, the weights between the EEG channels are updated to obtain an updated weight matrix , , , , in, , represents the updated weight of the i-th brain region in the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe, represents the weight of the i-th brain region before updating in the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe. The number of EEG channels is C. () indicates from the number Mapping of an EEG channel, The EEG channel set representing the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe brain regions;

[0016] Based on the updated weight matrix ,and , , , The weight matrix of the connections between different brain regions throughout the brain , building the topology of the brain ;

[0017] in, , , , , , are learnable parameters.

[0018] Optionally, for the lth layer in the hypergraph learning network inspired by brain topology, based on the brain topology structure, for each of the N nodes, l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l The intermediate features of the nodes of the layer include:

[0019] For the first l Layer, use represents the k-th convolution operation in the K-step convolution operation, based on the brain topology, for each of the N nodes in the first l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l The intermediate features of the nodes in the layer , where each node in l The intermediate features of the nodes in the layer are , Indicates that the i-th node among the N nodes is in l The node features of the layer, Cat[·] represents the connection operation, , where C is the number of EEG channels and F is the number of frequency bands;

[0020] The brain topology-inspired hypergraph learning module is also used to obtain the intermediate features of the nodes , projected to In space, project it onto The intermediate features of the nodes in space are projected onto space, and obtain the intermediate features of the nodes after deformation and projection of N nodes , is a specified dimension.

[0021] Optionally, the association matrix corresponding to the hypergraph and each node in the l The intermediate features of the nodes in the layer are hypergraph convolved to obtain the l +1 layer node features, including:

[0022] The intermediate features of the nodes after deformation and projection of the N nodes Perform hypergraph convolution, where the process of hypergraph convolution is: , is the weight matrix of the hyperedge, , represents the node degree matrix, represents the edge degree matrix, Indicates that each node l Hypergraph convolution result of the layer;

[0023] Through deformation and projection, the hypergraph convolution result of each node in the lth layer is Transform to space , as the node features of the l+1th layer, and input to the l+1th layer of the brain topology inspired hypergraph learning network.

[0024] Optionally, the loss function of the brain topology inspired hypergraph learning network is: , is a hyperparameter, is the cross entropy loss, Used to penalize the classification errors of emotion labels for EEG emotion recognition. Used to constrain associations between brain regions;

[0025] Cross Entropy Loss The calculation method is: , is the true emotion label corresponding to the EEG data, EEG emotion recognition results predicted by a hypergraph learning network inspired by brain topology for EEG data;

[0026] The calculation method is: , A weight matrix representing the associations between different brain regions.

[0027] The second aspect of the embodiment of the present application provides an EEG emotion recognition method based on a brain topology-guided hierarchical hypergraph neural network, which is applied to the EEG emotion recognition system based on a brain topology-guided hierarchical hypergraph neural network described in the first aspect, and the method includes:

[0028] By means of the EEG sample processing module, the EEG data to be identified is represented as a sequence comprising N nodes, each node in the sequence corresponds to an EEG sample to be identified in the EEG data to be identified, and N is the total number of EEG samples to be identified contained in the EEG data to be identified;

[0029] By using a hierarchical hypergraph construction module, N hyperedges are constructed for N nodes, and a hypergraph is constructed based on the hyperedges, wherein each hyperedge in the hypergraph connects one or more nodes among the N nodes, and multiple nodes connected to the same hyperedge belong to the same cluster;

[0030] Through the hypergraph learning module inspired by brain topology, using , , , , Respectively represent the functional relationship between each EEG channel in the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe brain regions, using Represents the association between different brain regions and models the brain topology , for the first l Layers, based on the brain topology , for each of the N nodes in the l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l The node intermediate features of the layer, as well as the association matrix corresponding to the hypergraph and each node in the l The intermediate features of the nodes in the layer are hypergraph convolved to obtain the l +1 layer node features, based on the Lth layer node features of each node in the hypergraph learning network inspired by brain topology, for EEG emotion recognition.

[0031] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method described in the second aspect.

[0032] In a fourth aspect of the embodiments of the present application, a computer program product is provided, including a computer program, which, when executed by a processor, implements the method described in the second aspect.

[0033] According to a fifth aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored, wherein when the computer program is executed by a processor, the method described in the second aspect is implemented.

[0034] Beneficial effects of this application:

[0035] The present application embodiment provides an EEG emotion recognition system based on a brain topology-guided hierarchical hypergraph neural network, the system comprising:

[0036] An EEG sample processing module, used for representing the EEG data as a sequence comprising N nodes, each node in the sequence corresponds to an EEG sample in the EEG data, and N is the total number of EEG samples contained in the EEG data;

[0037] A hierarchical hypergraph construction module, for constructing N hyperedges for N nodes, and constructing a hypergraph based on the hyperedges, wherein each hyperedge in the hypergraph connects one or more nodes among the N nodes, and multiple nodes connected to the same hyperedge belong to the same cluster;

[0038] Brain topology-inspired hypergraph learning module for using , , , , Respectively represent the functional relationship between each EEG channel in the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe brain regions, using Represents the association between different brain regions and models the brain topology , for the first l Layers, based on the brain topology , for each of the N nodes in the l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l The node intermediate features of the layer, as well as the association matrix corresponding to the hypergraph and each node in the l The intermediate features of the nodes in the layer are hypergraph convolved to obtain the l +1 layer node features, based on the Lth layer node features of each node in the hypergraph learning network inspired by brain topology, for EEG emotion recognition.

[0039] Through the EEG emotion recognition system based on brain topology guided hierarchical hypergraph neural network provided by the present application, the EEG sample processing module divides the collected EEG data of the subject and represents it as a sequence of N nodes, and then uses the hierarchical hypergraph construction module to construct hyperedges and hypergraphs for the N nodes in the sequence in turn to represent the association relationship between the N nodes, and then uses the hypergraph learning module inspired by brain topology to obtain the association between EEG channels in different brain regions to obtain the brain topology structure. According to each layer in the hypergraph learning network inspired by brain topology, K-step convolution operations are performed through the brain topology structure to obtain the node intermediate features of each node in each layer, and the node intermediate features of each node in each layer are hypergraph convolved with the association matrix of the hypergraph as the input of the next layer, until the output of the Lth layer, to obtain the node features used to represent the EEG data, and EEG emotion recognition is performed through the node features used to represent the EEG data.

[0040] It is possible to construct a hypergraph for the EEG data of subjects who need to perform emotion recognition, so as to obtain the correlation between each node, and then obtain the correlation between the EEG channels in different brain regions to obtain the brain topology. The N nodes are convolved with the hypergraph in combination with the brain topology. In this process, the intermediate features of the nodes are iterated in multiple layers in combination with the association matrix of the hypergraph, so as to obtain node features that can better represent the EEG data, thereby improving the accuracy of EEG emotion recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application.

[0042] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the description of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0043] Figure 1 is a schematic diagram of a different EEG emotion recognition method provided in the related art;

[0044] Figure 2 It is a schematic diagram of an EEG emotion recognition system based on a brain topology guided hierarchical hypergraph neural network provided by an embodiment of the present application;

[0045] Figure 3 It is a schematic diagram of a hierarchical hypergraph construction provided by an embodiment of the present application;

[0046] Figure 4 is a schematic diagram of EEG channels of different brain regions provided by an embodiment of the present application;

[0047] Figure 5 This is a flowchart of a brain topology-inspired hypergraph learning provided in one embodiment of the present application. DETAILED DESCRIPTION

[0048] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other.

[0049] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] Figure 1 is a schematic diagram of a different EEG emotion recognition method provided in the related art, such as Figure 1 shown.

[0051] As emotion recognition becomes increasingly important in human-computer interaction and brain-computer interfaces, computers need to be more intelligent in identifying human emotional states. Many previous studies have identified emotions from behavioral signals, including facial expressions in images, voice changes in audio, and sentiment analysis in text. These methods mainly rely on external manifestations of emotions. However, although behavioral signals are easy to obtain, they are inherently easy to manipulate and used for deception. In contrast, physiological signals such as electroencephalogram (EEG), electrocardiogram (ECG), and electromyography (EMG) provide more intrinsic and reliable measurements of an individual's true state because they are directly derived from body functions and processes. Thanks to its ability to directly reflect brain activity and its difficulty in falsification, EEG signals are increasingly used in emotion recognition.

[0052] In the past few decades, researchers have proposed many methods for EEG emotion recognition. One type of method builds a model to obtain the discriminative features of each EEG channel, such as learning single channel features from different brain hemispheres. Another type of method aims to extract more discriminative features by exploring the connections between multiple EEG channels, and it has received increasing attention, such as the dynamic graph convolutional neural network (DGCNN).

[0053] However, most existing methods still have the following three problems:

[0054] (1) The relationship between EEG samples is oversimplified. These methods treat samples individually or only focus on the correlation between pairs of samples, ignoring the complex connections that are crucial for accurate classification, including the complex interactions between samples and the distinction between outlier samples and normal samples. Figure 1 (b) and (d).

[0055] (2) The correlations of EEG channels are learned almost without restriction, and the learned topological structure is inconsistent with the findings of neuroscience, resulting in poor emotion recognition results. Figure 1 (c) in.

[0056] (3) The brain topology obtained from different subjects may be significantly different. If we only rely on the EEG channel relationship of each subject without incorporating some universal knowledge, the performance of the cross-subject emotion recognition task will be affected.

[0057] Based on the above problems, this application proposes an EEG emotion recognition system based on a brain topology guided hierarchical hypergraph neural network, which can more effectively capture the complex relationship between EEG samples and the correlation of EEG channels, thereby improving the accuracy of EEG emotion recognition.

[0058] Figure 2 is a schematic diagram of an EEG emotion recognition system based on a brain topology guided hierarchical hypergraph neural network provided by an embodiment of the present application, such as Figure 2 shown.

[0059] An embodiment of the present application provides an EEG emotion recognition system based on a brain topology-guided hierarchical hypergraph neural network, the system comprising:

[0060] An EEG sample processing module, used for representing the EEG data as a sequence comprising N nodes, each node in the sequence corresponds to an EEG sample in the EEG data, and N is the total number of EEG samples contained in the EEG data;

[0061] A hierarchical hypergraph construction module, for constructing N hyperedges for N nodes, and constructing a hypergraph based on the hyperedges, wherein each hyperedge in the hypergraph connects one or more nodes among the N nodes, and multiple nodes connected to the same hyperedge belong to the same cluster;

[0062] A brain-topology-inspired hypergraph learning module for using , , , , Respectively represent the functional relationship between each EEG channel in the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe brain regions, using Represents the association between different brain regions and models the brain topology , for the first l Layers, based on the brain topology , for each of the N nodes in the l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l The node intermediate features of the layer, as well as the association matrix corresponding to the hypergraph and each node in the l The intermediate features of the nodes in the layer are hypergraph convolved to obtain the l +1 layer node features, based on the Lth layer node features of each node in the hypergraph learning network inspired by brain topology, for EEG emotion recognition.

[0063] Specifically, the EEG emotion recognition system based on the brain topology guided hierarchical hypergraph neural network includes: an EEG sample processing module, a hierarchical hypergraph construction module and a brain topology inspired hypergraph learning module.

[0064] EEG data can be understood as the brain waves collected from the subjects in a preset time period (for example, 1 minute). After the EEG data is input into the EEG emotion recognition system based on the brain topology guided hierarchical hypergraph neural network, the EEG sample processing module is used to represent the EEG data as a sequence containing N nodes. For example, divide the EEG data Wie 1 minute of brain waves into 60 EEG samples, represents N EEG samples, Represents the dimension of EEG samples, that is, N*C*F. The total number of EEG samples contained in the EEG data is N. Each EEG sample contains data collected and calculated on C electrodes (for example, 62 electrodes), where one electrode is one EEG channel. C represents the number of EEG channels, and F represents the number of frequency bands.

[0065] Furthermore, through the hierarchical hypergraph construction module, N hyperedges are constructed for the N nodes in the sequence, and a hypergraph is constructed based on the N hyperedges. Each hyperedge in the hypergraph connects one or more nodes among the N nodes, and multiple nodes connected to the same hyperedge belong to the same cluster.

[0066] Finally, a hypergraph learning module inspired by brain topology is used to , , , , Respectively represent the functional relationship between each EEG channel (each EEG channel is an electrode) in the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe in the brain region, using Represents the association between different brain regions and models the brain topology , wherein the present application divides the scalp into 5 brain regions, namely the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe. Represents the frontal lobe, Represents the parietal lobe, Represents the occipital lobe, The left temporal lobe, Represents the right temporal lobe, represents the sparse connections between different brain regions throughout the brain, and according to , , , , , Modeling to obtain the topological structure of the brain .

[0067] Getting the topology of the brain Afterwards, for the first l Layers, through the brain topology , for each of the N nodes in the l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l The intermediate features of the nodes in the layer, at the same time, the association matrix corresponding to the hypergraph and each node in the l The intermediate features of the nodes in the layer are hypergraph convolved to obtain the l +1 layer of node features. Finally, after L layers of hypergraph convolution, EEG emotion recognition is performed based on the node features of the Lth layer finally output by each node in the hypergraph learning network inspired by brain topology.

[0068] Through the above embodiment, the collected EEG data of the subject is divided and represented as a sequence of N nodes through the EEG sample processing module, and then the hyperedges and hypergraphs are constructed in turn for the N nodes in the sequence through the hierarchical hypergraph construction module to represent the association relationship between the N nodes, and then the association between the EEG channels in different brain regions is obtained through the hypergraph learning module inspired by brain topology to obtain the brain topology structure. According to each layer in the hypergraph learning network inspired by brain topology, K-step convolution operations are performed through the brain topology structure to obtain the node intermediate features of each node in each layer, and the node intermediate features of each node in each layer are hypergraph convolved with the association matrix of the hypergraph as the input of the next layer, until the output of the Lth layer, to obtain the node features used to represent the EEG data, and EEG emotion recognition is performed through the node features used to represent the EEG data.

[0069] It is possible to construct a hypergraph for the EEG data of subjects who need to perform emotion recognition, so as to obtain the correlation between each node, and then obtain the correlation between the EEG channels in different brain regions to obtain the brain topology. The N nodes are convolved with the hypergraph in combination with the brain topology. In this process, the intermediate features of the nodes are iterated in multiple layers in combination with the association matrix of the hypergraph, so as to obtain node features that can better represent the EEG data, thereby improving the accuracy of EEG emotion recognition.

[0070] Optionally, constructing N hyperedges for N nodes and constructing a hypergraph based on the hyperedges includes:

[0071] Hyperedge construction phase based on hierarchical clustering: each of the N nodes is initialized as a cluster to obtain a cluster set, and for each cluster in the cluster set, the two clusters with the closest distance are merged each time, and the cluster set is updated until the clustering termination condition is met, wherein the clustering termination condition is ρ=S1 / S2, ρ is a hyperparameter, S1 is the number of clusters contained in the current state, and S2 is the number of clusters contained in the initial state;

[0072] Heuristic hyperedge enhancement stage: when the number of clusters contained in the current state is less than N, determine the first node that is farthest from multiple centroids of the clusters contained in the current state in the current state, the farthest distance between the first node and the multiple centroids is d, initialize the first node as cluster T, traverse the N nodes based on the cluster T, merge the nodes in the N nodes that are less than d from the first node into the cluster T, until the number of clusters contained in the current state is N;

[0073] Convert the N clusters contained in the current state into N hyperedges, where each hyperedge connects one or more nodes contained in the cluster corresponding to the hyperedge;

[0074] According to the connection relationship between each hyperedge and the node, an association matrix is ​​constructed, and a hypergraph is generated based on the association matrix.

[0075] Figure 3 is a schematic diagram of a hierarchical hypergraph construction provided by an embodiment of the present application, such as Figure 3 shown.

[0076] Specifically, in one embodiment, the specific steps of constructing a hyperedge and a hypergraph are as follows:

[0077] First, in Figure 3 In the paper, the hyperedge construction process includes two stages, namely the hyperedge construction stage based on hierarchical clustering and the heuristic hyperedge enhancement stage.

[0078] like Figure 3In the process of the hyperedge construction phase (HCHC) based on hierarchical clustering, each of the N nodes needs to be initialized as a cluster to obtain a cluster set, that is, the cluster set includes N clusters. For each cluster in the cluster set, the two clusters with the closest distance in the N clusters are merged each time, and after the merging, the cluster set is updated. After the merging, the number of clusters in the cluster set will decrease until the clustering termination condition is met, wherein the clustering termination condition is to satisfy ρ=S1 / S2, ρ is a hyperparameter, S1 is the number of clusters contained in the cluster set in the current state, and S2 is the number of clusters contained in the initial state. Before the merging, S1=S2. As the number of clusters decreases during the merging process, the value of S1 / S2 will become closer and closer to ρ. When the value of S1 / S2 is equal to ρ, it means that the hyperedge construction process in the hyperedge construction phase based on hierarchical clustering is completed. At this time, a cluster with a number of clusters not greater than the number of N nodes is obtained, and each cluster can cluster one or more nodes.

[0079] like Figure 3 In the process of heuristic hyperedge enhancement (HHE), each cluster has a centroid. When the number of clusters contained in the current state is less than N, the first node farthest from the multiple centroids of the multiple clusters obtained in the hyperedge construction phase based on hierarchical clustering is determined among the N nodes in the current state. The farthest distance between the first node and the multiple centroids is d, and the first node is initialized as cluster T. For cluster T, N nodes are traversed, and the nodes whose distance from the first node is less than d in the N nodes are merged into cluster T. After the merger, this process is repeated, and a new cluster is generated each time until the number of clusters contained in the current state is N, indicating that the heuristic hyperedge enhancement phase is completed.

[0080] Furthermore, the N clusters contained in the current state are converted into N hyperedges, each hyperedge connects one or more nodes contained in the cluster corresponding to the hyperedge, and an association matrix is ​​constructed according to the connection relationship between each hyperedge and the node, and a hypergraph is generated according to the association matrix.

[0081] As an example, Figure 3 , N is 10, indicating that there are 10 nodes in total, each node corresponds to an EEG sample, ρ=0.3, indicating that the clustering termination condition in the hyperedge construction stage based on hierarchical clustering is S1 / S2=0.3. First, each of the 10 nodes is initialized to 10 clusters, at this time S1=10, S2=10, and the position of each cluster is represented by the centroid. Each time, the two closest clusters in the 10 clusters are merged until only 3 clusters are left. At this time, S1 / S2=0.3, and the hyperedge construction process in the hyperedge construction stage based on hierarchical clustering is completed.

[0082] Then, for the three clusters obtained, each cluster contains some of the 10 nodes. An auxiliary circle is made for each node. The node farthest from the centroid of the three clusters is found as the first node. The distance between the first node and the centroid of the farthest cluster is d. A new cluster T is generated from the first node. The position of the first node of cluster T is the center of the circle and d is the radius. The nodes whose distance from the first node is less than d among the 10 nodes are merged into cluster T. At this time, 4 clusters are included. Among the 4 clusters, the node farthest from the centroid of the 4 clusters is found again as the first node... Repeat this process until the number of clusters is 10, and the heuristic hyperedge enhancement stage is completed.

[0083] Figure 4 is a schematic diagram of EEG channels of different brain regions provided by an embodiment of the present application, such as Figure 4 shown.

[0084] Optionally, the modeling obtains the brain topology ,include:

[0085] Given , , , , , ,in , , , , , respectively represent the weight matrices within the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe brain regions, a weight matrix representing the associations between different brain regions;

[0086] According to the connection relationship between the EEG channels, the weights between the EEG channels are updated to obtain an updated weight matrix , , , , in, , represents the updated weight of the i-th brain region in the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe, represents the weight of the i-th brain region before updating in the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe. The number of EEG channels is C. () indicates from the number Mapping of an EEG channel, Represents the EEG channel set of the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe;

[0087] Based on the updated weight matrix ,and , , , The weight matrix of the connections between different brain regions throughout the brain , building the topology of the brain ;

[0088] in, , , , , , are learnable parameters.

[0089] Specifically, in one embodiment, the brain topology is modeled The specific steps include:

[0090] First, given , , , , , ,in, , , , , Respectively represent the weight matrices within the five brain regions: frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe. A weight matrix representing the connections between different brain regions throughout the brain.

[0091] According to the connection relationship between each EEG channel, the weights between each EEG channel are updated to obtain the updated weight matrix , , , , ,in, , represents the updated weight of the i-th brain region in the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe, represents the weight of the i-th brain region before updating in the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe. The number of EEG channels is C. Indicates from the number Mapping of an EEG channel, Indicates from the number Mapping of an EEG channel, It represents the set of EEG channels of the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe. If there are edges between the electrodes of the corresponding channels, it means that there is a correlation between the functions of the brain regions corresponding to the electrodes. ,like and are all in the EEG channel set of the same brain region, then = ,otherwise =0.

[0092] Further, according to the updated weight matrix , , , , and a weight matrix representing the associations between different brain regions throughout the brain , building the topology of the brain .

[0093] The entire brain topology It can be modeled by the following formula:

[0094] + + + + + , where the ReLU function ensures that the values ​​in are non-negative.

[0095] Figure 5 is a flowchart of a brain topology-inspired hypergraph learning provided by an embodiment of the present application, such as Figure 5 shown.

[0096] Optionally, the first l layer, based on the brain topology, for each of the N nodes in the l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l The intermediate features of the nodes of the layer include:

[0097] For the first l Layer, use represents the k-th convolution operation in the K-step convolution operation, based on the brain topology, for each of the N nodes in the first l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l The intermediate features of the nodes in the layer , where each node in l The intermediate features of the nodes in the layer are , Indicates that the i-th node among the N nodes is in l The node features of the layer, Cat[·] represents the connection operation, , where C is the number of EEG channels and F is the number of frequency bands;

[0098] The brain topology-inspired hypergraph learning module is also used to obtain the intermediate features of the nodes , projected to In space, project it onto The intermediate features of the nodes in space are projected onto space, and obtain the intermediate features of the nodes after deformation and projection of N nodes , is a specified dimension.

[0099] Specifically, in one embodiment, Figure 5 ,The brain topology inspired hypergraph learning network contains a total of L layers, l Layer represents any layer in L layers. l The layer is to obtain the brain topology for each of the N nodes in the first l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l For the first layer of the hypergraph learning network inspired by brain topology, l Layer, you can use represents the kth convolution operation in the K-step convolution operation. Based on the brain topology, each of the N nodes is l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l The intermediate features of the nodes in the layer are , so as to perform hypergraph learning on the learned topology. In this process, each node l The intermediate features of the nodes in the layer can be expressed as: ,in, Indicates that the i-th node among the N nodes is in l The node features of the layer, Cat[·] represents the connection operation, , where C is the number of EEG channels and F is the number of frequency bands.

[0100] Optionally, the association matrix corresponding to the hypergraph and each node in the l The intermediate features of the nodes in the layer are hypergraph convolved to obtain the l +1 layer node features, including:

[0101] The node intermediate features after deformation and projection of the N nodes Perform hypergraph convolution, where the process of hypergraph convolution is: , is the weight matrix of the hyperedge, , represents the node degree matrix, represents the edge degree matrix, Indicates that each node l Hypergraph convolution result of the layer;

[0102] Through deformation and projection, the hypergraph convolution result of each node in the lth layer is Transform to space In, as the l +1 layer node features and input them into the first layer of the hypergraph learning network inspired by brain topology l +1 layer.

[0103] Specifically, in one embodiment, Figure 5 , when the corresponding association matrix based on the hypergraph and each node in the l After the hypergraph convolution is performed on the intermediate features of the nodes in the layer, it is necessary to further obtain the l +1 layer of node features, therefore, it is necessary to analyze the intermediate features of N nodes After deformation and projection, the node intermediate features are obtained Perform hypergraph convolution, that is, first Project to In space, , and then project it onto Space, get .

[0104] Among them, the process of hypergraph convolution is: , is the weight matrix of the hyperedge, , U is a matrix of dimension R2*R2, which is equivalent to a learnable fully connected layer with the same input and output dimensions. represents the node degree matrix, represents the edge degree matrix, Indicates that each node l Hypergraph convolution result of the layer.

[0105] Furthermore, through deformation and projection (i.e. Figure 5 T&R in the above example), the hypergraph convolution result of each node in the lth layer Transform to space In, as the l+ The node features of layer 1 are input into the hypergraph learning network inspired by brain topology. No.1+ 1 layer, the forward propagation process can be summarized as: , and repeat this process until the output of the Lth layer is obtained.

[0106] Optionally, the loss function of the brain topology inspired hypergraph learning network is: , is a hyperparameter, is the cross entropy loss, Used to penalize the classification errors of emotion labels for EEG emotion recognition. Used to constrain associations between brain regions;

[0107] Cross Entropy Loss The calculation method is: , is the true emotion label corresponding to the EEG data, EEG emotion recognition results predicted by a hypergraph learning network inspired by brain topology for EEG data;

[0108] The calculation method is: , A weight matrix representing the associations between different brain regions.

[0109] Specifically, in one embodiment, Figure 2 , the loss function of the brain topology inspired hypergraph learning network is: ,in, is a hyperparameter, is the cross entropy loss, Used to penalize the classification errors of emotion labels for EEG emotion recognition. Used to restrict the association between brain regions, cross entropy loss The calculation method is: ,in, is the true emotion label corresponding to the EEG data, EEG emotion recognition results predicted by a hypergraph learning network inspired by brain topology for EEG data. The calculation method is: ,in, A weight matrix representing the associations between different brain regions.

[0110] Based on the same inventive concept, another embodiment of the present application further provides an EEG emotion recognition method based on a brain topology guided hierarchical hypergraph neural network, which is applied to the EEG emotion recognition system based on a brain topology guided hierarchical hypergraph neural network described in the above embodiment, and the method includes:

[0111] Step S101, through the EEG sample processing module, the EEG data to be identified is represented as a sequence including N nodes, each node in the sequence corresponds to an EEG sample to be identified in the EEG data to be identified, and N is the total number of EEG samples to be identified contained in the EEG data to be identified;

[0112] Step S102, constructing N hyperedges for N nodes through a hierarchical hypergraph construction module, and constructing a hypergraph based on the hyperedges, wherein each hyperedge in the hypergraph connects one or more nodes among the N nodes, and multiple nodes connected to the same hyperedge belong to the same cluster;

[0113] Step S103, using the hypergraph learning module inspired by brain topology , , , , Respectively represent the functional relationship between each EEG channel in the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe brain regions, using Represents the association between different brain regions and models the brain topology , for the first l Layers, based on the brain topology , for each of the N nodes in the l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l The node intermediate features of the layer, as well as the association matrix corresponding to the hypergraph and each node in the l The intermediate features of the nodes in the layer are hypergraph convolved to obtain the l +1 layer node features, based on the Lth layer node features of each node in the hypergraph learning network inspired by brain topology, for EEG emotion recognition.

[0114] Optionally, constructing N hyperedges for N nodes and constructing a hypergraph based on the hyperedges includes:

[0115] Hyperedge construction phase based on hierarchical clustering: each of the N nodes is initialized as a cluster to obtain a cluster set, and for each cluster in the cluster set, the two clusters with the closest distance are merged each time, and the cluster set is updated until the clustering termination condition is met, wherein the clustering termination condition is ρ=S1 / S2, ρ is a hyperparameter, S1 is the number of clusters contained in the current state, and S2 is the number of clusters contained in the initial state;

[0116] Heuristic hyperedge enhancement stage: when the number of clusters contained in the current state is less than N, determine the first node that is farthest from multiple centroids of the clusters contained in the current state in the current state, the farthest distance between the first node and the multiple centroids is d, initialize the first node as cluster T, traverse the N nodes based on the cluster T, merge the nodes in the N nodes that are less than d from the first node into the cluster T, until the number of clusters contained in the current state is N;

[0117] Convert the N clusters contained in the current state into N hyperedges, where each hyperedge connects one or more nodes contained in the cluster corresponding to the hyperedge;

[0118] According to the connection relationship between each hyperedge and the node, an association matrix is ​​constructed, and a hypergraph is generated based on the association matrix.

[0119] Optionally, the modeling obtains the brain topology ,include:

[0120] Given , , , , , ,in , , , , , respectively represent the weight matrices within the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe brain regions, a weight matrix representing the associations between different brain regions;

[0121] According to the connection relationship between the EEG channels, the weights between the EEG channels are updated to obtain an updated weight matrix , , , , in, , represents the updated weight of the i-th brain region in the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe, represents the weight of the i-th brain region before updating in the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe. The number of EEG channels is C. () indicates from the number Mapping of an EEG channel, The EEG channel set representing the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe brain regions;

[0122] Based on the updated weight matrix ,and , , , The weight matrix of the connections between different brain regions throughout the brain , building the topology of the brain ;

[0123] in, , , , , , are learnable parameters.

[0124] Optionally, for the lth layer in the hypergraph learning network inspired by brain topology, based on the brain topology structure, for each of the N nodes, l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l The intermediate features of the nodes of the layer include:

[0125] For the first l Layer, use represents the k-th convolution operation in the K-step convolution operation, based on the brain topology, for each of the N nodes in the first l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l The intermediate features of the nodes in the layer , where each node in l The intermediate features of the nodes in the layer are , Indicates that the i-th node among the N nodes is in l The node features of the layer, Cat[·] represents the connection operation, , where C is the number of EEG channels and F is the number of frequency bands;

[0126] The brain topology-inspired hypergraph learning module is also used to obtain the intermediate features of the nodes , projected to In space, project it onto The intermediate features of the nodes in space are projected onto space, and obtain the intermediate features of the nodes after deformation and projection of N nodes , is a specified dimension.

[0127] Optionally, the association matrix corresponding to the hypergraph and each node in the l The intermediate features of the nodes in the layer are hypergraph convolved to obtain the l +1 layer node features, including:

[0128] The node intermediate features after deformation and projection of the N nodes Perform hypergraph convolution, where the process of hypergraph convolution is: , is the weight matrix of the hyperedge, , represents the node degree matrix, represents the edge degree matrix, Indicates that each node l Hypergraph convolution result of the layer;

[0129] Through deformation and projection, the hypergraph convolution result of each node in the lth layer is Transform to space , as the node features of the l+1th layer, and input to the l+1th layer of the brain topology inspired hypergraph learning network.

[0130] Optionally, the loss function of the brain topology inspired hypergraph learning network is: , is a hyperparameter, is the cross entropy loss, Used to penalize the classification errors of emotion labels for EEG emotion recognition. Used to constrain associations between brain regions;

[0131] Cross Entropy Loss The calculation method is: , is the true emotion label corresponding to the EEG data, EEG emotion recognition results predicted by a hypergraph learning network inspired by brain topology for EEG data;

[0132] The calculation method is: , A weight matrix representing the associations between different brain regions.

[0133] Based on the same inventive concept, another embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the EEG emotion recognition method based on brain topology guided hierarchical hypergraph neural network as described in any of the above embodiments.

[0134] Based on the same inventive concept, another embodiment of the present application also provides a computer program product, including a computer program, which is executed by a processor as the EEG emotion recognition method based on brain topology guided hierarchical hypergraph neural network as described in any of the above embodiments.

[0135] Based on the same inventive concept, another embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored, wherein when the program is executed by a processor, the EEG emotion recognition method based on the brain topology guided hierarchical hypergraph neural network as described in any of the above embodiments is implemented.

[0136] As for the device, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0137] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0138] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0139] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0140] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0141] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0142] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0143] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0144] The above is a detailed introduction to the EEG emotion recognition system based on brain topology guided hierarchical hypergraph neural network provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An EEG emotion recognition system based on brain topology guided hierarchical hypergraph neural network, characterized in that: The system comprises: An EEG sample processing module, used for representing the EEG data as a sequence comprising N nodes, each node in the sequence corresponds to an EEG sample in the EEG data, and N is the total number of EEG samples contained in the EEG data; A hierarchical hypergraph construction module, for constructing N hyperedges for N nodes, and constructing a hypergraph based on the hyperedges, wherein each hyperedge in the hypergraph connects one or more nodes among the N nodes, and multiple nodes connected to the same hyperedge belong to the same cluster; Brain topology-inspired hypergraph learning module for using , , , , Respectively represent the functional relationship between each EEG channel in the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe brain regions, using Represents the association between different brain regions and models the brain topology , for the first l Layers, based on the brain topology , for each of the N nodes in the l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l The node intermediate features of the layer, as well as the association matrix corresponding to the hypergraph and each node in the l The intermediate features of the nodes in the layer are hypergraph convolved to obtain the l +1 layer node features, based on the node features of each node in the Lth layer of the brain topology inspired hypergraph learning network, for EEG emotion recognition; Wherein, for the first layer in the hypergraph learning network inspired by brain topology, based on the brain topology structure, for each of the N nodes, l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l The intermediate features of the nodes in the layer include: l Layer, use represents the k-th convolution operation in the K-step convolution operation, based on the brain topology, for each of the N nodes in the first l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l The intermediate features of the nodes in the layer , where each node in l The intermediate features of the nodes in the layer are , Indicates that the i-th node among the N nodes is in l The node features of the layer, Cat[·] represents the connection operation, , where C is the number of EEG channels and F is the number of frequency bands; the brain topology-inspired hypergraph learning module is also used to obtain the intermediate features of the nodes , projected to In space, project it onto The intermediate features of the nodes in space are projected onto space, and obtain the intermediate features of the nodes after deformation and projection of N nodes , is a specified dimension. The hypergraph-based association matrix and each node in the l The intermediate features of the nodes in the layer are hypergraph convolved to obtain the l +1 layer of node features, including: intermediate node features of the N nodes after deformation and projection Perform hypergraph convolution, where the process of hypergraph convolution is: , is the weight matrix of the hyperedge, , represents the node degree matrix, represents the edge degree matrix, Indicates that each node l The hypergraph convolution result of the layer; through deformation and projection, the hypergraph convolution result of each node in the lth layer Transform to space , as the node features of the l+1th layer, and input to the l+1th layer of the brain topology inspired hypergraph learning network.

2. The EEG emotion recognition system based on brain topology guided hierarchical hypergraph neural network according to claim 1 is characterized in that: The step of constructing N hyperedges for N nodes and constructing a hypergraph based on the hyperedges includes: Hyperedge construction phase based on hierarchical clustering: each of the N nodes is initialized as a cluster to obtain a cluster set, and for each cluster in the cluster set, the two clusters with the closest distance are merged each time, and the cluster set is updated until the clustering termination condition is met, wherein the clustering termination condition is ρ=S1 / S2, ρ is a hyperparameter, S1 is the number of clusters contained in the current state, and S2 is the number of clusters contained in the initial state; Heuristic hyperedge enhancement stage: when the number of clusters contained in the current state is less than N, determine the first node that is farthest from multiple centroids of the clusters contained in the current state in the current state, the farthest distance between the first node and the multiple centroids is d, initialize the first node as cluster T, traverse the N nodes based on the cluster T, merge the nodes in the N nodes that are less than d from the first node into the cluster T, until the number of clusters contained in the current state is N; Convert the N clusters contained in the current state into N hyperedges, where each hyperedge connects one or more nodes contained in the cluster corresponding to the hyperedge; According to the connection relationship between each hyperedge and the node, an association matrix is ​​constructed, and a hypergraph is generated based on the association matrix.

3. The EEG emotion recognition system based on brain topology guided hierarchical hypergraph neural network according to claim 1 is characterized in that: The modeling obtains the brain topology ,include: Given , , , , , ,in , , , , , respectively represent the weight matrices within the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe brain regions, a weight matrix representing the associations between different brain regions; According to the connection relationship between the EEG channels, the weights between the EEG channels are updated to obtain an updated weight matrix , , , , in, , represents the updated weight of the i-th brain region in the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe, represents the weight of the i-th brain region before updating in the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe. The number of EEG channels is C. () indicates from the number Mapping of an EEG channel, The EEG channel set representing the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe brain regions; Based on the updated weight matrix ,and , , , The weight matrix of the connections between different brain regions throughout the brain , building the topology of the brain ; in, , , , , , are learnable parameters.

4. The EEG emotion recognition system based on brain topology guided hierarchical hypergraph neural network according to claim 3 is characterized in that: The loss function of the brain topology inspired hypergraph learning network is: , is a hyperparameter, is the cross entropy loss, Used to penalize the classification errors of emotion labels for EEG emotion recognition. Used to constrain associations between brain regions; Cross Entropy Loss The calculation method is: , is the true emotion label corresponding to the EEG data, EEG emotion recognition results predicted by a hypergraph learning network inspired by brain topology for EEG data; The calculation method is: , A weight matrix representing the associations between different brain regions.

5. An EEG emotion recognition method based on brain topology guided hierarchical hypergraph neural network, characterized in that: The method applied to the EEG emotion recognition system based on brain topology guided hierarchical hypergraph neural network as described in any one of claims 1 to 4 comprises: By means of the EEG sample processing module, the EEG data to be identified is represented as a sequence comprising N nodes, each node in the sequence corresponds to an EEG sample to be identified in the EEG data to be identified, and N is the total number of EEG samples to be identified contained in the EEG data to be identified; By using a hierarchical hypergraph construction module, N hyperedges are constructed for N nodes, and a hypergraph is constructed based on the hyperedges, wherein each hyperedge in the hypergraph connects one or more nodes among the N nodes, and multiple nodes connected to the same hyperedge belong to the same cluster; Through the hypergraph learning module inspired by brain topology, using , , , , Respectively represent the functional relationship between each EEG channel in the frontal lobe, parietal lobe, occipital lobe, left temporal lobe, and right temporal lobe brain regions, using Represents the association between different brain regions and models the brain topology , for the first l Layers, based on the brain topology , for each of the N nodes in the l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l The node intermediate features of the layer, as well as the association matrix corresponding to the hypergraph and each node in the l The intermediate features of the nodes in the layer are hypergraph convolved to obtain the l +1 layer node features, based on the node features of each node in the Lth layer of the brain topology inspired hypergraph learning network, for EEG emotion recognition; Wherein, for the first layer in the hypergraph learning network inspired by brain topology, based on the brain topology structure, for each of the N nodes, l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l The intermediate features of the nodes in the layer include: l Layer, use represents the k-th convolution operation in the K-step convolution operation, based on the brain topology, for each of the N nodes in the first l The node features of the layer are subjected to K-step convolution operations to obtain the node features of each node in the first l The intermediate features of the nodes in the layer , where each node in l The intermediate features of the nodes in the layer are , Indicates that the i-th node among the N nodes is in l The node features of the layer, Cat[·] represents the connection operation, , where C is the number of EEG channels and F is the number of frequency bands; the brain topology-inspired hypergraph learning module is also used to obtain the intermediate features of the nodes , projected to In space, project it onto The intermediate features of the nodes in space are projected onto space, and obtain the intermediate features of the nodes after deformation and projection of N nodes , is a specified dimension. The hypergraph-based association matrix and each node in the l The intermediate features of the nodes in the layer are hypergraph convolved to obtain the l +1 layer of node features, including: intermediate node features of the N nodes after deformation and projection Perform hypergraph convolution, where the process of hypergraph convolution is: , is the weight matrix of the hyperedge, , represents the node degree matrix, represents the edge degree matrix, Indicates that each node l The hypergraph convolution result of the layer; through deformation and projection, the hypergraph convolution result of each node in the lth layer Transform to space , as the node features of the l+1th layer, and input to the l+1th layer of the brain topology inspired hypergraph learning network.

6. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the EEG emotion recognition method based on brain topology guided hierarchical hypergraph neural network as described in claim 5.

7. A computer-readable storage medium, characterized in that: A computer program is stored thereon, wherein when the computer program is executed by a processor, the EEG emotion recognition method based on brain topology guided hierarchical hypergraph neural network as described in claim 5 is implemented.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the EEG emotion recognition method based on brain topology guided hierarchical hypergraph neural network as described in claim 5.

Images