Emotion recognition and interpretation method and device based on event-driven electroencephalogram pulse diagram

By constructing the initial EEG topology map and adaptive graph convolution network, dynamically adjusting the edge weights, the problem of failure to effectively model the topology relationship of EEG signals in traditional methods is solved, and accurate evaluation and physiological interpretability of emotion recognition are achieved, which is suitable for real-time monitoring and clinical diagnosis.

CN120561736APending Publication Date: 2025-08-29BEIJING INST OF TECH
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202510701279.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-05-28
Publication Date
2025-08-29

AI Technical Summary

Technical Problem

Traditional emotion recognition methods fail to effectively model the multi-channel spatial topological relationship of EEG signals, ignore the functional coupling information between the leads, and lack physiological interpretability and adaptive learning ability, making it difficult to meet clinical diagnostic needs.

Method used

Using an event-driven EEG pulse diagram method, by constructing an initial EEG topology map, a graph convolution neural network and pulse neural network that is perceived by adaptive spatial information is dynamically adjusted, edge weights are captured, emotional functional connection patterns are introduced, and a physiological interpretability analysis mechanism is introduced.

Benefits of technology

It realizes objective and accurate assessment of the subject's emotional state, is suitable for real-time monitoring, has physiological interpretability and adaptive learning ability, and enhances the credibility of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120561736A_ABST
    Figure CN120561736A_ABST
Patent Text Reader

Abstract

The invention discloses an event-driven electroencephalogram pulse diagram-based emotion recognition and interpretation method and device, and belongs to the field of artificial intelligence and biomedical engineering cross technologies, and the method comprises the following steps: S1, determining coordinates of electroencephalogram leads in a three-dimensional space, and establishing an initial electroencephalogram topological graph; s2, forming complete electroencephalogram pulse diagram data through training; s3, processing the electroencephalogram pulse diagram data from the S2 to obtain fusion features; s4, performing pulse time sequence dependency modeling on the fusion features, and using the model to complete classification decision; and S5, verifying the physiological rationality of the classification decision. According to the emotion recognition and interpretation method and device based on the event-driven electroencephalogram pulse diagram, brain dynamic characteristics are accurately captured, objective and accurate evaluation and decision making of the emotional state of a testee are achieved, a physiological interpretability analysis mechanism is introduced, and the reliability of a model is verified from the perspective of neurophysiology.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of intersection of artificial intelligence and biomedical engineering, and specifically relates to a method and device for emotion recognition and interpretation based on event-driven electroencephalogram (EEG). Background Art

[0002] Emotion recognition is a core technology in the fields of brain-computer interfaces and mental health monitoring. Traditional emotion recognition methods primarily fall into two categories: one is based on a combination of manual feature extraction and machine learning algorithms, such as support vector machines and random forests, which use manually designed feature parameters to classify emotions; the other is the use of deep neural networks, such as convolutional neural networks and recurrent neural networks, to achieve end-to-end emotion recognition.

[0003] However, traditional approaches have shortcomings: First, the multi-channel spatial topological relationships of EEG signals are ignored, and the functional coupling between leads is not effectively modeled; second, the continuous computational mechanism of traditional neural networks does not align with the pulse coding characteristics of biological neurons, resulting in low model energy efficiency and a lack of physiological interpretability; third, existing technologies often rely on static graph structures or fixed connection patterns, making them unable to adaptively learn the dynamic brain network characteristics related to emotions. Furthermore, most models lack neuroscientific interpretability of their classification decisions, making them difficult to meet the trustworthy requirements of clinical diagnosis. Therefore, a new approach is urgently needed. Summary of the Invention

[0004] The purpose of the present invention is to provide an emotion recognition and interpretation method and device based on event-driven EEG pulse graphs. By deeply fusing the spatiotemporal information of EEG signals, the dynamic characteristics of the brain are accurately captured, a complete brain activity state map is constructed, and objective and accurate assessment and decision-making of the subject's emotional state are achieved. A physiological interpretability analysis mechanism is introduced to verify the reliability of the model from a neurophysiological perspective.

[0005] To achieve the above objectives, the present invention provides an emotion recognition and interpretation method based on event-driven EEG pulse graphs, comprising the following steps:

[0006] S1. Based on the international 10-20 system and the extended 10-10 system standards, the coordinates of the physiological positions of the EEG leads in three-dimensional space are determined, and the Euclidean distances between the EEG leads are calculated based on the coordinates to construct an initial edge weight matrix, forming an undirected weighted graph with the leads as nodes and the Euclidean distances as edges, thereby obtaining an initial EEG topology map;

[0007] S2, obtain the initial EEG topology map and original EEG signals in S1, perform multi-level time slicing on the original EEG signals, and then convert the original brain signals into pulse trains. The spatiotemporal features are dynamically mapped to graph nodes through a trainable pulse encoder to obtain complete EEG pulse map data;

[0008] S3. Obtain the EEG pulse graph data in S2, and use the graph convolutional neural network with adaptive spatial information perception to hierarchically fuse the spatial correlation and pulse timing information between nodes. Dynamically adjust the edge weights through the adaptive edge weight optimization module to capture the emotional functional connection pattern, thereby achieving dynamic feature aggregation and obtaining the fused features after graph convolution processing;

[0009] S4: Modeling the pulse timing dependency of the fused features in S3 through a spiking neural network, simulating the information processing process of the biological nervous system, and generating a probability distribution of the emotional state based on the pulse firing rate statistics, completing the classification decision and obtaining the predicted result of the emotional state;

[0010] S5. By comparing the functional connectivity features learned by the model with the phase-locking values ​​calculated under different emotional states, the physiological rationality of the classification decision in S4 is verified.

[0011] Preferably, S1 comprises the following steps:

[0012] S101, based on the international 10-20 system and the extended 10-10 system standards, determine the coordinates of each lead in three-dimensional space according to anatomical landmarks to ensure that the electrode layout corresponds to the functional divisions of the cerebral cortex;

[0013] S102, using the preset electrode mapping relationship of the standardized brain model, the lead p i The position is converted into three-dimensional coordinates in the Cartesian coordinate system (p ix ,p iy ,p iz ), define an undirected weighted graph, expressed as:

[0014] G=(V,E,W,F);

[0015] Among them, V is the node set corresponding to the EEG lead, E is the edge set, the edge set is determined by the weight matrix W, and F is the node feature set. Initialized as pulse-coded time series data, d is the feature dimension, N is the number of nodes, and the weight matrix W is used to calculate the spatial proximity between leads based on the three-dimensional Euclidean distance. The calculation formula is:

[0016]

[0017] Among them, d ij For node p i With node p j The Euclidean distance between ix ,p iy ,p iz are the x, y, and z coordinates of the i-th lead in three-dimensional space, and p jx ,pjy ,p jz are the x, y, and z coordinates of the jth lead in three-dimensional space;

[0018] According to d ij The inverse relationship between the adjustment node p i and node p j The edge weight w between ij , defined as:

[0019]

[0020] Where k is the calibration constant;

[0021] When initializing the weight matrix, insignificant connections are removed. The calculation formula is as follows:

[0022]

[0023] Among them, θ is the distance threshold;

[0024] S103: Perform symmetric normalization on the weight matrix W. The calculation formula is as follows:

[0025]

[0026] Where D is the degree matrix D ii =∑ j w ij Used to eliminate node degree distribution bias.

[0027] Preferably, S2 comprises the following steps:

[0028] S201, dividing the original EEG signal into continuous segments of fixed duration to obtain first-level time segments, and further dividing the first-level time segments into T time steps to obtain second-level time segments;

[0029] S202. The LIF neuron model is used to simulate biological neuron behavior, including charge accumulation and leakage. When the membrane potential reaches the threshold, an action potential is generated and the membrane potential is reset, which is expressed as:

[0030] V(t+1)=k decay V(t)[1-H(V(t)-V th )]+I t+1 (p);

[0031] Where V(t) is the membrane potential at time step t, k decay is the hyperparameter attenuation factor, V th is the pulse emission threshold, when V(t)≥V th When the pulse is triggered and the membrane potential is reset, H(·) is the Heaviside step function, It+1 (p) is the input function;

[0032] S203. Use a trainable encoder built based on LIF neurons. The formula is as follows:

[0033]

[0034] in, is the cumulative membrane potential at time step t, To input EEG signals, is the potential coding sequence, W Encoder is the learnable encoder matrix.

[0035] Preferably, S3 also includes setting W * is the optimal weight matrix obtained through model training. The graph convolution process is expressed as:

[0036] y=g(L * )x=U * g(Λ * )U *T x;

[0037] Among them, x is the image signal, Λ * By W * The diagonal matrix composed of the eigenvalues ​​of g(L * ) is the Laplace matrix L acting on the graph * The filter function, U * is the eigenvector matrix, U *T For U * The transpose of L * is the normalized graph Laplace matrix, through the degree matrix D * and W * The adjacency matrix is ​​calculated using the following formula:

[0038] L * =D * -W * ;

[0039] Among them, D * is a diagonal matrix, and the Chebyshev polynomial is used to approximate the filter function g(·). The calculation formula is as follows:

[0040]

[0041] in, represents the normalized Λ * ,θ k are Chebyshev polynomial coefficients, T k is calculated as follows:

[0042]

[0043] Among them, T k (·) is a Chebyshev polynomial of order k, and the final graph convolution formula is:

[0044]

[0045] in, are the eigenvalues ​​of the normalized graph Laplacian matrix.

[0046] Preferably, in S4, a sigmoid function is used to approximate the pulse generation process, and the calculation method is as follows:

[0047]

[0048] And by calculating the derivative of the sigmoid function as an alternative gradient, the calculation method is as follows:

[0049]

[0050] Among them, α is a parameter.

[0051] The present invention also provides an emotion recognition and interpretation device based on event-driven EEG pulse graph, comprising the following modules:

[0052] A topology construction module is used to determine the coordinates of the physiological positions of EEG leads in three-dimensional space according to the international 10-20 system and the extended 10-10 system standards, and to calculate the Euclidean distance between EEG leads based on the coordinates to construct an initial edge weight matrix, forming an undirected weighted graph with leads as nodes and Euclidean distances as edges, thereby obtaining an initial EEG topology map;

[0053] A pulse encoding module, connected to the topology construction module, is used to perform multi-level time slicing processing on the original EEG signal, and then convert the original brain signal into a pulse sequence. By training the pulse encoder, the spatiotemporal features are dynamically mapped to the graph nodes to obtain complete EEG pulse graph data;

[0054] The pulse graph convolution module is connected to the pulse encoding module and uses a graph convolutional neural network with adaptive spatial information perception to hierarchically fuse the spatial correlation between nodes and the pulse timing information. The adaptive edge weight optimization module dynamically adjusts the edge weights to capture the emotional functional connection pattern, thereby achieving dynamic feature aggregation and obtaining fused features after graph convolution processing.

[0055] A spiking neural network decision module, connected to the spiking graph convolution module, models the pulse timing dependency of the fused features through a spiking neural network, simulates the information processing process of the biological nervous system, and generates a probability distribution of the emotional state based on the statistical results of the pulse firing rate, completes the classification decision, and obtains the prediction result of the emotional state;

[0056] The interpretability analysis module is connected to the pulse neural network decision module, and verifies the physiological rationality of the classification decision by comparing the functional connectivity features learned by the model with the phase locking value differences calculated under different emotional states.

[0057] Therefore, the present invention adopts the above-mentioned emotion recognition and interpretation method and device based on event-driven EEG pulse graph. Compared with the existing technology, the present invention has the following significant beneficial effects:

[0058] (1) The present invention overcomes the limitation of traditional methods that rely on predefined fixed graph structures by introducing a spatial topology modeling method that can be dynamically adjusted;

[0059] (2) The present invention adopts a mechanism that matches the pulse coding characteristics of biological neurons, replacing the traditional continuous activation mechanism, making the model more suitable for mobile terminal deployment requirements, especially for the field of brain-computer interface that requires real-time monitoring;

[0060] (3) The present invention integrates the advantages of graph convolutional networks and spiking neural networks to achieve collaborative optimization of spatial topology aggregation and spike timing modeling;

[0061] (4) The present invention establishes an explanatory mechanism that interfaces with neuroscience theory, allowing the brain network coupling basis behind classification decisions to be verified through comparative analysis between edge weight matrices and physiological indicators.

[0062] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 This is a flow chart of the emotion recognition and interpretation method based on event-driven EEG pulse graph of the present invention;

[0064] Figure 2 This is a LIF neuron workflow diagram of the present invention's method for emotion recognition and interpretation based on event-driven EEG pulse graphs;

[0065] Figure 3 This is a feature fusion framework diagram in the pulse graph convolution module of the emotion recognition and interpretation method based on event-driven EEG pulse graphs of the present invention. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used in the present invention should be the common meanings understood by people with ordinary skills in the field to which the present invention belongs.

[0067] Example 1

[0068] like Figure 1-Figure 3 As shown, the emotion recognition and interpretation method based on event-driven EEG pulse graph of the present invention includes the following steps:

[0069] S1, topology construction module, based on the physiological position of EEG leads, constructs an undirected weighted graph with leads as nodes and Euclidean distance as edges, and obtains the initial EEG topology map, specifically:

[0070] S101, based on the international 10-20 system and the extended 10-10 system standards, strictly follows anatomical landmarks to determine the coordinates of each lead in three-dimensional space, ensuring the correspondence between electrode layout and functional divisions of the cerebral cortex;

[0071] S102, using the preset electrode mapping relationship of the standardized brain model, the lead p i The position is converted into three-dimensional coordinates in the Cartesian coordinate system (p ix ,p iy ,p iz ), for example, the central lead point is fixed at the vertex of the midsagittal plane on the top of the head, and the other leads are distributed symmetrically according to the curvature of the skull. This process ensures the consistency of the spatial position of electrodes between different subjects through strict anatomical calibration, providing a scientific basis for the subsequent construction of physiologically driven EEG topology maps. Define an undirected weighted graph, expressed as:

[0072] G=(V,E,W,F);

[0073] Among them, V is the node set corresponding to the EEG lead, E is the edge set, the edge set is determined by the weight matrix W, and F is the node feature set. Initialized as pulse-coded time series data, d is the feature dimension, N is the number of nodes, and the weight matrix W is used to calculate the spatial proximity between leads based on the three-dimensional Euclidean distance. The calculation formula is:

[0074]

[0075] Among them, d ij For node p iWith node p j The Euclidean distance between ix ,p iy ,p iz are the x, y, and z coordinates of the i-th lead in three-dimensional space, and p jx ,p jy ,p jz are the x, y, and z coordinates of the j-th lead in three-dimensional space.

[0076] According to d ij The inverse relationship between the adjustment node p i and node p j The edge weight w between ij , defined as:

[0077]

[0078] Where k is the calibration constant.

[0079] In order to ensure higher connection strength between adjacent leads and reduce the synchronization relationship between leads that are far apart, insignificant connections are removed when initializing the weight matrix to improve computational efficiency and focus features. The calculation formula is as follows:

[0080]

[0081] Among them, θ is the distance threshold, which determines which connections between leads are considered significant, thereby optimizing the construction of the edge weight matrix in the initial stage;

[0082] S103: Perform symmetric normalization on the weight matrix W to obtain the final adjacency matrix. The calculation formula is as follows:

[0083]

[0084] Where D is the degree matrix D ii =∑ j w ij Used to eliminate node degree distribution deviation;

[0085] S2: Trainable Leaky Integrate-and-Fire (LIF) neurons encode the original continuous EEG digital signal into discrete pulse signals, and map the corresponding pulse signals to the corresponding lead nodes. The pulse signals of all leads at each time step constitute the node feature set F of the undirected weighted graph G in S1 corresponding to the current time step. Specifically, the following steps are included:

[0086] S201. Time segmentation processing. The first-level time segment is to divide the original EEG signal into continuous segments of fixed duration to expand the data set. This segment is the minimum segment for input to the model. The second-level time segment is to split the first-level time segment into T time steps again, and pulse encoding is performed on each time step in sequence. The second-level time segment is used for multi-time step membrane potential accumulation training within the model.

[0087] S202. Use the LIF neuron model to simulate the pulse coding mechanism of biological neurons. Use the LIF neuron model to simulate the behavior of biological neurons, including charge accumulation and leakage. When the membrane potential reaches the threshold, an action potential is generated and the membrane potential is reset, expressed as:

[0088] V(t+1)=k decay V(t)[1-H(V(t)-V th )]+I t+1 (p);

[0089] Where V(t) is the membrane potential at time step t, k decay is the hyperparameter attenuation factor, V th is the pulse emission threshold, when V(t)≥V th When the pulse is triggered and the membrane potential is reset, H(·) is the Heaviside step function, when x<0, H(x)=0, when x≥0, H(x)=1, I t+1 (p) is the input function used to convert and process the input spike data with a time step of t+1;

[0090] S203. Since the original EEG signal is a time series signal composed of continuous floating-point numbers, and the Spiking Neural Network (SNN) can only receive discrete spike sequences as input, when performing spike encoding on the original EEG signal, the present invention adopts a trainable encoding scheme to achieve the conversion of the original EEG signal to a spike signal. The encoder is constructed based on LIF neurons and converts the original EEG signal into a spike sequence. The formula is as follows:

[0091]

[0092] in, is the cumulative membrane potential at time step t, To input EEG signals, is the potential coding sequence, W Encoder It is a learnable encoder matrix that adaptively optimizes the signal-to-pulse mapping relationship as the neural network trains;

[0093] S3. Use graph convolution to process graph data signals. During the convolution process, the Laplacian matrix of the undirected weighted graph G is used , then L is calculated as L = DW. Since D and W are symmetric matrices, the Laplacian matrix of the undirected weighted graph G also has the properties of a real symmetric matrix. This property allows the Laplacian matrix to be decomposed into eigenvalues, expressed as:

[0094] L=UΛU -1 =UΛU T ;

[0095] Among them, U is an orthogonal matrix, that is, U T =U -1 , is an orthogonal matrix composed of the eigenvectors of the Laplace matrix, Λ=diag([λ0,...,λ N-1 ]) is the diagonal matrix of eigenvalues, for the input signal The Fourier transform of the graph can be expressed as;

[0096]

[0097] in, is the transformed signal, and the inverse transform of Fourier transform is expressed as:

[0098]

[0099] Defining graph signals and Its graph convolution x* G y is defined as:

[0100] x* G y=U((U T x)⊙(U T y));

[0101] Where ⊙ is the Hadamard product, that is, the product of corresponding elements, and a graph filter g(L) is defined, which is a function of the Laplace matrix L. Specifically, the result y after the signal x is processed by the graph filter can be expressed as:

[0102]

[0103] This formula is the graph convolution of x and Udiag(g(Λ)). The present invention adopts a trainable weight adjacency matrix, and sets W * is the optimal weight matrix obtained through model training. The graph convolution process is expressed as:

[0104] y=g(L * )x=U * g(Λ * )U *T x;

[0105] Among them, x is the image signal, Λ * By W * The diagonal matrix composed of the eigenvalues ​​of g(L * ) is the Laplace matrix L acting on the graph * The filter function, U * is the eigenvector matrix, U *T For U * The transpose of L * To normalize the graph Laplacian matrix, we can use the degree matrix D * and W * The adjacency matrix is ​​calculated using the following formula:

[0106] L * =D * -W * ;

[0107] Among them, D * is a diagonal matrix, and the Chebyshev polynomial is used to approximate the filter function g(·), thereby simplifying the calculation and improving the efficiency. The calculation formula is as follows:

[0108]

[0109] in, represents the normalized Λ * ,θ k are Chebyshev polynomial coefficients, T k is calculated as follows:

[0110]

[0111] Among them, T k (·) is a Chebyshev polynomial of order k, and the final graph convolution formula is:

[0112]

[0113] in, are the eigenvalues ​​of the normalized graph Laplacian matrix.

[0114] Based on the final graph convolution formula, a weight training (WT) module is introduced. The input of the WT module consists of graph structure data, where the node data represents the EEG signal encoded by the encoder. For each node, its receptive field is K-hop, and multiple convolution operations are performed on its K-hop neighbors, where K corresponds to the order of the Chebyshev polynomial. In each operation, the node updates its own features based on the features of its neighbors, and the updated feature set of all nodes forms a new feature map. In addition to fusing node information during training, the feature fusion weights within the receptive field range are also modified through backpropagation to obtain the optimal weight matrix under the current receptive field. By modifying the receptive field hyperparameters, the global optimal model weight matrix can be learned.

[0115] S4, the linear layer projects the input signal into a high-dimensional space to enhance feature separability; then, the LIF neuron processes the signal time-step by time-step through the membrane potential dynamic equation:

[0116] V(t+1)=k decay V(t)[1-H(V(t)-V th )]+I t+1 (p);

[0117] The membrane potential is reset after the pulse is triggered, forming an event-driven sparse computational flow. Through a multi-layer stacked structure, the network gradually extracts high-order temporal dependency features, and ultimately calculates the pulse firing rate of the last layer to generate the emotion probability distribution. To overcome the problem of the non-differentiable step function, the sigmoid function is used to approximate the pulse generation process. The calculation method is as follows:

[0118]

[0119] To ensure that the gradient can be propagated back while preserving the dynamic characteristics of the pulse, a sigmoid approximation-based alternative gradient mechanism is designed to calculate the pulse-triggered gradient. The calculation method is as follows:

[0120]

[0121] Here, α is a parameter that adjusts the steepness of the gradient curve, balancing nonlinear expressiveness with training stability. A high α enhances nonlinearity but can be prone to oscillation, while a low α improves stability but weakens discriminative power. Combined with the cross-entropy loss function, the backpropagation algorithm jointly optimizes the network weights, the trainable edge weight matrix, and the encoder parameters, achieving end-to-end training.

[0122] S5. The model's interpretability of EEG functional connectivity patterns was verified by comparing the edge connection features obtained from model training with the phase locking values. First, the EEG data was divided into different emotion categories, and the multi-subject average PLV matrix was calculated for each emotional state. PLV is defined as a measure of inter-channel phase synchronization and is expressed as:

[0123]

[0124] Among them, N t is the number of time points, and Channel i and j are in the nth t The signal phase angle at a given time point, PLV, has a value range of [0,1], with larger values ​​indicating stronger phase synchronization between the two channels. We further constructed a PLV difference matrix between emotions and calculated the absolute difference between the PLV matrices of different emotion categories to locate brain connectivity pathways where phase synchronization significantly changes during emotional state transitions.

[0125] To verify the correlation between the model's learned features and physiological mechanisms, the adaptive edge weight matrix of the trained model was extracted and spatial distribution consistency analysis was performed on it and the PLV difference matrix. The overall correlation between the two matrices was quantified using the Pearson correlation coefficient, and the spatial patterns of high-frequency connection pathways were compared using visualization techniques. For example, if high-weight edges in the prefrontal-limbic system overlap with strongly connected regions in the PLV difference matrix, it indicates that the model has captured a key functional coupling network related to emotion regulation.

[0126] In addition, parametric test methods (Shapiro-Wilk normality test and Levene homogeneity of variance test) were used to verify the statistical characteristics of the PLV data. When the conditions were met, paired tests (two-category tasks) or repeated measures analysis of variance (multi-category tasks) were performed for different emotional conditions. For example, in the valence dimension (two-category) analysis of the DEAP dataset, the PLV differences between emotional states were significant, p < 0.001; in the three-category task of the SEED dataset, the ANOVA test showed that the main effect of emotional category was significant, p < 0.001. After Bonferroni correction, the differences between the groups were statistically significant, p < 0.05. This analysis system verified the consistency of the model edge weight matrix and the PLV features from a statistical level, providing neuroscientific theoretical support for emotion classification decisions, while enhancing the credibility of the model in clinical scenarios.

[0127] The workflow of the emotion recognition and interpretation device based on event-driven EEG pulse graph of the present invention is as follows:

[0128] S1. The topology construction module uses a professional 64-lead EEG acquisition device and different emotion induction schemes. This involves inducing emotions in subjects by watching long videos of sadness, neutrality, and happiness. At the same time, the EEG digital signal data of the subjects in different emotional states is collected, and the EEG signal data set is constructed and assigned emotion labels. Based on the international 10-20 system and the extended 10-10 system standards, a lead topology diagram is constructed. An undirected weighted graph is constructed, which is represented as:

[0129] G = (V, E, W, F);

[0130] Among them, the node set V corresponds to the lead, and the weight matrix W calculates the spatial proximity d between the leads through the three-dimensional Euclidean distance ij , and set the threshold θ = 0.5 to remove weak connections. The weight matrix is ​​symmetrically normalized and the calculation formula is as follows:

[0131]

[0132] Eliminate node degree deviation and output the initial EEG topology map;

[0133] S2, the pulse encoding module is connected to the topology construction module. The pulse encoding module divides the raw EEG signal of each lead into continuous segments of 4 seconds, and each segment is further divided into 20 time steps. A trainable encoder is constructed based on the LIF neuron model. The membrane potential update formula is:

[0134]

[0135] When the membrane potential exceeds the threshold V th =1.0 triggers a pulse and resets when an event occurs, and optimizes the encoding matrix W through back propagation Encoder , converting the continuous signal into a sparse pulse sequence. The 64-lead pulse sequence constitutes the feature set F of the undirected weighted graph G;

[0136] S3, the pulse graph convolution module is connected to the pulse encoding module. The pulse graph convolution module uses the weight matrix W as a trainable parameter and dynamically optimizes the optimal edge weight matrix W through gradient descent. * , capturing emotion-related functional connectivity patterns.

[0137] Normalized Laplace matrix L based on Chebyshev polynomial expansion * , design a 6-order graph convolution kernel Aggregate multi-hop neighborhood spatial information to achieve deep fusion of spatiotemporal features;

[0138] S4, the spiking neural network decision module is connected to the spiking graph convolution module. The spiking neural network decision module constructs a two-layer spiking neural network, and the LIF neuron membrane potential update formula is:

[0139] V(t+1)=k decay V(t)[1-H(V(t)-V th )]+I t+1 (p);

[0140] The sigmoid function is used to approximate the pulse trigger, and the alternative gradient is calculated by taking the derivative:

[0141]

[0142] Combined with cross entropy loss, network parameters are optimized end-to-end, and the final layer generates emotional state output by counting pulse firing rates.

[0143] S5. The interpretability analysis module is connected to the pulse neural network decision module. The interpretability analysis module calculates the phase locking value matrix under different emotional states and constructs the PLV difference matrix under different emotional states, such as the PLV difference matrix D for the happy-sad emotional state. PLV . Extract the edge weight matrix W after model training * , from D PLV and W * The top 10% high-weight connections were screened, and Pearson correlation analysis was performed between the two (r=0.72, p<0.001). The spatial distribution consistency was verified by permutation test (p>0.05), thereby revealing key emotional pathways such as the prefrontal-limbic system.

[0144] Therefore, the present invention adopts the above-mentioned emotion recognition and interpretation method and device based on event-driven EEG pulse graph, deeply integrates the spatiotemporal information of EEG signals, accurately captures the dynamic characteristics of the brain, constructs a complete brain activity state map, and realizes objective and accurate assessment and decision-making of the emotional state of the subjects. It also introduces a physiological interpretability analysis mechanism to verify the reliability of the model from a neurophysiological perspective.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. An emotion recognition and interpretation method based on event-driven EEG pulse graph, characterized in that: The following steps are involved: S1. Based on the international 10-20 system and the extended 10-10 system standards, the coordinates of the physiological positions of the EEG leads in three-dimensional space are determined, and the Euclidean distances between the EEG leads are calculated based on the coordinates to construct an initial edge weight matrix, forming an undirected weighted graph with the leads as nodes and the Euclidean distances as edges, thereby obtaining an initial EEG topology map; S2, obtain the initial EEG topology map and original EEG signals in S1, perform multi-level time slicing on the original EEG signals, and then convert the original brain signals into pulse trains. The spatiotemporal features are dynamically mapped to graph nodes through a trainable pulse encoder to obtain complete EEG pulse map data; S3. Obtain the EEG pulse graph data in S2, and use the graph convolutional neural network with adaptive spatial information perception to hierarchically fuse the spatial correlation and pulse timing information between nodes. Dynamically adjust the edge weights through the adaptive edge weight optimization module to capture the emotional functional connection pattern, thereby achieving dynamic feature aggregation and obtaining the fused features after graph convolution processing; S4: Modeling the pulse timing dependency of the fused features in S3 through a spiking neural network, simulating the information processing process of the biological nervous system, and generating a probability distribution of the emotional state based on the pulse firing rate statistics, completing the classification decision and obtaining the predicted result of the emotional state; S5. By comparing the functional connectivity features learned by the model with the phase-locking values ​​calculated under different emotional states, the physiological rationality of the classification decision in S4 is verified.

2. The method for emotion recognition and interpretation based on event-driven EEG pulse graph according to claim 1, characterized in that: S1 includes the following steps: S101, based on the international 10-20 system and the extended 10-10 system standards, determine the coordinates of each lead in three-dimensional space according to anatomical landmarks to ensure that the electrode layout corresponds to the functional divisions of the cerebral cortex; S102, using the preset electrode mapping relationship of the standardized brain model, the lead p i The position is converted into three-dimensional coordinates in the Cartesian coordinate system (p ix ,p iy ,p iz ), define an undirected weighted graph, expressed as: G = (V, E, W, F); Among them, V is the node set corresponding to the EEG lead, E is the edge set, the edge set is determined by the weight matrix W, and F is the node feature set. Initialized as pulse-coded time series data, d is the feature dimension, N is the number of nodes, and the weight matrix W is used to calculate the spatial proximity between leads based on the three-dimensional Euclidean distance. The calculation formula is: Among them, d ij For node p i With node p j The Euclidean distance between ix ,p iy ,p iz are the x, y, and z coordinates of the i-th lead in three-dimensional space, and p jx ,p jy ,p jz are the x, y, and z coordinates of the jth lead in three-dimensional space; According to d ij The inverse relationship between the adjustment node p i and node p j The edge weight w between ij , defined as: Where k is the calibration constant; When initializing the weight matrix, insignificant connections are removed. The calculation formula is as follows: Among them, θ is the distance threshold; S103: Perform symmetric normalization on the weight matrix W. The calculation formula is as follows: Where D is the degree matrix D ii =∑ j w ij Used to eliminate node degree distribution bias.

3. The method for emotion recognition and interpretation based on event-driven EEG pulse graph according to claim 1, characterized in that: S2 includes the following steps: S201, dividing the original EEG signal into continuous segments of fixed duration to obtain first-level time segments, and further dividing the first-level time segments into T time steps to obtain second-level time segments; S202. The LIF neuron model is used to simulate biological neuron behavior, including charge accumulation and leakage. When the membrane potential reaches the threshold, an action potential is generated and the membrane potential is reset, which is expressed as: V(t+1)=k decay V(t)[1-H(V(t)-V th )]+I t+1 (p); Where V(t) is the membrane potential at time step t, k decay is the hyperparameter attenuation factor, V th is the pulse emission threshold, when V(t)≥V th When the pulse is triggered and the membrane potential is reset, H(·) is the Heaviside step function, I t+1 (p) is the input function; S203. Use a trainable encoder built based on LIF neurons. The formula is as follows: in, is the cumulative membrane potential at time step t, To input EEG signals, is the potential coding sequence, W Enocoder is the learnable encoder matrix.

4. The method for emotion recognition and interpretation based on event-driven EEG pulse graph according to claim 1, characterized in that: S3 also includes, let W * is the optimal weight matrix obtained through model training. The graph convolution process is expressed as: y=g(L * )x=U * g(Λ * )U *T x; Among them, x is the image signal, Λ * By W * The diagonal matrix composed of the eigenvalues ​​of g(L * ) is the Laplace matrix L acting on the graph * The filter function, U * is the eigenvector matrix, U *T For U * The transpose of L * is the normalized graph Laplace matrix, through the degree matrix D * and W * The adjacency matrix is ​​calculated using the following formula: L * =D * -W * ; Among them, D * is a diagonal matrix, and the Chebyshev polynomial is used to approximate the filter function g(·). The calculation formula is as follows: in, represents the normalized Λ * ,θ k are Chebyshev polynomial coefficients, T k is calculated as follows: Among them, T k (·) is a Chebyshev polynomial of order k, and the final graph convolution formula is: in, are the eigenvalues ​​of the normalized graph Laplacian matrix.

5. The method for emotion recognition and interpretation based on event-driven EEG pulse graph according to claim 1, characterized in that: In S4, the sigmoid function is used to approximate the pulse generation process, and the calculation method is as follows: And by calculating the derivative of the sigmoid function as an alternative gradient, the calculation method is as follows: Among them, α is a parameter.

6. An emotion recognition and interpretation device based on event-driven EEG pulse graph, characterized in that: Includes the following modules: A topology construction module is used to determine the coordinates of the physiological positions of EEG leads in three-dimensional space according to the international 10-20 system and the extended 10-10 system standards, and to calculate the Euclidean distance between EEG leads based on the coordinates to construct an initial edge weight matrix, forming an undirected weighted graph with leads as nodes and Euclidean distances as edges, thereby obtaining an initial EEG topology map; A pulse encoding module, connected to the topology construction module, is used to perform multi-level time slicing processing on the original EEG signal, and then convert the original brain signal into a pulse sequence. By training the pulse encoder, the spatiotemporal features are dynamically mapped to the graph nodes to obtain complete EEG pulse graph data; The pulse graph convolution module is connected to the pulse encoding module and uses a graph convolutional neural network with adaptive spatial information perception to hierarchically fuse the spatial correlation between nodes and the pulse timing information. The adaptive edge weight optimization module dynamically adjusts the edge weights to capture the emotional functional connection pattern, thereby achieving dynamic feature aggregation and obtaining fused features after graph convolution processing. A spiking neural network decision module, connected to the spiking graph convolution module, models the pulse timing dependency of the fused features through a spiking neural network, simulates the information processing process of the biological nervous system, and generates a probability distribution of the emotional state based on the statistical results of the pulse firing rate, completes the classification decision, and obtains the prediction result of the emotional state; The interpretability analysis module is connected to the pulse neural network decision module, and verifies the physiological rationality of the classification decision by comparing the functional connectivity features learned by the model with the phase locking value differences calculated under different emotional states.

Citation Information

Cited By

  • Two-way emotion recognition method based on electroencephalogram micro-state features

    CN121278476A