Neurophysiology heuristic electroencephalogram emotion recognition method based on deep learning
By using a neurophysiology-inspired Transformer and CNN fusion architecture, the problem of underutilization of the spatiotemporal interaction and temporal resolution of EEG signals in existing technologies is solved, achieving high accuracy and stability in emotion recognition and improving the robustness and generalization ability of the model.
Patent Information
- Application Number
- CN202610046043.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-14
- Publication Date
- 2026-04-28
AI Technical Summary
Existing technologies have failed to fully explore the spatiotemporal interaction representation of EEG signals and the long-term and short-term dependencies at different time resolution scales in emotion recognition, resulting in limited ability of emotion decoding models to represent complex neurophysiological responses.
We employ a neurophysiology-inspired Transformer and CNN fusion architecture, and achieve refined modeling of key neural representations in the process of emotion cognition by constructing spatiotemporal proximity modeling, independent temporal embedding with multiple temporal resolutions, and hierarchical dual attention mechanism.
It improves the accuracy and stability of emotion decoding and enhances the robustness and generalization ability of the model in different subjects and emotion tasks.
Smart Images

Figure CN121926595A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of signal processing technology, and specifically relates to a neurophysiological heuristic EEG emotion recognition method based on deep learning. Background Technology
[0002] Emotions are a fundamental component of daily human life, driving cognition and behavior and playing a crucial role in rational decision-making, interaction, perception, and communication. Emotion recognition has wide applications in assessing emotion-related mental disorders such as anxiety, depression, and autism. Traditional research typically relies on non-physiological signals, such as facial expressions, gestures, text, and speech, for emotion recognition. However, these signals are easily influenced by external factors and subjective consciousness. In contrast, physiological signals such as electroencephalography (EEG), electromyography (EMG), and electrooculography (EOG) are spontaneously recorded by the central nervous system, are less prone to falsification, and contain more information, providing a more effective method for emotion recognition.
[0003] Among physiological signals, EEG can stably reflect the electrophysiological activity of brain neurons in the cerebral cortex, offering advantages such as high temporal resolution, low acquisition cost, and accurate reflection of emotional states. EEG signals are data recorded over time by electrodes with different spatial distributions; therefore, simultaneously exploring spatial and temporal information and their interactions is crucial. In recent years, researchers have combined the spatial topology and temporal dependence of EEG with neurophysiological prior knowledge incorporated into neural networks to extract EEG features and improve the performance of emotion decoding. Common prior knowledge includes consideration of hemisphere asymmetry, neurophysiologically meaningful graph connectivity, and the temporal dynamics of EEG. However, existing techniques are often limited to single-dimensional feature extraction, failing to fully explore the potential spatiotemporal interactive representations and long-short-term dependencies at different temporal resolution scales in the emotional cognition process. This may not fully describe the emotional cognition process, thus limiting the ability of emotion decoding models to represent complex neurophysiological responses. Summary of the Invention
[0004] Neurophysiological studies have shown that emotions are not generated independently by a single region, but rather by multiple brain regions, including the prefrontal cortex, limbic system, and sensory cortex, forming a spatiotemporal collaborative activation network within a specific time window. Different brain regions exhibit dynamic functional coupling and decoupling processes during the emotion induction, evaluation, and maintenance stages. Simultaneously, this network exhibits a clear hierarchical organizational structure in the temporal dimension, with lower-level cortices responding to external stimuli on a millisecond-level timescale, while higher-level emotional and cognitive cortices integrate and modulate emotional states on a longer timescale. Based on this, to overcome the shortcomings of existing technologies, this invention provides an EEG emotion recognition method based on a neurophysiologically inspired Transformer and CNN fusion architecture. This method introduces neurophysiological prior guidance at the model structure level, constructing spatiotemporal proximity modeling, independent temporal embeddings at multiple time resolutions, and a hierarchical dual attention mechanism. This enables refined modeling of key neural representations in the emotion cognition process, thereby improving the accuracy and stability of emotion decoding.
[0005] A deep learning-based neurophysiological heuristic EEG emotion recognition method includes the following steps: Step 1: First, perform preprocessing operations such as downsampling and filtering on the EEG signal, and then calculate the differential entropy to construct potential feature inputs for characterizing emotional states.
[0006] Step 2: Input the latent features obtained in Step 1 into the emotion response proximity embedding module. This module simulates the spatiotemporal co-activation mechanism of the cerebral cortex network during emotion cognition, maps the latent features to a high-dimensional feature space, and assigns adaptive weights to different electrodes in the channel perception layer, thereby enhancing the response of emotion-related channels. At the same time, based on the high-dimensional features, the emotion dynamic response layer dynamically models the synchronous and asynchronous emotion response relationships between brain regions, forming a spatiotemporal view of the emotion state that reflects the collaborative activity of emotion-related brain regions.
[0007] Step 3: Input the spatiotemporal view of the emotional state obtained in Step 2 into the hierarchical spatiotemporal Transformer module to learn the deep spatiotemporal contextual dependencies between channels and time, and independently modulate the features of different electrodes through a gating mechanism to output an enhanced emotional channel representation. Step 4: Input the emotion channel representation obtained in Step 3 into the channel-independent temporal embedding module. By simulating the neural temporal hierarchy, model the temporal dynamics of each channel at multiple temporal resolutions to generate a multi-view temporal representation across time scales. Step 5: Input the multi-view temporal representation obtained in Step 4 into the hierarchical dual attention Transformer module, use two sets of self-attention mechanisms to model the short-term dynamic change features and long-term dependency features in parallel, and obtain a temporal representation that simultaneously contains global emotion evolution and local emotion fluctuation information through the fusion of long and short-term information. Step 6: Based on the time representation, determine the EEG emotional state and output the corresponding emotion recognition result.
[0008] The beneficial effects of this invention are: First, this invention achieves collaborative modeling of spatiotemporal interaction features and dynamic features at both long and short time scales in the process of emotion cognition by introducing emotion response proximity embedding, hierarchical spatiotemporal Transformer, independent temporal embedding at multiple time resolutions, and a dual attention mechanism. This structure enables the model to simultaneously characterize the co-activation relationships between brain regions and the dynamic laws governing the evolution of emotional states over time. This results in stronger separability of the extracted emotional features across different emotion categories and higher consistency within the same emotion category, thereby improving the stability and accuracy of emotion recognition.
[0009] Secondly, by employing a simulated emotion-driven spatiotemporal collaborative modeling and multi-timescale neural dynamic modeling mechanism, this invention can stably extract neural representations highly correlated with the emotional cognitive process under different EEG emotion data distribution conditions. This improves the robustness and generalization ability of the model in different subjects and different emotional tasks, and has good engineering application value. Attached Figure Description
[0010] Figure 1 This is a flowchart of the neurophysiological heuristic EEG emotion recognition method based on deep learning of the present invention; Figure 2 Visualize the data distribution; Figure 3 The brain region saliency activation maps were drawn from five subjects selected from three publicly available datasets for this invention. Detailed Implementation
[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0012] See Figure 1 The present invention provides a neurophysiological heuristic EEG emotion recognition method based on deep learning, which specifically includes the following steps: Step 1: Define the raw EEG signal as ,in This represents the i-th EEG trial containing C channels and N sample points. This represents the corresponding emotion label. F represents the sampling rate, and T represents the duration of each trial. After preprocessing each trial, DE features are extracted from each frequency band using a 0.5s non-overlapping window. As a potential feature, , This represents the number of frequency bands.
[0013] Step Two: Since EEG signals are the result of mixing multiple cortical sources, the contribution of different electrodes to emotion varies significantly. Therefore, before modeling the spatiotemporal dynamics of emotion, we first explicitly distinguish and weight emotion-related channels to highlight brain regions with high emotional information content and suppress redundant and noisy channels. Specifically, we use DE features... Mapping to a high-dimensional implicit temporal space , Time parameter With sampling rate The product value. In the DEAP, SEED, and SEED-IV datasets. = [128, 200, 200]. Then, with As input, a 1D channel attention map is derived in the channel-aware layer. , Broadcast along spatial dimensions and then by element Multiplication yields perceptual features. In the channel perception layer, spatial information is aggregated using Global Average Pooling (GAP) and Global Max Pooling (GMP) operations, respectively, and then input into a fully connected layer with shared weights to learn perceptual information about the spatial context. The computation of the channel perception layer can be represented as: in Indicates element-wise multiplication. Represents the sigmoid activation function. Represents the GELU activation function For the shared weights, where The scaling factor is used. Through a learnable channel weighting method, the channel perception layer can effectively extract highly informative channel features related to emotion.
[0014] Emotional cognition processes exhibit short-term stability over time, and their neurophysiological characteristics manifest as complex spatiotemporal co-activation across different brain regions. This may be reflected in EEG signals as synchronous and asynchronous response patterns between electrodes at different times under specific emotional states. To extract the dynamic response patterns between electrodes under specific emotional states, this invention designs an emotional dynamic response layer, utilizing a specific ratio of the EEG sampling rate. Design a sliding window with multiple time scales This simulates the hierarchical temporal organization of emotional neural processing, from rapid response to sustained modulation. These windows slide channel-by-channel with a specific step size s, capturing the aforementioned perceptual features. Each channel sequence is overlapped and divided into segments of different lengths to obtain segment token features. This segmentation approach allows the model to simultaneously observe neural activity of the same emotional state across multiple temporal receptive fields, providing a structured temporal basis for subsequent synchronous and asynchronous response modeling. Represents the number of segments, A sliding window can be defined as: in, j It is the spatiotemporal scale hierarchy of the window. The window ratios representing different levels, j The ratio coefficients are set to [0.5, 0.25, 0.125] for the spatiotemporal levels from 1 to 3, extracting diverse response representations above 2Hz. Then, to simulate the phase difference and time lag phenomena existing in different brain regions during emotional processing, this invention reassembles segments of the same channel in adjacent time slices by constructing a segment index matrix and a learnable offset matrix, thereby explicitly constructing an information embedding reflecting asynchronous neural modulation across time in the feature space. This process corresponds to the asynchronous modulation and functional rearrangement mechanism between different brain regions in the emotion-related cortical network. Specifically, through channel-by-channel... Generate the corresponding fragment index matrix and learnable offset matrix .in According to the segment order of each channel at the current time scale, divide 0 to... A deterministic index matrix, composed of repeated integer indices along the channel dimension, is used to represent the original temporal order of segments within that channel. This is a learnable offset vector generated for each channel, with initial values randomly sampled from a uniform distribution, used to simulate the time lag and phase difference that exist in different brain regions during emotional processing. By... and after the broadcast The recombined index matrix is obtained by summing and then taking the modulo. This ensures the index is within a valid range for fragment reconstruction. Specifically, it extends to... This allows for dimensional reconstructing across time steps. Taking a single time step as an example, this can be easily extended to batch processing dimensions. and It can be represented as: Where U represents a uniform distribution. Using Features of the original fragment token By performing proximity embedding and reshaping dimensions, asynchronous response information of each electrode at nearby times can be obtained. Furthermore, the original fragment token features Reshape dimensions to obtain synchronous response information at the same moment. ,in .Then, and Simultaneously input into a shared weight Fully connected layers mapped to dimensions The algorithm learns feature representations between synchronous and asynchronous operations. Finally, it sums the mapping results of the fully connected layers with shared weights and combines them with the perceptual features obtained through convolution. Multiplication yields a spatiotemporal view of emotional states. The calculation process is defined as follows: in Represents a reorganization dimension operation; `gather` operates from the fragment dimension. Perform index reconstruction. , This is the LeakRelu activation function.
[0015] Step 3: The Layered Spatiotemporal Transformer module consists of two stages. In the first stage, the spatiotemporal view of the emotional state output from Step 2 is first processed. Normalization is performed to eliminate distributional differences among response features. Then, a sliding time window with the same kernel size and step size is used at different levels. A set of large-kernel convolutions enhances the spatiotemporal receptive field, thereby enabling the extraction of spatiotemporally globally dependent features. The spatiotemporal hierarchy ratio coefficients of j from 1 to 3 are consistent with those in 3.1, kernel. KThe sampling rates are [1 / 2, 1 / 4, 1 / 8] to learn diverse spatiotemporal representations. While large kernel convolutions can achieve a larger spatiotemporal receptive field, they suffer from high computational cost. Therefore, a depth-separable design is adopted for large kernel convolutions to effectively reduce computational costs. Next, the features are input into a local convolutional group. This local convolutional group contains two up-dimensional point convolutions and two down-dimensional point convolutions with a kernel size of 1, both with a stride of 1. In the up-dimensional point convolution, the ratio is used... Dimensionality expansion is performed to increase nonlinearity and spatiotemporal information interaction. Then, dimensionality-reduced pointwise convolution extracts spatiotemporal local dependencies point-by-point in the high-dimensional space. Specifically, for... A set of padding is added to the edges of the features to help learn the underlying spatiotemporal context information. The first stage of the hierarchical spatiotemporal Transformer module can be represented as: in DWConv represents deep large kernel convolution, EPConv represents up-dimensional convolution, RPConv represents down-dimensional convolution, LN represents layer norm, and a droupout layer is added at the end.
[0016] In the second phase of the hierarchical spatiotemporal Transformer module, The aim is to learn more refined channel representations. The second stage consists of three branches: a noise reduction branch, a spatial branch, and a gating branch. First, it undergoes a linear layer with a 2x dimensional expansion to obtain... Then it is divided into two segments of the same dimension. One of the segments... The data is transferred to the spatial branch, utilizing the fact that the kernel size is equal to the dimension. Deep convolutions learn fine-grained and nearest-neighbor features across channels, then combine them with LeakyReLU for activation. The gated branch becomes another segment. A fully connected mapping is applied, followed by sigmoid activation to generate a gating factor for each channel. Weights are then redistributed element-wise with the spatial branch to obtain a more refined representation of the emotion channels. EEG has a low signal-to-noise ratio, so the denoising branch utilizes average pooling to reduce the impact of noise and feature dimensionality. The second-stage computation process can be represented as follows: In the second phase of the hierarchical spatiotemporal Transformer module, The aim is to learn more refined channel representations. The second stage consists of three branches: a noise reduction branch, a spatial branch, and a gating branch. First, it undergoes a linear layer with a 2x dimensional expansion to obtain... First apply the same as the gating branch The fully connected map is then split into two segments of the same dimension. One of the segments... The data is transferred to the spatial branch, utilizing the fact that the kernel size is equal to the dimension. Deep convolutions learn fine-grained and nearest-neighbor features across channels, then combine them with LeakyReLU for activation. The gated branch becomes another segment. A fully connected mapping is applied, followed by sigmoid activation to generate a gating factor for each channel. Weights are then redistributed element-wise with the spatial branch to obtain a more refined representation of the emotion channels. EEG has a low signal-to-noise ratio, therefore the denoising branch is designed for… Using average pooling to reduce the impact of noise and feature dimensionality, we obtain the features. Finally, the emotional channel representation passes through a fully connected layer. Mapping and Add them together to obtain the features. The second stage of the calculation process can be represented as follows: in Shared weight Features at three levels The data is concatenated and batch-normalized, then multi-scale feature representations between layers are learned using one-dimensional convolutions with a kernel size of 1, resulting in enhanced emotion channel representations. The fusion process can be represented as: Where BN stands for batch normalization, and the kernel size and stride in Conv are both 1. Reduced to Two-thirds of the dimension, L= .
[0017] Step Four: In the temporal dimension, EEG signals contain brain activity information across different frequency bands. This temporal information is crucial, reflecting the dynamic response and non-stationary characteristics of the nervous system at different time scales, especially the hierarchical oscillation patterns exhibited by different brain regions on short-term and long-term time scales during emotional cognition. Therefore, in extracting temporal features, complex dependencies across channels should be minimized to more accurately capture the temporal dynamics within each channel and simulate the neural temporal hierarchy. Specifically, firstly, the output of Step Three... Obtain by expanding dimensions Then, three time windows of different scales were designed. Sliding along dimension L, where t represents the corresponding time window scale. Short windows capture transient, rapid responses; medium windows capture mid-term integration processes; and long windows capture slow, long-term evolutionary patterns, thus enabling... Perform independent temporal embedding. t will be at temporal resolution levels 1 to 3. Temporal resolution windows were set to scales of [16, 8, 4] respectively. During the embedding process at different levels, Appropriate padding was performed to maintain dimensionality consistency, where the input features for each channel were expanded into an independent sequence, which was then divided into G blocks of different resolutions at different levels. Mapped to the D dimension, this forms a multi-view temporal representation spanning multiple time scales. ,in G=L / / s This multi-timescale independent channel embedding can simultaneously establish short-term transient changes and long-term evolution in the emotional cognition process, providing a neurophysiologically reasonable input representation for the global and local spatiotemporal modeling of the subsequent hierarchical dual attention Transformer module.
[0018] Step 5: The hierarchical dual-attention Transformer module contains a long-term attention layer and a short-term attention layer. The long-term attention layer affects the output of Step 4. Reorganize to combine channels and the dimension across time steps get ,in This method combines blocks across time steps, treating each block as an independent token, thereby enabling... Multi-head self-focus is employed to learn inter-block correlations and extract long-range, coarse-grained temporal dynamic representations. The long-term attention layer effectively models global interaction representations between different time segments, revealing cross-temporal dependencies within the scope of emotional cognition. Furthermore, the short-term attention layer... Further dimensional expansion yielded ,Will Perform remapping and combination , and G got ,in Then, each sampling point within a block spanning multiple time steps is treated as an independent token, and thus... Similarly, multi-head attention is performed to extract fine-grained temporal dynamic representations of short distances within blocks. Short-term attention layers can effectively capture microscopic features within fragments, enhancing the representation of dynamic changes and detailed structures at local temporal scales.
[0019] Specifically, taking the long-term attention layer as an example, following the standard multi-head attention protocol, the following steps are taken: A linear mapping to the query, key, and value space is represented as follows: , The long-term global concern characterization is obtained through calculation. The calculation process for the long-term attention layer can be represented as follows: in , , , Let be the parameter matrix corresponding to the linear transformation. The dimension representing each attention head. Short-term localized concerns are characterized as follows: .
[0020] Through the above steps, the temporal global attention representations of each level were obtained. and temporal local concern representation The long-short aggregation layer aims to learn a complete multi-level global and local temporal representation in neurophysiology across multiple temporal resolutions and temporal distances. Specifically, it will... and All reshaped back The data is then stitched together according to dimension D. Then, it is processed through a linear layer. Mapping and layer normalization are performed to further integrate global and local synchronization time representations. Subsequently, multi-level features... The data is then stitched together and standardized, and a one-dimensional convolution is used to extract temporal dynamic comprehensive representations at different resolutions. .at last, The input features required for the emotion classification task are generated for emotion recognition. The calculation method for the long-short-term aggregation layer is as follows: in , Weights for classification tasks , This represents the corresponding deviation.
[0021] See Figure 2By utilizing t-SNE to map EEG features to a two-dimensional space for visualization, the features extracted in this invention exhibit more pronounced inter-class separation on discrete sentiment datasets, indicating a better ability to distinguish different sentiment categories. Furthermore, the features extracted in this invention show a more concentrated distribution within the same sentiment category, reflecting its ability to better learn the internal consistency of sentiment categories. See also... Figure 3 This invention can effectively identify specific activation patterns in emotion-related brain regions and verify the neurophysiological mechanisms of emotion cognition. For example... Figure 3 As shown in the first row, the frontal, temporal, parietal, and occipital lobes of the subjects' brains were significantly activated on the DEAP dataset. Asymmetric activation in the prefrontal, parietal, and temporal lobes was closely related to arousal valence recognition, while the weight of the occipital lobe electrodes was lower than that of electrodes in other brain regions. This indicates that the frontal, parietal, and temporal lobes provide more emotion-related information, consistent with previous research. As shown in the second and third rows, the electrode weights in the frontal, temporal, and occipital lobes were significantly increased on the SEED and SEED-IV datasets, consistent with previous work. The activation of the frontal and occipital lobes was more significant in the SEED dataset than in the SEED-IV dataset. The difference in brain region activation between the SEED and SEED-IV datasets may be due to some bias in the 3-class and 4-class classification tasks. Occipital bone activity provided some information on all three datasets, possibly because the subjects' emotions were induced by music video stimuli.
[0022] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.
Claims
1. A neurophysiological heuristic EEG emotion recognition method based on deep learning, characterized in that, Includes the following steps: Step 1: Preprocess the EEG signal and calculate the differential entropy to construct the potential feature input representing the emotional state; Step 2: Input the latent features into the emotion response proximity embedding module to form a spatiotemporal view of the emotional state that reflects the coordinated activity of emotion-related brain regions; Step 3: Input the spatiotemporal view of emotional state into the hierarchical spatiotemporal Transformer module, and output the enhanced emotional channel representation; Step 4: Input the emotion channel representation into the channel-independent temporal embedding module, and generate a multi-view temporal representation across time scales by simulating the neural temporal hierarchy structure; Step 5: Input the multi-view temporal representation into the hierarchical dual attention Transformer module to obtain a temporal representation that simultaneously contains information on global emotion evolution and local emotion fluctuations; Step 6: Determine the EEG emotional state based on time representation and output the corresponding emotion recognition result.
2. The neurophysiological heuristic EEG emotion recognition method based on deep learning according to claim 1, characterized in that, In step one, the raw EEG signal is defined as ,in This represents the i-th EEG trial containing C channels and N sample points. This represents the corresponding emotion label; F is the sampling rate, and T is the duration of the trial. After preprocessing each trial, DE features are extracted from each frequency band using a non-overlapping window. As a potential feature, , This represents the number of frequency bands.
3. The deep learning-based neurophysiological heuristic EEG emotion recognition method according to claim 2, characterized in that, The emotion response proximity embedding module is specifically implemented as follows: Step 2.1: Based on latent features, simulate the spatiotemporal co-activation mechanism of the cerebral cortex network during the emotional cognition process, and map the latent features to high-dimensional features; Step 2.2: Based on high-dimensional features, dynamically model the synchronous and asynchronous emotional response relationships between brain regions to form a spatiotemporal view of emotional states that reflects the collaborative activities of emotion-related brain regions.
4. The deep learning-based neurophysiological heuristic EEG emotion recognition method according to claim 3, characterized in that, Step 2.1 is implemented as follows: DE features Mapping to a high-dimensional implicit temporal space , Time parameter With sampling rate The product value; As input, the channel attention graph is derived in the channel-aware layer. , Broadcast along spatial dimensions and then by element Multiplication yields perceptual features. ; In the channel perception layer, spatial information is aggregated by using global average pooling and global max pooling operations respectively, and then input into a fully connected layer with shared weights to learn the perceptual information of the spatial context. The computation of the channel perception layer is represented as follows: in Indicates element-wise multiplication. Represents the sigmoid activation function. Represents the GELU activation function For shared weights, where This is the scaling ratio.
5. The deep learning-based neurophysiological heuristic EEG emotion recognition method according to claim 4, characterized in that, Step 2.2 is implemented as follows: Design an emotion dynamic response layer, utilizing the ratio of EEG sampling rates. Design a sliding window with multiple time scales This simulates the hierarchical temporal organization structure of emotional neural processing, from reaction to sustained modulation; the window slides channel by channel according to a step size s, displaying perceptual features. Each channel sequence is overlapped and divided into segments of different lengths to obtain segment features. ; For the number of segments, The sliding window is: in, j It is the spatiotemporal scale hierarchy of the window. Window ratios representing different levels; Then, by constructing a segment index matrix and a learnable offset matrix, segments of the same channel in adjacent time slices are recombined. This explicitly constructs an information embedding in the feature space that reflects asynchronous neural modulation across time steps. Specifically, this is done channel-by-channel. Generate the corresponding fragment index matrix and learnable offset matrix ;in According to the segment order of each channel at the current time scale, from 0 to... A deterministic index matrix, formed by repeating integer indices along the channel dimension, represents the original temporal order of segments within that channel; It is a learnable offset vector generated for each channel, with initial values randomly sampled from a uniform distribution, simulating the time lag and phase difference in different brain regions during emotional processing; by... and after the broadcast The recombined index matrix is obtained by summing and then taking the modulo. ; use Fragment features By performing proximity embedding and reshaping the dimensions, asynchronous response information of each electrode at nearby times can be obtained. ; fragment features Reshaping dimensions to obtain synchronous response information at the same moment. ,Then, and Simultaneously input into a shared weight The fully connected layer maps the fragment length to the dimension. The algorithm learns feature representations between synchronous and asynchronous operations; finally, it sums the mapping results of the fully connected layers with shared weights and combines them with the perceptual features obtained through convolution. Multiplication yields a spatiotemporal view of emotional states. The calculation process is defined as follows: in Representing a reorganization dimension operation, `gather` operates from the fragment dimension. Perform index reconstruction. This is the LeakRelu activation function.
6. The neurophysiological heuristic EEG emotion recognition method based on deep learning according to claim 5, characterized in that, Step three is implemented as follows: Step 3.1: Input the spatiotemporal view of emotional state into the hierarchical spatiotemporal Transformer module to learn the deep spatiotemporal contextual dependencies between channels and time. Step 3.2: Deep spatiotemporal context dependency uses a gating mechanism to independently modulate the features of different electrodes, outputting an enhanced emotional channel representation.
7. The neurophysiological heuristic EEG emotion recognition method based on deep learning according to claim 6, characterized in that, Step 3.1 is implemented as follows: First, a spatiotemporal view of emotional states. Normalization is performed, and then a sliding time window with the same kernel size and step size is used at different levels. A set of depthwise separable convolutions enhances the spatiotemporal receptive field and extracts spatiotemporally globally dependent features. Next, the features The input is fed into a local convolutional group, which contains two up-dimensional point convolutions and a down-dimensional point convolution with a kernel size of 1. The up-dimensional point convolution is then processed at a ratio... Perform dimensionality expansion, then perform dimensionality reduction and point convolution to extract spatiotemporal local dependencies point by point. .
8. The neurophysiological heuristic EEG emotion recognition method based on deep learning according to claim 7, characterized in that, Step 3.2 is implemented as follows: Construct three branches: noise reduction branch, spatial branch, and gating branch; First, dimensional expansion is performed through a linear layer to obtain... It is then divided into two segments of the same dimension, one of which... The data is fed into the spatial branch, where it first undergoes the same fully connected mapping as the gated branch, and then utilizes a kernel size equal to the dimension. The deep convolution learns fine-grained and nearest-neighbor features across channels, then combines them with LeakyReLU for activation; the gated branch is another segment. A fully connected mapping is applied, followed by sigmoid activation to generate a gating factor for each channel. Weights are then redistributed through element-wise multiplication with the spatial branch to obtain the emotion channel representation; the noise reduction branch targets… By using average pooling to reduce the impact of noise and feature dimensionality, the features are obtained. Finally, the emotional channel representation is mapped through a fully connected layer and... Add them together to obtain the features. ; Features at different levels The data is concatenated and batch-normalized, then multi-scale feature representations between layers are learned using one-dimensional convolutions with a kernel size of 1, resulting in enhanced emotion channel representations. .
9. The neurophysiological heuristic EEG emotion recognition method based on deep learning according to claim 8, characterized in that, Step four is specifically implemented as follows: First, the output of step three is processed... Dimensional expansion Then, three time windows of different scales were designed. Slide along dimension L. This represents the time window corresponding to the scale. Independent temporal embedding is performed, during which the input features of each channel are unfolded into an independent sequence, and these sequences are divided into G blocks of different resolutions at different levels. Mapped to the D dimension, forming a multi-view temporal representation across time scales. .
10. The neurophysiological heuristic EEG emotion recognition method based on deep learning according to claim 9, characterized in that, Step five is specifically implemented as follows: the hierarchical dual-attention Transformer module includes a long-term attention layer and a short-term attention layer; the long-term attention layer... Reorganize to combine channels and the dimension across time steps get ,right Execute multi-head self-focused learning to learn the correlations between blocks and extract long-range coarse-grained temporal dynamic representations. The short-term attention layer, on the other hand, is... Dimensional expansion obtained ,Will Perform a remapping process to obtain ,in Then, each sampling point within a block spanning multiple time steps is treated as an independent token for... Similarly, multi-head attention is performed to extract fine-grained temporal dynamic representations of short distances within blocks. ; Will and After reshaping the dimensions, the parts are then stitched together. Then through a linear layer pair Perform mapping and layer normalization to fuse global and local synchronization time representations. ; then, The data is stitched together, and the stitched results are standardized. Then, a 1D convolution is used to extract the temporal dynamic comprehensive representation at different resolutions. ;at last, The linear layer generates the input features required for the emotion classification task, which are time representations that simultaneously contain information on global emotion evolution and local emotion fluctuations, and then performs emotion recognition.
