An electroencephalogram signal emotion recognition method with multi-feature fusion adaptive step length

Abstract

This invention discloses a multi-feature fusion adaptive step-size EEG signal emotion recognition method, belonging to the field of signal processing technology. The method involves acquiring the EEG signal of the user to be identified; the EEG signal includes four channels, each channel comprising four fixed frequency bands. For each frequency band of any channel, an adaptive step-size is determined based on the energy, signal fluctuation intensity, and dominant frequency component of the EEG signal. The EEG signal is segmented using the adaptive step-size, and the differential entropy of each segment is calculated. The differential entropy sequences of the four frequency bands in the four channels are stacked into three-dimensional data. Spatiotemporal features of the three-dimensional data are extracted. A global spatiotemporal dependency is established through dynamic learnable positional coding and multi-head attention mechanisms to determine the coded features. The coded features are classified to obtain the emotion recognition result. This method can ensure high accuracy of emotion recognition results using portable EEG devices.

Description

Technical Field

[0001] This invention relates to the field of signal processing technology, and in particular to a method for emotion recognition of EEG signals with multi-feature fusion and adaptive step size. Background Technology

[0002] Currently, emotion is not only a key factor in human decision-making, planning, reasoning, and other higher cognitive activities, but also an indispensable component of human-computer interaction. Emotion recognition is a part of the field of artificial intelligence, and its research and application have covered many aspects of daily life. Emotion is also an individual's attitudinal experience and corresponding behavioral response to objective things, permeating all aspects of human perception, decision-making, and interpersonal communication. Electroencephalogram (EEG) signals, as a direct reflection of neuronal activity in the brain, provide a physiological basis for studying emotion. By analyzing EEG signals, researchers can gain a deeper understanding of the mechanisms of emotion generation and its representation in the brain. Therefore, emotion recognition research based on EEG signals can not only provide objective indicators for the diagnosis and treatment of mental illnesses, but also promote affective computing and intelligent interaction.

[0003] Currently, EEG signal acquisition mainly relies on devices such as electrode caps. Commonly used electrode systems include the international 10-20 system and its extended versions. These devices are typically bulky and expensive, primarily suitable for laboratory environments. In recent years, the rise of portable EEG devices has facilitated the acquisition of fewer-channel EEG signals. For example, devices such as the OpenBCI Cyton and g.tec Unicorn Hybrid Black require only a few electrodes to acquire effective EEG signals, providing a new data source for emotion recognition research. Portable EEG devices contain only a few electrodes, are small in size, inexpensive, and easy to carry, making them more suitable for everyday applications and consumer products.

[0004] However, portable EEG devices can only capture a limited number of brain signals. How to achieve a universally applicable recognition accuracy while utilizing portable EEG devices has become an urgent problem to be solved. Summary of the Invention

[0005] Therefore, it is necessary to provide a multi-feature fusion adaptive step size EEG signal emotion recognition method to address the above-mentioned technical problems. This method can ensure high accuracy of emotion recognition results while utilizing portable signal acquisition equipment.

[0006] The present invention adopts the following technical solution:

[0007] This invention provides a method for emotion recognition using EEG signals with multi-feature fusion and adaptive step size, comprising:

[0008] Acquire the EEG signal of the user to be identified; the EEG signal includes four channels of EEG signal, each channel including four fixed frequency bands of EEG signal;

[0009] For each frequency band of the EEG signal in any channel, the energy, signal fluctuation intensity, and dominant frequency component of the EEG signal are acquired, and the adaptive step size is determined based on the energy, signal fluctuation intensity, and dominant frequency component of the EEG signal.

[0010] The EEG signal was segmented with an adaptive step size using a sliding window technique to obtain multiple EEG signal segments, and the differential entropy of each EEG signal segment was calculated. A sequence was formed based on the differential entropy of all EEG signals in each frequency band, and the sequences of the four frequency bands in the four channels were stacked into three-dimensional data.

[0011] Spatial features of 3D data are extracted and extended in the time dimension to obtain spatiotemporal features. Based on the spatiotemporal features, a global spatiotemporal dependency is established through dynamic learnable position coding and multi-head attention mechanism to determine the coding features of EEG signals.

[0012] The encoded features are classified to obtain the emotion recognition results of the user to be identified.

[0013] Optionally, the adaptive step size is determined based on the energy, signal fluctuation intensity, and dominant frequency component of the EEG signal, including:

[0014] Weights were assigned to the energy, signal fluctuation intensity, and dominant frequency components of the EEG signal to determine the regulatory factors;

[0015] The base step size is adjusted proportionally according to the adjustment factor to obtain the adjustment step size;

[0016] By superimposing a uniformly distributed random perturbation on the adjustment step size, an adaptive step size is obtained.

[0017] Optionally, the formula for calculating the adjustment factor is:

[0018]

[0019] in, Indicates the regulating factor. Indicates energy. Indicates the intensity of signal fluctuations. This represents the main frequency component.

[0020] Optionally, the formula for calculating the adaptive step size is:

[0021]

[0022] in, Indicates adaptive step size, Indicates the adjustment step size. Indicates the basic step size. Indicates the range of fluctuation Uniform randomization within, As a disturbance factor, Indicates the minimum limit value. Indicates the maximum limit value. Indicates will Limited to Within the range.

[0023] Optionally, emotion recognition results are obtained through an emotion recognition model; the emotion recognition model includes a spatiotemporal feature extraction module, a Transformer encoding module, and a classification module connected in sequence.

[0024] The spatiotemporal feature extraction module includes a two-dimensional convolutional neural network and a two-dimensional temporal convolutional network connected in series. The two-dimensional convolutional neural network includes a convolutional layer, a batch normalization layer, and an activation function layer connected in series. The two-dimensional temporal convolutional network includes three convolutional blocks connected in series and an average pooling layer. Each convolutional block includes a two-dimensional dilated convolutional layer, a batch normalization layer, an activation function layer, and a regularization layer connected in series. The dilation rate and the number of channels of the three convolutional blocks are different.

[0025] The Transformer coding module includes multiple Transformer coding units connected in series. Each Transformer coding unit includes a position coding unit, a multi-head attention unit, a feedforward network, and a normalization layer connected in series.

[0026] The classification module includes a KAN classifier; the KAN classifier consists of a two-layer KAN network architecture.

[0027] Optionally, spatial features of the three-dimensional data are extracted, and these spatial features are extended along the time dimension to obtain spatiotemporal features, including:

[0028] Three-dimensional data is input into a two-dimensional convolutional neural network, and then passed through a convolutional layer, a batch normalization layer, and an activation function layer in sequence to obtain the spatial features of the three-dimensional data;

[0029] Spatial features are input into a two-dimensional temporal convolutional network and passed through three convolutional blocks and an average pooling layer in sequence to obtain spatiotemporal features.

[0030] Optionally, global spatiotemporal dependencies are established through dynamic learnable location encoding and multi-head attention mechanisms to determine the encoded features of EEG signals, including:

[0031] For any Transformer coding unit, the spatiotemporal features are input into the Transformer coding unit, and the spatiotemporal features are positionally encoded using the positional encoding matrix in the positional encoding unit to obtain the positionally encoded features;

[0032] Attention features are obtained by extracting attention from position-encoded features using the attention mechanism in a multi-head attention unit.

[0033] The attention features are processed by a feedforward network to obtain feedforward features;

[0034] The feedforward features and attention features are added together to obtain the fused features;

[0035] The fused features are nonlinearly transformed by a normalization layer to obtain the encoded features.

[0036] Optionally, the elements in the position coding matrix are calculated as follows:

[0037]

[0038]

[0039] in, This represents the position at position pos in the position encoding matrix. dimensional features, This represents the position at position pos in the position encoding matrix. dimensional features, and These represent the learnable frequency and phase parameters, respectively. d This indicates the dimension of the position encoding matrix.

[0040] Optionally, the encoded features are classified to obtain the emotion recognition results of the user to be identified, including:

[0041] The encoded features are input into the classification module, and the KAN classifier is used to classify the encoded features to obtain the emotion recognition result.

[0042] This invention provides a multi-feature fusion adaptive step-size EEG signal emotion recognition device, comprising:

[0043] The acquisition module is used to acquire the EEG signals of the user to be identified; the EEG signals include four channels of EEG signals, each channel including four fixed frequency bands of EEG signals;

[0044] The determination module is used to acquire the energy, signal fluctuation intensity, and dominant frequency component of the EEG signal for each frequency band of any channel, and determine the adaptive step size based on the energy, signal fluctuation intensity, and dominant frequency component of the EEG signal.

[0045] The segmentation module is used to segment EEG signals with an adaptive step size using a sliding window technique to obtain multiple EEG signal segments, and calculate the differential entropy of each EEG signal segment; and form a sequence based on the differential entropy of all EEG signals in each frequency band; and stack the sequences of the four frequency bands in the four channels into three-dimensional data.

[0046] The extraction module is used to extract the spatial features of the three-dimensional data and expand the spatial features in the time dimension to obtain spatiotemporal features; based on the spatiotemporal features, a global spatiotemporal dependency relationship is established through position encoding and multi-head attention mechanism to determine the encoding features of the EEG signal;

[0047] The classification module is used to classify the encoded features to obtain the emotion recognition results of the user to be identified.

[0048] The present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for EEG signal emotion recognition with multi-feature fusion and adaptive step size.

[0049] The present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-mentioned multi-feature fusion adaptive step size EEG signal emotion recognition method.

[0050] The above-mentioned at least one technical solution adopted in this invention can achieve the following beneficial effects:

[0051] In this invention, an adaptive step size is determined based on the signal quality of the EEG signal. The EEG signal is segmented using a sliding window technique to obtain multiple segments of the EEG signal. This compensates for the limited number of acquisition channels in portable EEG devices due to size and cost limitations, thus improving the analysis quality of the EEG signal. Then, by using spatiotemporal features and attention mechanisms, a global correlation across time and space is established, further compensating for the insufficient number of channels. This design is equivalent to using algorithmic advantages to compensate for hardware defects, enabling the limited channel signals acquired by the portable EEG device to be fully utilized, thereby ensuring the accuracy of emotion recognition. Attached Figure Description

[0052] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0053] Figure 1 This invention provides a flowchart illustrating a method for emotion recognition of EEG signals with multi-feature fusion and adaptive step size.

[0054] Figure 2This is a schematic diagram of the overall process of an EEG signal emotion recognition method with multi-feature fusion and adaptive step size provided by the present invention;

[0055] Figure 3 This is a schematic diagram of the structure of each module in an emotion recognition model provided by the present invention;

[0056] Figure 4 A schematic diagram of the window step size distribution of subjects in four frequency bands provided by the present invention;

[0057] Figure 5 This invention provides a four-band EEG scalp topography map of a subject's four-channel EEG signal;

[0058] Figure 6 This is a schematic diagram of a computer device for implementing a method for emotion recognition of EEG signals with adaptive step size through multi-feature fusion, as provided by the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0060] Physiological signals are irreplaceable in fields such as health monitoring and human-computer interaction. Compared to other emotion recognition methods, physiological signals (such as EEG and ECG) can objectively reflect health status or emotional state and are less affected by light. They are also difficult to fake and reduce the risk of privacy breaches, making them more suitable for applications such as mental illness assessment, thus becoming a common tool in emotion recognition research. However, while traditional high-density EEG acquisition devices can provide rich emotional information, their excessive complexity and high cost limit their application in consumer products.

[0061] In the field of human-computer interaction, electroencephalography (EEG) plays an increasingly important role. By directly capturing brain activity, it provides a novel interaction method and intelligent support for human-computer interaction systems. EEG has significant applications in neuroscience and human-computer interaction, including but not limited to emotion recognition and computation, being a core data source for brain-computer interfaces (BCI), and monitoring cognitive load and attention. Emotional BCIs are closed-loop emotion computing systems that construct interactive environments; EEG can reflect the user's emotional state (such as pleasure, anxiety, etc.) in real time. In the field of BCI, EEG is the foundation of BCI technology; by decoding EEG signals, users can directly control external devices with their brains. This invention focuses on exploring the problems of EEG in the field of emotion recognition and classification.

[0062] In the field of human-computer interaction, with the rapid development of deep learning technology, EEG emotion recognition methods based on deep learning have achieved remarkable results. Deep models such as Convolutional Neural Networks (CNNs), Long Short-Term Memory Networks (LSTMs), Generative Adversarial Networks (GANs), and Graph Convolutional Networks (GCNs) are widely used in EEG emotion recognition tasks. These methods overcome the limitations of traditional machine learning methods in handling nonlinear and non-stationary EEG signals by automatically extracting high-level features from EEG signals, thus improving the accuracy of emotion recognition. In the future, with the continuous innovation and improvement of deep learning technology, emotion recognition methods based on EEG signals are expected to play an even more important role in the field of human-computer interaction.

[0063] In existing technologies, one approach proposes an ST-GCN model based on spectral graph convolution and temporal analysis, which improves recognition accuracy through channel information reconstruction layers and spatiotemporal feature fusion. Another approach proposes a Spatiotemporal Graph BERT Network (STGB), which extracts EEG spatial features through subgraph partitioning, node embedding, and attention-based feature updates, and utilizes LSTM to learn temporal correlations, achieving an accuracy of 89.39% on the SEED dataset. Until the Transformer, proposed in 2017, completely abandoned the RNN structure, relying solely on self-attention mechanisms and feedforward networks to process sequence data.

[0064] Currently, time-domain, frequency-domain, and time-frequency-domain analysis are core methods for signal feature extraction. With in-depth research, frequency-domain and time-frequency-domain analysis methods have received widespread attention due to their advantages in revealing the non-stationary characteristics of signals, and have achieved significant results in multiple application areas. For example, power spectral density (PSD) from frequency-domain analysis is used as a feature, and a support vector machine (SVM) classifier is used for classification. With the significant development of deep learning, it has replaced traditional manual design by automatically extracting multi-level features, and is particularly suitable for processing high-dimensional unstructured data. For example, power spectral density features of frequency bands were extracted for evaluation, and DANN was used for automatic feature extraction and classification. This ultimately achieved good results in independent experiments with subjects, demonstrating that the joint feature extraction by manual and deep learning can improve accuracy. Simultaneously, deep learning, through an end-to-end learning approach, unifies feature extraction with task objectives, significantly improving model performance and generalization ability. An end-to-end Local-Global Graph Attention Network (LG-GAT) directly learns time-frequency features from raw EEG signals and models brain functional connectivity through graph structure, achieving good results on SEED.

[0065] With the use of portable signal acquisition devices, achieving a universally applicable level of recognition accuracy has become a pressing issue. Although multi-channel EEG devices can capture fewer brain signals, some high-noise channels (such as Fp1 and Fp2) or invalid channels can affect recognition. Therefore, selecting appropriate channels is fundamental to ensuring accuracy. However, achieving high-precision emotion recognition through algorithm and model design remains one of the most important research challenges.

[0066] Therefore, this invention aims to explore emotion recognition based on low-channel EEG configuration. EEG signals exhibit significant non-stationarity and time-varying characteristics, providing a theoretical basis for the adaptive step-size window sliding technique. Specifically, EEG signals exhibit two key characteristics: first, neural oscillations at different frequencies possess significantly differentiated temporal dynamics, while transient bursts of high-frequency gamma waves (>30Hz) last only a few hundred milliseconds; second, emotion-related event-related potentials (ETPs) exhibit precise time-locking characteristics (time jitter <50ms), and a fixed step size may lead to the segmentation of key components. Based on these characteristics, this invention proposes an adaptive step-size window technique. Its core is to dynamically calculate the step size based on the signal's stationarity by real-time calculation of activity indicators such as energy and gradient, then segment the EEG signal based on the step size, extract differential entropy (DE) features, and further extract features for recognition and classification.

[0067] The technical solutions provided by the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0068] Figure 1This is a flowchart illustrating a multi-feature fusion adaptive step-size EEG signal emotion recognition method according to the present invention, which specifically includes the following steps:

[0069] S101, acquire the EEG signal of the user to be identified; the EEG signal includes four channels of EEG signal, each channel including four fixed frequency bands of EEG signal.

[0070] Optionally, the four channels can be F3, F4, T7, and T8.

[0071] S102: For each frequency band of the EEG signal in any channel, acquire the energy, signal fluctuation intensity, and dominant frequency component of the EEG signal, and determine the adaptive step size based on the energy, signal fluctuation intensity, and dominant frequency component of the EEG signal; segment the EEG signal with the adaptive step size using the sliding window technique to obtain multiple segments of EEG signal, and calculate the differential entropy of each segment of EEG signal; form a sequence based on the differential entropy of all EEG signals in each frequency band; and stack the sequences of the four frequency bands in the four channels into three-dimensional data.

[0072] The sliding window technique divides continuous time-series data into fixed-length windows and slides these windows along the time axis with a certain step size to extract local features. It allows for the analysis of local features while maintaining temporal resolution and avoids the loss of detail caused by global averaging. This method can not only capture the dynamic changes of signals but also adapt to the characteristics of non-stationary signals.

[0073] EEG signals are typical non-stationary signals, and their statistical characteristics (such as mean, variance, and frequency components) change rapidly over time. Furthermore, EEG neural activity exhibits millisecond-level temporal dynamics. Smaller window sliding steps result in higher temporal accuracy, but also increase computational complexity. Based on this, this invention proposes an adaptive step-size window sliding technique, which dynamically adjusts the step size based on three key signal characteristics: energy content, temporal variability, and spectral composition. For each input signal segment, the following are first calculated: 1) the normalized energy as mean square amplitude, 2) the average absolute gradient characterizing signal fluctuation intensity, and 3) the dominant frequency component obtained through spectral analysis (excluding DC and Nyquist frequencies). These characteristics are obtained through efficient computational methods: energy is calculated using direct arithmetic mean, gradients are calculated using the first-order finite difference method, and spectral analysis uses a fast Fourier transform (FFT) with intelligent frequency band selection. Therefore, the energy and signal fluctuation intensity of the EEG signal are calculated using formulas (1) and (2).

[0074] (1)

[0075] (2)

[0076] in, Indicates energy. Indicates the intensity of signal fluctuations. This indicates the number of data points in the electroencephalogram (EEG) signal. Indicates the first in the EEG signal i Data points, Indicates the first in the EEG signal i +1 data point.

[0077] In one embodiment, determining an adaptive step size based on the energy, signal fluctuation intensity, and dominant frequency component of the EEG signal includes: assigning weights to the energy, signal fluctuation intensity, and dominant frequency component of the EEG signal to determine an adjustment factor; adjusting the base step size proportionally according to the adjustment factor to obtain an adjustment step size; and superimposing a uniformly distributed random perturbation on the adjustment step size to obtain an adaptive step size.

[0078] The step size adjustment employs a weighted multi-feature fusion algorithm, where each feature contributes differently to the final decision. The adjustment factor is calculated using formula (3). The weighting determined experimentally (60% energy, 30% signal fluctuation intensity, and 10% dominant frequency component) reflects a priority given to signal stability over rapid changes. This adjustment factor maintains natural constraints. At the same time, the basic step size Adjust proportionally.

[0079] The formula for calculating the adjustment factor is:

[0080] (3)

[0081] in, Indicates the regulating factor. Indicates energy. Indicates the intensity of signal fluctuations. This represents the main frequency component.

[0082] To prevent algorithm stagnation in periodic signals, a controlled random fluctuation mechanism is introduced, which involves superimposing a uniformly distributed random perturbation onto the adjusted step size. The final step size is limited to a preset range and rounded to ensure window alignment accuracy. This approach combines deterministic feature-driven tuning with beneficial randomization, ensuring responsiveness to signal content while enhancing robustness against local optima. The formula for calculating the adaptive step size is:

[0083] (4)

[0084] in, Indicates adaptive step size, Indicates the adjustment step size. Indicates the basic step size. Indicates the range of fluctuation Uniform randomization within, As a disturbance factor, Indicates the minimum limit value. Indicates the maximum limit value. Indicates will Limited to Within the range. , .

[0085] Among them, the EEG signal is segmented with an adaptive step size using the sliding window technique to obtain multiple segments of EEG signal, including: segmenting the EEG signal with an adaptive step size according to a preset window length to obtain multiple segments of EEG signal.

[0086] Differential entropy, a continuous form of information entropy, measures the uncertainty or complexity of a random variable. It accurately reflects the energy distribution and complexity of a signal across different frequency bands and is a commonly used feature extraction method in EEG signal analysis, demonstrating excellent performance. Its differential entropy... The mathematical definition is: Let X be the probability density function of the random variable. It is the variance of brainwave signals.

[0087] (5)

[0088] The differential entropy of the EEG signal can be calculated using formula (5).

[0089] This invention introduces an adaptive step-size sliding window (AS) technique to enhance the analysis quality of EEG signals.

[0090] For the differential entropy features calculated for each frequency band, a sequence of length 265 is constructed. Then, the sequences constructed for the four frequency bands of the four channels are stacked into a 4×4×265 three-dimensional data.

[0091] S103, extract the spatial features of the three-dimensional data, and expand the spatial features in the time dimension to obtain spatiotemporal features; based on the spatiotemporal features, establish a global spatiotemporal dependency relationship through dynamic learnable position coding and multi-head attention mechanism to determine the coding features of EEG signals.

[0092] Optionally, emotion recognition results are obtained through an emotion recognition model; the emotion recognition model includes a spatiotemporal feature extraction module (2DST), a Transformer encoding module (DTK), and a classification module connected in series.

[0093] The spatiotemporal feature extraction module includes a two-dimensional convolutional neural network and a two-dimensional temporal convolutional network connected in series. The two-dimensional convolutional neural network includes a convolutional layer, a batch normalization layer, and an activation function layer connected in series. The two-dimensional temporal convolutional network includes three convolutional blocks connected in series and an average pooling layer. Each convolutional block includes a two-dimensional dilated convolutional layer, a batch normalization layer, an activation function layer, and a regularization layer connected in series. The dilation rate and the number of channels of the three convolutional blocks are different.

[0094] The emotion recognition model first utilizes a two-dimensional convolutional neural network (2D-CNN) to extract local spatial features from the input data, focusing on capturing local patterns in the spatial dimension. Then, through dilated convolution operations in a two-dimensional temporal convolutional network (2DTCN), it further mines spatiotemporal dynamic features based on the spatial features extracted by the CNN, effectively capturing the evolutionary patterns of the data in both time and space. Finally, a Transformer module is introduced, leveraging its multi-head attention mechanism to establish global spatiotemporal dependencies, comprehensively modeling long-range associations between different spatiotemporal locations. This progressive feature extraction framework from local to global, combined with the nonlinear transformation capabilities of the feedforward network and the stabilization design of residual connections and layer normalization, enables the model to progressively deepen feature understanding through a multi-layer Transformer structure, ultimately forming a powerful representation and analysis capability for complex spatiotemporal data.

[0095] In one embodiment, the spatial features of three-dimensional data are extracted and extended in the time dimension to obtain spatiotemporal features, including: inputting three-dimensional data into a two-dimensional convolutional neural network, passing it sequentially through a convolutional layer, a batch normalization layer, and an activation function layer to obtain the spatial features of the three-dimensional data; and inputting the spatial features into a two-dimensional temporal convolutional network, passing it sequentially through three convolutional blocks and an average pooling layer to obtain the spatiotemporal features.

[0096] Because the original EEG signal has a high sampling rate, even after downsampling preprocessing, the time series length is still quite long. Directly using sequence models such as Transformer would face problems such as high computational complexity and difficulty in modeling long-range dependencies. Therefore, this invention first proposes a hybrid architecture (spatiotemporal feature extraction module) combining a two-dimensional convolutional neural network (2D-CNN) and a two-dimensional temporal convolutional network (2DTCN). Considering the relatively low density of the EEG channels used in this invention, instead of arranging the channel data into an 8×9 matrix, this invention simply stacks the data from each channel vertically to form a two-dimensional dataset.

[0097] The proposed spatiotemporal feature extraction module (2DST) architecture aims to effectively capture spatial and temporal dependencies in sequence data. In a two-dimensional convolutional neural network, this module first uses a 3×3 convolutional kernel to perform sliding computation along the spatial dimensions (height and width) and feature fusion along the channel dimension, effectively extracting spatial features using local receptive fields to capture the association patterns between adjacent data points. Secondly, it consists of multiple hierarchically arranged convolutional blocks, each performing joint spatiotemporal feature extraction using two-dimensional dilated convolutions. Specifically, each convolutional block uses a two-dimensional convolutional layer with a configurable kernel size (default 3×3), operating along both spatial and temporal dimensions, with dilation applied only in the temporal dimension to progressively expand the receptive field and enhance the ability to model long-term dependencies. After each convolution, batch normalization, ReLU activation, and a dropout regularization layer (p=0.2) are added to stabilize training. The main 2DST structure stacks three such blocks with progressively increasing expansion rates (1, 2, 4) and channel dimensions (16, 32, 64), enabling the network to capture multi-scale temporal patterns while preserving spatial information. Finally, a 2×2 average pooling layer reduces both spatial and temporal dimensions. This architecture processes input tensors of shape [batch, channels, height, time] and outputs compressed spatiotemporal features, making it particularly suitable for tasks requiring joint modeling of spatial configuration and temporal dynamics.

[0098] This hybrid architecture fully leverages the complementary strengths of the two networks: 2D-CNN focuses on extracting spatial-frequency features and controlling sequence length, while 2DTCN addresses the modeling of long temporal dependencies. This collaborative approach reduces the computational complexity of the model while enhancing its ability to represent EEG signal features, laying a solid foundation for subsequent deep learning tasks.

[0099] The Transformer model consists of an encoder and a decoder. The task of this invention is to capture sequence features and perform recognition and classification; therefore, no task generation is required, and this invention only uses the encoder part of the Transformer.

[0100] The Transformer encoding module is responsible for converting the input sequence into a series of hidden representations. The Transformer encoding module consists of multiple identical Transformer encoding units stacked together. Each Transformer encoding unit layer contains two main sub-layers: a multi-head attention mechanism and a feedforward network (FFN), and each sub-layer uses residual connections and layer normalization (LN).

[0101] Specifically, the Transformer coding module includes multiple Transformer coding units connected in series. Each Transformer coding unit includes a position coding unit, a multi-head attention unit, a feedforward network, and a normalization layer connected in series.

[0102] The Transformer encoding module consists of multiple identical stacked layers used to transform the input sequence into a series of hidden representations. Each layer is equipped with residual connections. Multi-head attention units can focus on information at different locations in parallel across multiple subspaces, improving sequence modeling capabilities; the feedforward network performs non-linear mapping on the extracted features, further enhancing the model's understanding and classification performance of EEG signal context information.

[0103] In one embodiment, a global spatiotemporal dependency is established through dynamic learnable location encoding and a multi-head attention mechanism to determine the encoding features of the EEG signal. This includes: for any Transformer encoding unit, inputting spatiotemporal features into the Transformer encoding unit, encoding the spatiotemporal features using the location encoding matrix in the location encoding unit to obtain location encoding features; extracting attention from the location encoding features using the attention mechanism in the multi-head attention unit to obtain attention features; processing the attention features using a feedforward network to obtain feedforward features; adding the feedforward features and attention features to obtain fused features; and performing a nonlinear transformation on the fused features using a normalization layer to obtain encoding features.

[0104] Specifically, the multi-head attention unit includes a multi-head attention mechanism, which computes multiple self-attention heads in parallel, projects the query (Q), key (K), and value (V) vectors to h subspaces (i.e., multiple "heads") respectively, and obtains the final output attention features through linear transformation.

[0105] The feedforward network performs a non-linear mapping of features obtained from the multi-head attention mechanism, which helps the model capture unique features in the input sequence. Residual connections are also used between these layers. This architecture is designed to fully leverage the self-attention mechanism and feedforward network in the Transformer model to effectively capture contextual information in EEG signals and improve classification performance.

[0106] In actual clinical trials, the length of EEG recordings varies greatly. To effectively capture EEG sequence information, this invention proposes a Dynamically Learnable Position Encoding Mechanism (DLPE) to replace traditional position encoding. This module, while retaining the alternating pattern of using sine and cosine functions, introduces adaptive frequency and phase parameters, enabling parameter adjustment. Unlike the original Transformer's fixed-frequency position encoding, DLPE automatically learns the optimal position representation pattern through end-to-end training, while retaining dynamic support for sequences of arbitrary length, thus overcoming the limitations of fixed frequencies and static position encoding.

[0107] This module contains two core learnable parameters: the logarithmic frequency parameter (log_freq) and the phase offset parameter (phase). The former controls the logarithmic transform parameter at the position-encoded wavelength scale, initialized to the original Transformer's reference frequency (10,000). A negative exponential mapping ensures the frequency is always positive. The latter introduces a learnable phase offset for each frequency component, initialized to zero to maintain initial consistency with standard sinusoidal coding.

[0108] During the calculation process, for an input sequence of length L, the module generates an L×d position encoding matrix in real time. The elements in the position encoding matrix are calculated as follows:

[0109] (6)

[0110] (7)

[0111] in, This represents the position at position pos in the position encoding matrix. dimensional features, This represents the position at position pos in the position encoding matrix. dimensional features, and These represent the learnable frequency and phase parameters, respectively. d This indicates the dimension of the position encoding matrix.

[0112] This invention replaces the traditional normalization layer with DyT technology, enabling the Transformer to maintain comparable or better performance across various tasks. The calculation formula for the normalization layer is:

[0113] (8)

[0114] in, Represents normalized features. For learnable scalar parameters, , All are channel-level affine parameters. This is a feature of fusion.

[0115] S104, classify the encoded features to obtain the emotion recognition results of the user to be identified.

[0116] The classification module includes a KAN classifier; the KAN classifier consists of a two-layer KAN network architecture.

[0117] Optionally, the encoded features are classified to obtain the emotion recognition result of the user to be identified, including: inputting the encoded features into the classification module, classifying the encoded features through the KAN classifier, and obtaining the emotion recognition result.

[0118] KAN abandons the linear weight matrix and fixed activation function structure of the multilayer perceptron, instead using a more flexible spline function for fitting. During training, the KAN network does not need to optimize the entire multivariate space; it only needs to adjust the number of training parameters and the fineness of the data using grids and splines. This effectively reduces the number of parameters while enhancing the model's ability to learn subtle patterns. Furthermore, KAN networks perform better in tasks that require understanding and interpreting fundamental physical principles.

[0119] This invention uses a KAN network as a classifier to optimize the connection patterns and structure of the neural network. The expression for the two-layer KAN network architecture in the classifier is shown in the equation.

[0120] (9)

[0121] (10)

[0122] (11)

[0123] (12)

[0124] in, x for n 3D input vector; It is an internal function; It is an external function; Basic functions; It is the Swish activation function, a variant of the sigmoid function; It is a spline function; These are the spline parameters; For predefined basic spline functions; As the activation function, it is and The sum, These are trainable parameters.

[0125] It should be noted that the classification module can also be a multilayer perceptron, which can be used to achieve classification.

[0126] In one embodiment, such as Figure 2 As shown, Figure 2 This is a schematic diagram of the overall process of an EEG signal emotion recognition method with multi-feature fusion and adaptive step size. It extracts EEG signals from four channels (F3, F4, T7, T8) across multiple frequency bands (θ, α, β, γ), calculates the adaptive step size, and performs signal truncation based on this step size. Based on the truncated signal, it calculates the differential entropy (DE), which is then input into the emotion recognition model. A spatiotemporal feature extraction module extracts spatiotemporal features, and a Transformer encoding module extracts the encoded features of the EEG signals. Finally, a KAN classifier is used for classification to obtain the emotion recognition result. Figure 2 In this model, the spatiotemporal features include a two-dimensional convolutional neural network and a two-dimensional temporal convolutional network connected in series. The two-dimensional convolutional neural network includes a 16-channel convolutional layer, a batch normalization layer, and a ReLU (Rectified Linear Unit) connected in series. The ReLU (Rectified Linear Unit) is the activation function layer. The two-dimensional temporal convolutional network includes three convolutional blocks (module 1) connected in series and a 2D average pooling layer. The Transformer coding module includes multiple Transformer coding units (DLPE-Transformer) connected in series.

[0127] Optionally, the classification module may also include two cascaded multilayer perceptrons, each including a fully connected layer. Specifically, this invention utilizes two fully connected layers as the final output for calculation. All outputs from the transformer encoding module are first flattened into a data format acceptable to the fully connected layer for connection calculation, as shown in the following formula:

[0128] (13)

[0129] These are the weights of the fully connected layer. For bias parameters, The activation function is the Sigmoid function, which is used for final classification. The Sigmoid formula is shown below:

[0130] (14)

[0131] like Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of each module in the emotion recognition model; Figure 3 Figure (a) shows a schematic diagram of the feedforward network and normalization layer (DyT) in the Transformer coding unit. Figure 3 Figure (b) in the diagram is a schematic diagram of the structure of a multilayer perceptron. Figure 3 Figure (c) in the diagram is a schematic diagram of a two-layer KAN network architecture.

[0132] In one specific embodiment, the SEED dataset is a publicly available emotion dataset constructed by a university's Brain and Computational Science Laboratory (BCMI), primarily for research in the fields of affective computing and brain-computer interfaces. This dataset contains EEG (electroencephalogram) and eye-tracking data of 15 Chinese subjects (7 males and 8 females, mean age 23.27 years) while watching emotion-inducing videos. Each subject was asked to watch approximately 4 minutes of 15 video clips covering positive, neutral, and negative emotions. These clips were carefully edited to effectively evoke the target emotion, with 5 clips corresponding to each emotion category. EEG recordings were acquired using the ESI NeuroScan system with a 62-channel electrode distribution, following the international 10-20 standard system. The dataset underwent preprocessing, including downsampling to 200Hz and applying a 0-75Hz bandpass filter to remove artifacts. This invention uses a three-class classification method to evaluate the model's emotion recognition performance on the SEED dataset.

[0133] Electroencephalogram (EEG) signals are typically acquired through high-density data collection. However, processing high-density data often faces problems such as excessive computational load, high resource consumption, and long processing time, significantly reducing the efficiency of feature extraction. To address this challenge, this invention focuses on exploring which channels are more sensitive to emotional changes under different emotional states. The aim is to optimize the data processing flow by selecting key channels, thereby improving the efficiency of feature extraction and the overall performance of the model.

[0134] From a neurobiological perspective, emotional processing involves the synergistic action of multiple brain regions, with the prefrontal cortex playing a central regulatory role and exhibiting significant hemispheric lateralization. According to Davidson's theory of frontal asymmetry, the left prefrontal cortex (e.g., F3, AF3) is associated with positive emotions, while the right prefrontal cortex (e.g., F4, AF4) is more involved in processing negative emotions. Temporal lobe regions (e.g., T7, T8) play a prominent role in emotional memory and high arousal states. Existing studies have used feature analysis and empirical data analysis to identify several key pathways, validating the brain region localization consistency of the above theory.

[0135] Regarding neural oscillations, different frequency bands exhibit region specificity: theta and alpha bands are mainly distributed in the temporal and parietal lobes, participating in emotion and attention regulation; the beta band reflects cortical excitation in the prefrontal, parietal, and central regions; and the gamma band is closely related to high arousal emotions in the prefrontal, occipital, and T8 regions. One study found that the beta band can achieve classification accuracy close to that of multi-channel bands in a single-channel setup, indicating that it contains rich emotional information.

[0136] Based on the aforementioned neural mechanisms and frequency band characteristics, this invention constructs an emotion recognition model that integrates four-channel (F3, F4, T7, T8) and multi-band (θ, α, β, γ) features. This model simplifies electrode configuration while effectively preserving neural information closely related to emotional states. In summary, this invention constructs an emotion recognition model based on the fusion of four-channel (F3, F4, T7, T8) and multi-band (θ, α, β, γ) features. This model design simplifies electrode configuration (conforming to the principle of simplified montage) while maximizing the preservation of key neural oscillation information through multi-band feature extraction. This modeling method based on neuroscience theory not only improves classification performance but also ensures good physiological interpretability, providing a new methodological reference for EEG emotion recognition research. The research results deepen the understanding of the neural mechanisms of emotion and lay a theoretical foundation for developing emotion computing systems based on physiological signals.

[0137] In this invention, the optimal time window size is selected by comparing different time window sizes, and the maximum and minimum step sizes of the sliding window are set to 2s and 1s, respectively. In the dynamically positioned Transformer model, this invention will... The parameter is set to 64. The learning rate is set to 4. The fully connected layers for the classification task are designed as an FC1-1000 structure. To accelerate model convergence, the Adam optimizer is used for training, and the learning rate is set to [value missing]. Furthermore, to effectively prevent overfitting, a Dropout mechanism is introduced, and the random dropout rate is set to 0.4 to further improve the model's generalization ability.

[0138] To effectively prevent errors such as learning effects and fatigue effects, and thus ensure the independence and reliability of the data, this invention employs independent experiments with individual subjects. First, features such as DE (deductive reasoning) are extracted, and leave-one-out cross-validation is used to ensure that each subject is used solely as test data, thereby minimizing the impact of individual differences on the experimental results.

[0139] To verify the interpretability of EEG emotion recognition based on neuroscience theory discussed in the channel selection phase, this invention explores the following two aspects.

[0140] like Figure 4As shown, subjects 10 exhibited significant differences in the sliding window step size distribution across four frequency bands (θ, α, β, γ), with the θ band showing the most significant step size variation, suggesting that this band may be less sensitive to emotional fluctuations than other bands. Furthermore, neuroscientific analysis indicates that these differences in step size distribution across frequency bands are physiologically significant. The high coefficient of variation in the θ band reflects its association with the limbic system (anterior cingulate cortex and hippocampus). The stability of the α / β bands aligns with the rhythmic characteristics of the sensorimotor cortex, while the high temporal precision of the γ band supports its role in the integration of emotional information across brain regions. These results validate the aforementioned theory of neural oscillation hierarchy, namely that different frequency bands collaboratively participate in emotional processing through specific time scales.

[0141] Figure 5 Using EEG scalp topography of subjects 10 across four frequency bands, the DE distribution patterns of the prefrontal (F3 / F4) and temporal (T7 / T8) channels were shown to be consistent with theories of neural mechanisms of emotional processing. The Alpha band showed lower DE values ​​in the left prefrontal (F3) lobe than in the right prefrontal (F4) lobe, consistent with Davidson's theory of frontal lateralization. The Gamma band showed higher DE values ​​in the right temporal lobe (T8), supporting the correlation between high-frequency temporal lobe activity and high-arousal emotions. The Beta band activity in the prefrontal (F3 / F4) lobe reflected cortical excitation, while the higher DE values ​​in the Delta band in the temporal lobe may correspond to emotional memory processing.

[0142] Based on the requirements of the research task and the characteristics of EEG data, the selection of window length (T) is also a key step. Therefore, to explore the impact of T on the experimental results of the SEED dataset, this invention verified the results through experiments. T = 2, 3, 4, 5, 6 were selected respectively to divide the EEG samples accordingly, and the samples were input into the emotion recognition model. Leave-one-out cross-validation was used for the experiment, and the experimental results are shown in Table 1.

[0143] Table 1

[0144]

[0145] The results show that using the adaptive step-size window sliding technique can reduce the impact of window size on classification accuracy. The classification performance is optimal at T = 3, and the number of windows is less than that at T = 2, thus reducing computational cost. Therefore, in subsequent experiments, this invention uses a 3-second time window for segmentation to ensure optimized model performance.

[0146] Furthermore, to comprehensively verify the performance of the proposed method (MFAS-2DST-DTK) in the emotion recognition and classification task, experimental data was compared with current state-of-the-art (SOTA) methods. Table 2 shows the comparison results of this model with advanced methods such as Domain Adversarial Neural Networks (DANN), Multilayer Perceptron (MLP), Bidirectional Long Short-Term Memory (BiLSTM), Graph Neural Network (GNN), and Transformer on the SEED dataset. The data shows that MFAS-2DST-DTK achieves good accuracy on the SEED data, improving the accuracy by 5.58% compared to the 1D-CNN-GRU-BiLSTM model on 4 channels.

[0147] Table 2

[0148]

[0149] To better evaluate the contribution of each module to the MFAS-2DST-DT model, an ablation experiment was conducted on the SEED dataset. Specifically, key modules were removed from the MFAS-2DST-DTM network in sequence: (1) the DTM module was removed, leaving only 2DST for feature extraction and classification; (2) the positional encoding module was removed; (3) DLPE was replaced with the traditional positional encoding method; (4) the MFAS module (the adaptive step-size sliding window technique based on multi-feature fusion provided by this invention) was removed and replaced with a fixed sliding window parameter for EEG signal segmentation; (5) the original fully connected layer was replaced with an interpretable KAN network; and (6) the normalization layer in the Transformer was replaced with the DyT activation function. Table 3 shows the accuracy of the new classification task in these five cases.

[0150] Table 3

[0151]

[0152] As shown in Table 3, each module significantly impacts model performance. Removing the DTM module leads to a 3.11% decrease in accuracy; therefore, processing dynamic signals like EEG with long time spans requires utilizing DTM to dynamically capture long-distance temporal relationships. In contrast, introducing the KAN classifier not only improves accuracy but also enhances model interpretability, further validating KAN's application potential in EEG emotion recognition tasks. Furthermore, while replacing the normalization layer with DyT results in approximately a 2% decrease in accuracy, it improves computational efficiency, demonstrating its advantages in lightweight and efficient inference scenarios.

[0153] Finally, to verify the feasibility of using Dynamic Tanh (DyT) to replace the normalization layer in the Transformer, a comparative experiment was designed. Two network models were constructed, and data conforming to the network input format of this paper was randomly generated as test samples. By inputting the same data into different networks, Table 4 shows the time taken for a single-layer network and a complete network to complete 100 forward propagations.

[0154] Table 4

[0155]

[0156] Experimental results show that DyT layers significantly reduce computation time compared to traditional layer normalization (LN). This finding suggests that DyT may be a promising optimization option for network architecture design aimed at high-efficiency computing.

[0157] In EEG-based emotion recognition research, the optimized few-channel electrode configuration significantly reduces computational complexity. By combining bandpass filtering, differential entropy calculation, and window sliding techniques, discriminative features can be extracted from EEG signals, providing a reliable data foundation for neural network training and inference. This method not only improves model performance but also enhances the ability to capture the dynamic characteristics of signals, offering an effective solution for EEG signal analysis. Simultaneously, the low-channel adaptive step-size technique, through dynamic step-size adjustment, better adapts to the non-stationary characteristics of EEG signals, thereby improving signal analysis performance. Experimental results further validate the effectiveness of this method, demonstrating that the adaptive step-size technique improves the accuracy and robustness of EEG signal processing across various tasks. By rationally selecting the window length and combining it with the adaptive step-size technique, this invention provides an efficient and reliable solution for feature extraction and analysis of EEG signals. Furthermore, EEG signals exhibit strong temporal dependencies; the addition of dynamic position encoding enhances the model's ability to model temporal relationships in EEG signals and better adapts to different sampling rates and time scales, especially when handling long-range dependencies, compensating for the shortcomings of pure attention mechanisms in absolute position perception.

[0158] To replace normalization layers without increasing training costs, DyT avoids the overhead of calculating the mean α and variance σ² found in traditional normalization layers, and eliminates the need to store or update statistics, thus saving memory and bandwidth. Comparative analysis shows that it not only significantly reduces computational cost but also achieves essentially the same recognition accuracy. In terms of hardware deployment, DyT achieves higher computational efficiency, lower energy consumption, and smaller hardware resource footprint by eliminating statistical calculations, simplifying hardware instructions, and reducing memory requirements, while maintaining model performance in EEG emotion recognition.

[0159] This invention designs a multi-feature fusion adaptive step-size EEG signal emotion recognition method, dynamically adapting to the time-varying characteristics of EEG signals and individual differences, achieving multi-level spatiotemporal feature extraction, thereby improving the accuracy of emotion recognition. An adaptive step-size is introduced to enhance the analysis quality of EEG signals. The spatiotemporal feature extraction module realizes hierarchical modeling of spatial-local features, and the Transformer encoding module automatically optimizes the representation of positional information through Dynamically Learnable Positional Encoding (DLPE), improving the ability to model long-term dependencies. To verify the effectiveness of the method, this invention selects four channels (F3, F4, T7, and T8) using prior knowledge of neuroanatomy for validation. The results show that the emotion recognition model maintains high accuracy while verifying the feasibility of simplified montage in emotion monitoring. Compared with traditional high-density systems, the proposed portable method significantly improves practicality and deployment flexibility while maintaining high performance. Finally, it employs… Replace the normalization layer of the Transformer to simulate the nonlinear mapping behavior of the normalization layer with extremely low computational cost.

[0160] When applying the multi-feature fusion adaptive step-size EEG signal emotion recognition method provided by this invention, it is not necessary to rely on... Figure 1 The steps shown are executed in sequence. The specific execution order of each step can be determined as needed, and this invention does not impose any restrictions on it.

[0161] The above describes a multi-feature fusion adaptive step-size EEG signal emotion recognition method provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding multi-feature fusion adaptive step-size EEG signal emotion recognition device, which includes:

[0162] The acquisition module is used to acquire the EEG signals of the user to be identified; the EEG signals include four channels of EEG signals, each channel including four fixed frequency bands of EEG signals;

[0163] The determination module is used to acquire the energy, signal fluctuation intensity, and dominant frequency component of the EEG signal for each frequency band of any channel, and determine the adaptive step size based on the energy, signal fluctuation intensity, and dominant frequency component of the EEG signal.

[0164] The segmentation module is used to segment EEG signals with an adaptive step size using a sliding window technique to obtain multiple EEG signal segments, and calculate the differential entropy of each EEG signal segment; and form a sequence based on the differential entropy of all EEG signals in each frequency band; and stack the sequences of the four frequency bands in the four channels into three-dimensional data.

[0165] The extraction module is used to extract the spatial features of the three-dimensional data and expand the spatial features in the time dimension to obtain spatiotemporal features; based on the spatiotemporal features, a global spatiotemporal dependency relationship is established through position encoding and multi-head attention mechanism to determine the encoding features of the EEG signal;

[0166] The classification module is used to classify the encoded features to obtain the emotion recognition results of the user to be identified.

[0167] Specific limitations regarding the multi-feature fusion adaptive step size EEG signal emotion recognition device can be found in the limitations of the multi-feature fusion adaptive step size EEG signal emotion recognition method described above, and will not be repeated here. Each module in the aforementioned multi-feature fusion adaptive step size EEG signal emotion recognition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0168] The present invention also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 The proposed method is a multi-feature fusion adaptive step size EEG signal emotion recognition method.

[0169] The present invention also provides Figure 6 The schematic diagram of the computer device shown is as follows: Figure 6 As shown, at the hardware level, this computer device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then executes it to achieve the above. Figure 1 The proposed method is a multi-feature fusion adaptive step size EEG signal emotion recognition method.

[0170] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0171] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this invention.

Claims

1. A method for emotion recognition using EEG signals with multi-feature fusion and adaptive step size, characterized in that, include: Acquire the brainwave signals of the user to be identified; The EEG signal includes four channels of EEG signal, with each channel containing four fixed frequency bands of EEG signal; For each frequency band of the EEG signal in any channel, the energy, signal fluctuation intensity, and dominant frequency component of the EEG signal are acquired, and the adaptive step size is determined based on the energy, signal fluctuation intensity, and dominant frequency component of the EEG signal. The EEG signal was segmented with an adaptive step size using the sliding window technique to obtain multiple segments of the EEG signal, and the differential entropy of each segment of the EEG signal was calculated. The sequences are formed based on the differential entropy of all EEG signals in each frequency band; and the sequences of the four frequency bands in the four channels are stacked into three-dimensional data. Spatial features of 3D data are extracted and extended in the time dimension to obtain spatiotemporal features. Based on the spatiotemporal features, a global spatiotemporal dependency is established through dynamic learnable position coding and multi-head attention mechanism to determine the coding features of EEG signals. The encoded features are classified to obtain the emotion recognition results of the user to be identified; The adaptive step size is determined based on the energy, signal fluctuation intensity, and dominant frequency component of the EEG signal. This includes: assigning weights to the energy, signal fluctuation intensity, and dominant frequency component of the EEG signal to determine the adjustment factor; adjusting the base step size proportionally according to the adjustment factor to obtain the adjustment step size; and superimposing a uniformly distributed random perturbation on the adjustment step size to obtain the adaptive step size. The formula for calculating the adjustment factor is: in, Indicates the regulating factor. Indicates energy. Indicates the intensity of signal fluctuations. Indicates the main frequency component; The formula for calculating the adaptive step size is: in, Indicates adaptive step size, Indicates the adjustment step size. Indicates the basic step size. Indicates the range of fluctuation Uniform randomization within, As a disturbance factor, Indicates the minimum limit value. Indicates the maximum limit value. Indicates will Limited to Within the range.

2. The method according to claim 1, characterized in that, Emotion recognition results are obtained through an emotion recognition model; the emotion recognition model includes a spatiotemporal feature extraction module, a Transformer encoding module, and a classification module connected in sequence. The spatiotemporal feature extraction module includes a two-dimensional convolutional neural network and a two-dimensional temporal convolutional network connected in series. The two-dimensional convolutional neural network includes a convolutional layer, a batch normalization layer, and an activation function layer connected in series. The two-dimensional temporal convolutional network includes three convolutional blocks connected in series and an average pooling layer. Each convolutional block includes a two-dimensional dilated convolutional layer, a batch normalization layer, an activation function layer, and a regularization layer connected in series. The dilation rate and the number of channels of the three convolutional blocks are different. The Transformer coding module includes multiple Transformer coding units connected in series. Each Transformer coding unit includes a position coding unit, a multi-head attention unit, a feedforward network, and a normalization layer connected in series. The classification module includes a KAN classifier; the KAN classifier consists of a two-layer KAN network architecture.

3. The method according to claim 2, characterized in that, Spatial features are extracted from 3D data and then extended along the time dimension to obtain spatiotemporal features, including: Three-dimensional data is input into a two-dimensional convolutional neural network, and then passed through a convolutional layer, a batch normalization layer, and an activation function layer in sequence to obtain the spatial features of the three-dimensional data; Spatial features are input into a two-dimensional temporal convolutional network and passed through three convolutional blocks and an average pooling layer in sequence to obtain spatiotemporal features.

4. The method according to claim 2, characterized in that, By establishing global spatiotemporal dependencies through dynamic learnable location encoding and multi-head attention mechanisms, the encoded features of EEG signals are determined, including: For any Transformer coding unit, the spatiotemporal features are input into the Transformer coding unit, and the spatiotemporal features are positionally encoded by the position encoding matrix in the position encoding unit to obtain the position encoded features; Attention features are obtained by extracting attention from position-encoded features using the attention mechanism in a multi-head attention unit. The attention features are processed by a feedforward network to obtain feedforward features; The feedforward features and attention features are added together to obtain the fused features; The fused features are nonlinearly transformed by a normalization layer to obtain the encoded features.

5. The method according to claim 4, characterized in that, The elements in the position encoding matrix are calculated as follows: in, This represents the position at position pos in the position encoding matrix. Dimensional features, This represents the position at position pos in the position encoding matrix. Dimensional features, and These represent the learnable frequency and phase parameters, respectively. d This indicates the dimension of the position encoding matrix.

6. The method according to claim 2, characterized in that, The encoded features are classified to obtain the emotion recognition results of the user to be identified, including: The encoded features are input into the classification module, and the KAN classifier is used to classify the encoded features to obtain the emotion recognition result.

Images