Device and method for determining emotion category

Abstract

An embodiment of the present invention provides a device and method for determining emotion categories. The device includes: an acquisition unit for acquiring electroencephalogram (EEG) signals; a time-frequency feature extraction unit comprising two consecutive two-dimensional convolutional layers for extracting features from each channel of the EEG signals to obtain a time-frequency feature map; a symmetric channel feature extraction unit comprising two convolutional layers with multiple convolution kernels for extracting emotion lateralization features from the time-frequency feature map to obtain a symmetric channel feature map; a spatial feature extraction unit for extracting spatial features from the symmetric channel feature map to obtain a spatial feature map; a fully connected unit comprising multiple neurons for flattening the spatial feature map to obtain neurons with emotion category confidence; and a processing unit for discriminating emotion categories based on the neurons with emotion category confidence. This device fully utilizes brain lateralization features, improving learning efficiency and the accuracy of emotion determination.

Description

Technical Field

[0001] The present invention relates to the field of electroencephalogram (EEG) signal recognition, and in particular to a device and method for determining emotion categories. Background Art

[0002] There are many existing methods for emotion recognition using EEG signals. These typically involve extracting EEG features and then feeding them into a machine learning classifier. Traditional feature extractors and classifiers have reached their limits. For example, using data from all channels can introduce redundant workloads, increasing operational complexity and making the model "bloated." This results in suboptimal performance in emotion recognition, significantly limiting its application. Summary of the Invention

[0003] The purpose of the embodiments of the present invention is to provide a device and method for determining emotion categories, which fully utilizes the lateralization characteristics of the brain to improve learning efficiency and the accuracy of emotion determination.

[0004] In order to achieve the above-mentioned purpose, an embodiment of the present invention provides a device for determining emotion categories, which includes: an acquisition unit for acquiring an EEG signal; a time-frequency feature extraction unit, including at least two consecutive two-dimensional convolution layers, for extracting features of each channel of the EEG signal to obtain a time-frequency feature map; a symmetric channel feature extraction unit, including at least two convolution layers with multiple convolution kernels, for extracting emotion lateralization features of the time-frequency feature map to obtain a symmetric channel feature map; a spatial feature extraction unit, including at least two layers of two-dimensional convolution blocks with multiple convolution kernels, for extracting spatial features of the symmetric channel feature map to obtain a spatial feature map; a fully connected unit, including multiple neurons, for flattening the spatial feature map to obtain neurons with emotion category confidence; and a processing unit, for discriminating the emotion category based on the neurons with emotion category confidence.

[0005] Optionally, the emotion category is determined based on the neuron with emotion category confidence, including: the output values of the neuron with emotion category confidence include: 0, 1 and 2; if the output value of the neuron with emotion category confidence is 0, the corresponding emotion category is sadness; if the output value of the neuron with emotion category confidence is 1, the corresponding emotion category is calm; if the output value of the neuron with emotion category confidence is 2, the corresponding emotion category is happiness.

[0006] Optionally, the at least two consecutive two-dimensional convolutional layers include at least: a first convolutional layer and a second convolutional layer; the convolution kernel sizes of the first convolutional layer and the second convolutional layer are both 1×100, and the heights of the convolution kernels are both 1.

[0007] Optionally, the at least two convolutional layers include at least a third convolutional layer and a fourth convolutional layer; the size of the convolution kernel of the third convolutional layer is 2×100, the horizontal step length of the convolution kernel is 1, and the vertical step length is 2; the fourth convolutional layer is a depth-separable convolution, the size of the convolution kernel of the fourth convolutional layer is 1×100, and the height of the convolution kernel is 1.

[0008] Optionally, the height of the convolution kernel in the spatial feature extraction unit is the total number of channels of the electroencephalogram divided by 2, and the width of the convolution kernel is 1.

[0009] Optionally, the device further comprises: a training unit for training the time-frequency feature extraction unit, the symmetric channel feature extraction unit and the spatial feature extraction unit to update the parameters of each extraction unit, including adopting an initial learning rate of 10 -4 The optimizer optimizes the loss function that minimizes the cross entropy, and uses cosine annealing to optimize the learning rate.

[0010] On the other hand, the present invention provides a method for determining emotion categories, which includes: obtaining an EEG signal; subjecting the EEG signal to at least two consecutive two-dimensional convolution layers to extract features of each channel of the EEG signal to obtain a time-frequency feature map; subjecting the time-frequency feature map to at least two convolution layers with multiple convolution kernels to extract emotion lateralization features of the time-frequency feature map to obtain a symmetric channel feature map; subjecting the symmetric channel feature map to at least two layers of two-dimensional convolution blocks with multiple convolution kernels to extract spatial domain features of the symmetric channel feature map to obtain a spatial domain feature map; flattening the spatial domain feature map to obtain neurons with emotion category confidence; and discriminating the emotion category based on the neurons with emotion category confidence.

[0011] Optionally, the discriminating the emotion category based on the neuron with emotion category confidence includes: determining the classification of the emotion category based on the polarity of the emotion category confidence; the output values of the neuron with emotion category confidence include: 0, 1 and 2; if the output value of the neuron with emotion category confidence is 0, the corresponding emotion category is sadness; if the output value of the neuron with emotion category confidence is 1, the corresponding emotion category is calm; if the output value of the neuron with emotion category confidence is 2, the corresponding emotion category is happiness.

[0012] Optionally, the two consecutive two-dimensional convolutional layers include at least: a first convolutional layer and a second convolutional layer; the convolution kernel size of the first convolutional layer and the second convolutional layer is 1×100, and the height of the convolution kernel is 1.

[0013] Optionally, the at least two convolutional layers having multiple convolution kernels include at least: a third convolutional layer and a fourth convolutional layer; the size of the convolution kernel of the third convolutional layer is 2×100, the horizontal step length of the convolution kernel is 1, and the vertical step length is 2; the fourth convolutional layer is a depthwise separable convolution, the size of the convolution kernel of the fourth convolutional layer is 1×100, and the height of the convolution kernel is 1.

[0014] Optionally, the height of the convolution kernel of the two-dimensional convolution block is the total number of channels of the electroencephalogram divided by 2, and the width of the convolution kernel is 1.

[0015] Optionally, the method further includes: optimizing the loss function that minimizes cross entropy using an optimizer with an initial learning rate of 10-4, and optimizing the learning rate using cosine annealing.

[0016] A device for determining emotion category according to the present invention includes: an acquisition unit for acquiring electroencephalogram (EEG) signals; a time-frequency feature extraction unit comprising at least two consecutive two-dimensional convolutional layers for extracting features from each channel of the EEG signals to generate a time-frequency feature map; a symmetric channel feature extraction unit comprising at least two convolutional layers with multiple convolution kernels for extracting emotion lateralization features from the time-frequency feature map to generate a symmetric channel feature map; a spatial feature extraction unit comprising at least two layers of two-dimensional convolutional blocks with multiple convolution kernels for extracting spatial features from the symmetric channel feature map to generate a spatial feature map; a fully connected unit comprising multiple neurons for flattening the spatial feature map to generate neurons with emotion category confidence; and a processing unit for discriminating emotion category based on the neurons with emotion category confidence. This device significantly reduces redundant computation by fully utilizing brain lateralization features and eliminating feature pre-extraction characteristics. It also requires no excessive number of electrodes for EEG signal acquisition and is wearable. The accuracy on the public emotion dataset SEED is significantly improved, enhancing learning efficiency and the accuracy of emotion determination.

[0017] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following detailed description, they are used to explain the embodiments of the present invention, but do not constitute a limitation of the embodiments of the present invention. In the accompanying drawings:

[0019] Figure 1 is a schematic diagram of a device for determining emotion categories according to the present invention;

[0020] Figure 2-Figure 4 It is the convolution training process of the present invention;

[0021] Figure 5 This is a schematic diagram of a flow chart for determining emotion categories according to the present invention;

[0022] Figure 6-Figure 7 Schematic diagram comparing the present invention and the prior art.

[0023] Description of Reference Numerals

[0024] 100-Get unit;

[0025] 101-time-frequency feature extraction unit;

[0026] 102- symmetric channel feature extraction unit;

[0027] 103-spatial feature extraction unit;

[0028] 104- fully connected unit;

[0029] 105-Processing unit. DETAILED DESCRIPTION

[0030] The following describes the specific implementation of the embodiment of the present invention in detail with reference to the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the embodiment of the present invention and is not used to limit the embodiment of the present invention.

[0031] Through research, the inventors discovered that existing emotion recognition models based on EEG signals ignore the limitations of spatial representation. When people are emotionally aroused, only certain areas of the cerebral cortex respond laterally. Using data from all channels can introduce redundant workload, increasing operational complexity and making the model "bulky." Furthermore, this method requires too many electrodes to acquire EEG signals, making them practically unwearable, which significantly limits their practical applications.

[0032] Figure 1 Schematic diagram of a device for determining emotion categories of the present invention, such as Figure 1 As shown, an embodiment of the present invention provides a device for determining emotion categories. The present invention first selects EEG (ElectroEncephaloGram) channels that are closely related to emotional responses and rearranges symmetric channels in adjacent positions in an array. It then proposes an end-to-end convolutional neural network based on neuroscience prior design. The neural network includes a time-frequency feature extraction unit, a symmetric channel feature extraction unit, and a spatial feature extraction unit. The EEG (ElectroEncephaloGram) is an electroencephalogram (EEG).

[0033] According to a specific embodiment, the device for determining emotion categories includes: an acquisition unit 100, a time-frequency feature extraction unit 101, a symmetric channel feature extraction unit 102, a spatial feature extraction unit 103, a fully connected unit 104, and a processing unit 105.

[0034] Specifically, the acquisition unit 100 is used to acquire an electroencephalogram (EEG). The EEG signal is an electroencephalogram (EEG). Specifically, the data in the EEG may include at least one of average amplitude, peak value, latency, waveform, and topography. According to a specific embodiment, the EEG records the electrical activity of the brain via electrodes, which are typically embedded in an electrode cap. Generally, such an electrode cap takes 5 minutes to install on the user and adjust the electrodes to obtain a good signal.

[0035] The time-frequency feature extraction unit 101 includes at least two consecutive two-dimensional convolutional layers, which are used to extract features from each channel of the EEG signal to generate a time-frequency feature map. The present invention uses two consecutive two-dimensional convolutional layers to calculate the signal for each EEG channel in a deep learning framework. Convolutional neural networks are a type of feedforward neural network. They were proposed based on the biological mechanism of receptive fields. Receptive fields primarily refer to certain properties of neurons in the auditory, proprioceptive, and visual systems. For example, in the visual system, the receptive field of a neuron refers to a specific area on the retina; only stimuli within this area can activate the neuron. Convolutional neural networks have three structural characteristics: local connectivity, weight sharing, and spatial or temporal subsampling. A two-dimensional convolutional layer performs a sliding window operation on a feature map in both the width and height directions, multiplying and summing the corresponding positions. In contrast, the convolution in a one-dimensional convolutional layer is different; the convolution kernel is also two-dimensional, but the sliding window is performed in a single direction, either width or height, and the multiplication and summation are performed.

[0036] In a two-dimensional convolutional layer, a two-dimensional input array and a two-dimensional kernel array are cross-correlated to produce a two-dimensional output array. Specifically, in this two-dimensional cross-correlation operation, the convolution window begins at the top left of the input array and slides across the input array from left to right and top to bottom. When the convolution window reaches a certain position, the input sub-array in the window is element-wise multiplied with the kernel array and the sum is calculated to produce the element at the corresponding position in the output array. The two-dimensional convolutional layer performs a cross-correlation operation on the input and the convolution kernel, and adds a scalar bias to produce the output. The convolutional layer model parameters include the convolution kernel and the scalar bias.

[0037] The at least two consecutive two-dimensional convolutional layers in the time-frequency feature extraction unit include at least: a first convolutional layer and a second convolutional layer; the convolution kernel sizes of the first convolutional layer and the second convolutional layer are both 1×100, and the heights of the convolution kernels are both 1.

[0038] The technical benefits of the time-frequency feature extraction unit can include the following aspects: From the perspective of convolution technology, time-frequency domain feature extraction is a one-dimensional operation on tensors. The present invention sets the convolution kernel height to 1, allowing it to extract features only from each channel and ensuring that the kernel focuses solely on time-frequency domain features. From a frequency perspective, since emotions are closely related to high-frequency EEG, in EEGNet (electroencephalogram network), setting the convolution kernel size to half the sampling rate can capture frequency information above 2Hz. From a time-frequency perspective, long convolution kernels are more capable of learning long-term series features. For EEG data sets with long-term series and low signal-to-noise ratio, L1 and L2 regularization are used in the loss function calculation of the first convolutional layer. L1 and L2 are regularization terms, also known as penalty terms, which are added to the loss function to limit model parameters and prevent overfitting. Specifically, L1 obtains the sum of the absolute values of each model parameter, and L2 obtains the square root of the sum of the squares of each model parameter. L1 tends to produce a small number of features, with all other features being 0. Its optimal parameter value is likely to appear on the coordinate axis, where the weight of one dimension is 0, resulting in a sparse weight matrix. L2 selects more features, and these features are close to 0.

[0039] L1 and L2 regularization are penalty terms added to the loss function. The added formulas are as follows:

[0040]

[0041]

[0042] J0 represents the original loss function, α and λ represent the regularization coefficients, and ω represents the feature coefficient.

[0043] L1 regularization can remove features that have no contribution or very little contribution to the extraction of time-frequency feature units, thereby generating a sparse weight matrix. L2 regularization can constrain the convolutional layer parameters in the time-frequency feature extraction unit, preventing overfitting problems while avoiding the problem of model generalization differences.

[0044] Emotions are the brain's response to stimuli, and the collected EEG signals lag behind. Therefore, this invention pads the edges of the EEG to prevent EEG signal loss at the end of the sequence. The first and second convolutional layers have 8 and 16 filters, respectively. By gradually increasing the network depth, this invention prevents convolution filtering from overly focusing on noise in the initial features.

[0045] Following the convolutional layer, an average pooling layer is connected. (Average pooling divides the image into fixed-size grids and takes the average value of all pixels within the grid. Pooling is a key operation in convolutional neural networks.) This is similar to downsampling, or decimation, and is used for multi-rate signal processing. Average pooling reduces computational and memory requirements while preserving background information. Compared to maximum pooling, average pooling at the beginning of convolution improves model generalization and prevents subsequent feature extraction modules from being limited by prominent artifacts.

[0046] The symmetric channel feature extraction unit 102 includes at least two convolutional layers with multiple convolution kernels, which are used to extract the emotional lateralization features of the time-frequency feature map to obtain a symmetric channel feature map. The two convolutional layers with multiple convolution kernels include at least a third convolutional layer and a fourth convolutional layer; the convolution kernel of the third convolutional layer has a size of 2×100, a horizontal stride of 1, and a vertical stride of 2; the fourth convolutional layer is a depthwise separable convolution, and the convolution kernel of the fourth convolutional layer has a size of 1×100 and a height of 1.

[0047] Specifically, the symmetric channel feature extraction unit is based on a rearranged array, and the third convolutional layer of this module focuses on extracting horizontal features. Due to the vertical stride, the convolution kernel performs channel hopping, skipping convolution sites for asymmetric channels to avoid weight sharing, and only performing convolution on symmetric channels. The setting of the convolution kernel length means that secondary time-frequency features are captured simultaneously in the time-frequency feature map. After this convolution layer, the number of feature channels is halved.

[0048] The fourth convolutional layer is a depthwise separable convolution, which consists of two main steps: depthwise convolution and pointwise convolution. Specifically, depthwise convolution computes feature values, changing the conventional convolution operation from considering both channels and regions to first considering only regions and then channels, achieving channel-region separation. Pointwise convolution performs a 1x1 convolution to extract features at a single point. The main benefits of separable convolution include: the number of parameters is approximately one-third of that of depthwise separable convolution, and it explicitly decouples relationships within and across feature maps. When applied to EEG, this operation learns to separate summarizing individual feature maps (depthwise convolution) and optimizing combined feature maps (pointwise convolution). The convolution kernel size matches that of the time-frequency domain module, meaning that the fused time-frequency and laterality features are extracted from a single original channel. Finally, this module is connected to a max pooling layer. The max pooling layer calculates the maximum value in the pooled data as its output.

[0049] The spatial feature extraction unit 103 includes at least two layers of two-dimensional convolution blocks with multiple convolution kernels, which are used to extract spatial features from the symmetrical channel feature map to obtain a spatial feature map. The spatial domain is the pixel domain, and processing in the spatial domain is equivalent to processing at the pixel level. For example, after Fourier transform of pixel-level image superposition, the resulting image spectrum is represented by the energy gradient of the image. The height of the convolution kernel in the spatial feature extraction unit is the total number of EEG channels divided by 2, and the convolution kernel width is 1.

[0050] The fully connected unit 104 includes multiple neurons that flatten the spatial feature map to obtain neurons with confidence scores for emotion categories, and output neurons corresponding to the number of categories. The flattening process removes redundant, heavy, and complex decorative effects. Specifically, it removes unnecessary perspective, texture, gradient, and 3D effect elements, thereby allowing the "information" itself to be highlighted as the core.

[0051] The processing unit 105 is configured to discriminate an emotion category based on the neurons with emotion category confidence. The discriminative emotion category based on the neurons with emotion category confidence includes: output values of the neurons with emotion category confidence include: 0, 1, and 2; if the output value of the neurons with emotion category confidence is 0, the corresponding emotion category is sadness; if the output value of the neurons with emotion category confidence is 1, the corresponding emotion category is calm; and if the output value of the neurons with emotion category confidence is 2, the corresponding emotion category is happiness.

[0052] The device for determining emotion categories further includes a training unit for training the time-frequency feature extraction unit, the symmetric channel feature extraction unit, and the spatial feature extraction unit to update parameters of each extraction unit.

[0053] According to a preferred embodiment, Figure 2-Figure 4 This is the convolution training process of the present invention, such as Figure 2-4 As shown, the initial learning rate is 10 -4The optimizer optimizes the loss function to minimize cross-entropy and uses cosine annealing to optimize the learning rate. Adam (Adaptive moment estimation) is preferred. Entropy is used to measure the degree of disorder in a system and represents the total amount of information in the system; a larger entropy value indicates greater uncertainty in the system. Information is a measure of the uncertainty of a particular event, while entropy is a measure of the uncertainty of a system (all events). Cross-entropy is the sum of the entropy of the true value distribution and the KL-divergence. Specifically, first initialize the time-frequency feature extraction unit, the symmetric channel feature extraction unit, and the spatial domain feature extraction unit, give them an input vector and a target output, then obtain the output of each unit of the hidden layer and the output layer, then obtain the target value and the actual output bias, and determine whether the actual output bias is within the allowable range: when the actual output bias is not within the allowable range, calculate the error of the neurons in the network layer, then obtain the error gradient, and use it to update the weights, then re-obtain the output of each unit of the hidden layer and the output layer, until the actual output bias is within the allowable range, the training ends, and the weights and thresholds are fixed. Among them, the layers other than the input layer and the output layer are called hidden layers. The hidden layers do not directly receive external signals, nor do they directly send signals to the outside world. The hidden layers are black boxes in the neural network. The hidden layers abstract the features of the input data into another dimensional space to display its more abstract features, which can be better linearly divided.

[0054] The present invention also provides a method for determining emotion categories, which includes: obtaining an EEG signal; subjecting the EEG signal to at least two consecutive two-dimensional convolution layers to extract features of each channel of the EEG signal to obtain a time-frequency feature map; subjecting the time-frequency feature map to at least two convolution layers with multiple convolution kernels to extract emotion lateralization features of the time-frequency feature map to obtain a symmetric channel feature map; subjecting the symmetric channel feature map to at least two layers of two-dimensional convolution blocks with multiple convolution kernels to extract spatial domain features of the symmetric channel feature map to obtain a spatial domain feature map; flattening the spatial domain feature map to obtain neurons with emotion category confidence; and discriminating the emotion category based on the neurons with emotion category confidence.

[0055] The method of discriminating the emotion category based on the neurons with emotion category confidence includes: determining the classification of the emotion category based on the polarity of the emotion category confidence; the output values of the neurons with emotion category confidence include: 0, 1 and 2; if the output value of the neuron with emotion category confidence is 0, the corresponding emotion category is sadness; if the output value of the neuron with emotion category confidence is 1, the corresponding emotion category is calm; if the output value of the neuron with emotion category confidence is 2, the corresponding emotion category is happiness.

[0056] The two consecutive two-dimensional convolutional layers include at least: a first convolutional layer and a second convolutional layer; the convolution kernel sizes of the first convolutional layer and the second convolutional layer are 1×100, and the height of the convolution kernel is 1.

[0057] The at least two convolutional layers having multiple convolution kernels include at least a third convolutional layer and a fourth convolutional layer; the convolution kernel of the third convolutional layer has a size of 2×100, a horizontal stride of 1, and a vertical stride of 2; the fourth convolutional layer is a depthwise separable convolution, and the convolution kernel of the fourth convolutional layer has a size of 1×100 and a height of 1. The height of the convolution kernel of the two-dimensional convolution block is the total number of channels of the electroencephalogram divided by 2, and the convolution kernel width is 1.

[0058] The method for determining the emotion category further includes: optimizing the loss function that minimizes cross entropy using an optimizer with an initial learning rate of 10-4, and optimizing the learning rate using cosine annealing.

[0059] Figure 5 This is a flow chart of determining emotion categories of the present invention. According to a specific implementation method, Figure 5 As shown, step S501 is to obtain an EEG signal; step S502 is to subject the EEG signal to at least two consecutive two-dimensional convolutional layers to extract the features of each channel of the EEG signal and obtain a time-frequency feature map. The two consecutive two-dimensional convolutional layers include at least a first convolutional layer and a second convolutional layer; the convolution kernel size of the two consecutive two-dimensional convolutional layers is 1×100, and the convolution kernel height is 1.

[0060] The EEG signal is an electroencephalogram (EEG). Specifically, the data in the EEG may include at least one of average amplitude, peak value, latency, waveform, and topography. According to a specific embodiment, the EEG records the brain's electrical activity via electrodes, which are typically embedded in an electrode cap. Such an electrode cap typically takes five minutes to install on the user and adjust the electrodes to obtain a good signal.

[0061] Step S503 involves subjecting the time-frequency feature map to at least two convolutional layers with multiple convolution kernels to extract the emotional lateralization features of the time-frequency feature map and obtain a symmetric channel feature map. The two convolutional layers with multiple convolution kernels include at least a third convolutional layer and a fourth convolutional layer; the convolution kernel of the third convolutional layer is 2×100 in size, with a horizontal stride of 1 and a vertical stride of 2; the fourth convolutional layer is a depthwise separable convolution, with a convolution kernel of 1×100 in size and a height of 1.

[0062] Step S504 involves subjecting the symmetric channel feature map to at least two layers of two-dimensional convolution blocks with multiple convolution kernels to extract spatial features from the symmetric channel feature map, thereby obtaining a spatial feature map. The height of the convolution kernels in the two layers of two-dimensional convolution blocks with multiple convolution kernels is equal to the total number of EEG channels divided by 2, and the convolution kernel width is 1.

[0063] Step S505 flattens the spatial feature map to obtain neurons with confidence scores for emotion categories, and outputs neurons corresponding to the number of categories. This flattening removes redundant, heavy, and complex decorative effects. Specifically, it removes unnecessary perspective, texture, gradients, and elements that create 3D effects, allowing the "information" itself to be highlighted as the core.

[0064] Step S506 is to discriminate the emotion category based on the neurons with emotion category confidence. Discriminant the emotion category based on the neurons with emotion category confidence includes: determining the classification of the emotion category based on the polarity of the emotion category confidence; the output values of the neurons with emotion category confidence include: 0, 1, and 2; if the output value of the neuron with emotion category confidence is 0, the corresponding emotion category is sadness; if the output value of the neuron with emotion category confidence is 1, the corresponding emotion category is calm; if the output value of the neuron with emotion category confidence is 2, the corresponding emotion category is happiness.

[0065] Figure 6-Figure 7 Schematic diagram comparing the present invention with the prior art, such as Figure 6As shown, the present invention compares the proposed model with seven existing methods to evaluate the emotion recognition ability of SCEEGNet (symmetric EEG channel convolutional neural network), and the seven existing methods include: a deep neural network based on residual blocks for emotion recognition by extracting features of electrode frequency distribution map (EFDM), an EEG emotion recognition algorithm based on dynamic graph convolutional neural network, a new deep neural network combining convolutional neural network (CNN), sparse autoencoder (SAE) and deep neural network (DNN) for EEG emotion classification, regularized graph neural network (RGNN) for EEG-based emotion recognition, optimized residual network and VGG for EEG signal classification, recursive neural network (RNN) combined with fast Fourier transform (FFT), and emotion detection of EEG signals based on differential entropy linear formula (LF-DfE) feature extractor and BiLSTM network classifier.

[0066] Figure 7 The objective evaluation criteria for the above algorithm and the comparison algorithm based on SEED (emotional EEG) dataset are average accuracy, standard deviation, number of channels, number of parameters and whether feature pre-extraction is performed. Figure 7 As shown, the average accuracy and standard deviation of the proposed method are 98.37% and 1.95%, respectively, and the number of channels and model parameters used are the lowest among all methods. Furthermore, no feature pre-extraction is required. Therefore, comparative experiments fully demonstrate that SCEEGNet, based on the SEED dataset, outperforms other algorithms in EEG emotion recognition tasks and has greater application potential. By fully leveraging brain lateralization features, this method improves learning efficiency and the accuracy of emotion determination.

[0067] A device for determining emotion category according to the present invention includes: an acquisition unit for acquiring electroencephalogram (EEG) signals; a time-frequency feature extraction unit comprising at least two consecutive two-dimensional convolutional layers for extracting features from each channel of the EEG signals to generate a time-frequency feature map; a symmetric channel feature extraction unit comprising at least two convolutional layers with multiple convolution kernels for extracting emotion lateralization features from the time-frequency feature map to generate a symmetric channel feature map; a spatial feature extraction unit comprising at least two layers of two-dimensional convolutional blocks with multiple convolution kernels for extracting spatial features from the symmetric channel feature map to generate a spatial feature map; a fully connected unit comprising multiple neurons for flattening the spatial feature map to generate neurons with emotion category confidence; and a processing unit for discriminating emotion category based on the neurons with emotion category confidence. This device significantly reduces redundant computation by fully utilizing brain lateralization features and eliminating feature pre-extraction characteristics. It also requires no excessive number of electrodes for EEG signal acquisition and is wearable. The accuracy on the public emotion dataset SEED is significantly improved, enhancing learning efficiency and the accuracy of emotion determination.

[0068] The above describes in detail the optional implementation methods of the embodiments of the present invention in conjunction with the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above implementation methods. Within the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the scope of protection of the embodiments of the present invention.

[0069] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any appropriate manner without contradiction. To avoid unnecessary repetition, the embodiments of the present invention will not further describe various possible combinations.

[0070] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0071] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A device for determining emotion category, characterized in that: The device includes: an acquisition unit, for acquiring EEG signals; a time-frequency feature extraction unit, comprising at least two consecutive two-dimensional convolutional layers, for extracting features of each channel of the EEG signal to obtain a time-frequency feature map; a symmetric channel feature extraction unit, comprising at least two convolutional layers with multiple convolution kernels, for extracting the emotion lateralization feature of the time-frequency feature map to obtain a symmetric channel feature map; A spatial feature extraction unit, comprising at least two layers of two-dimensional convolution blocks with multiple convolution kernels, for extracting spatial features of the symmetric channel feature map to obtain a spatial feature map; A fully connected unit, comprising a plurality of neurons, configured to flatten the spatial feature map to obtain neurons having confidence levels of emotion categories; a processing unit, configured to discriminate an emotion category based on the neurons having the emotion category confidence; The at least two consecutive two-dimensional convolutional layers include at least: a first convolutional layer and a second convolutional layer; The convolution kernel sizes of the first convolution layer and the second convolution layer are both 1×100, and the heights of the convolution kernels are both 1; The at least two convolutional layers having multiple convolution kernels include at least a third convolutional layer and a fourth convolutional layer; The size of the convolution kernel of the third convolutional layer is 2×100, the horizontal step length of the convolution kernel is 1, and the vertical step length is 2; The fourth convolutional layer is a depth-wise separable convolution, the size of the convolution kernel of the fourth convolutional layer is 1×100, and the height of the convolution kernel is 1; The extraction of the spatial domain features of the symmetrical channel feature map includes pixel-level superposition and Fourier transform of the symmetrical channel feature map. The height of the convolution kernel in the spatial domain feature extraction unit is the total number of EEG channels divided by 2, and the convolution kernel width is 1.

2. The device according to claim 1, characterized in that The step of discriminating the emotion category based on the neurons having the emotion category confidence comprises: The output values of the neuron with emotion category confidence include: 0, 1 and 2; If the output value of the neuron with emotion category confidence is 0, the corresponding emotion category is sadness; If the output value of the neuron with emotion category confidence is 1, the corresponding emotion category is calm; If the output value of the neuron with emotion category confidence is 2, the corresponding emotion category is happy.

3. The device according to claim 1, characterized in that The device further includes: a training unit for training the time-frequency feature extraction unit, the symmetric channel feature extraction unit, and the spatial feature extraction unit to update the parameters of each extraction unit, including: The initial learning rate is 10 -4 The optimizer optimizes the loss function that minimizes the cross entropy, and uses cosine annealing to optimize the learning rate.

4. A method for determining emotion categories, characterized in that: The method includes: Acquire EEG signals; Submitting the EEG signal to at least two consecutive two-dimensional convolutional layers to extract features of each channel of the EEG signal to obtain a time-frequency feature map; Bringing the time-frequency feature map into at least two convolution layers with multiple convolution kernels to extract the emotion lateralization features of the time-frequency feature map to obtain a symmetrical channel feature map; Bringing the symmetric channel feature map into at least two layers of two-dimensional convolution blocks with multiple convolution kernels to extract spatial features of the symmetric channel feature map to obtain a spatial feature map; Flattening the spatial feature map to obtain neurons with emotion category confidence; discriminating the emotion category according to the neurons having the emotion category confidence; The at least two consecutive two-dimensional convolutional layers include at least: a first convolutional layer and a second convolutional layer; The convolution kernel sizes of the first convolution layer and the second convolution layer are both 1×100, and the heights of the convolution kernels are both 1; The at least two convolutional layers having multiple convolution kernels include at least a third convolutional layer and a fourth convolutional layer; The size of the convolution kernel of the third convolutional layer is 2×100, the horizontal step length of the convolution kernel is 1, and the vertical step length is 2; The fourth convolutional layer is a depth-wise separable convolution, the size of the convolution kernel of the fourth convolutional layer is 1×100, and the height of the convolution kernel is 1; The extracting of the spatial domain features of the symmetric channel feature map includes performing pixel-level superposition and Fourier transform on the symmetric channel feature map. The height of the convolution kernel of the two-dimensional convolution block is the total number of channels of the electroencephalogram divided by 2, and the convolution kernel width is 1.

5. The method according to claim 4, characterized in that The step of discriminating the emotion category based on the neurons having the emotion category confidence comprises: Determining the classification of the emotion category according to the polarity of the emotion category confidence; The output values of the neuron with emotion category confidence include: 0, 1 and 2; If the output value of the neuron with emotion category confidence is 0, the corresponding emotion category is sadness; If the output value of the neuron with emotion category confidence is 1, the corresponding emotion category is calm; If the output value of the neuron with emotion category confidence is 2, the corresponding emotion category is happy.

6. The method according to claim 4, characterized in that The method further includes: The optimizer with an initial learning rate of 10-4 is used to optimize the loss function that minimizes the cross entropy, and cosine annealing is used to optimize the learning rate.

Images