A method and system for emotion recognition based on two-stage KL divergence
By employing a two-level KL divergence-based emotion recognition method, which calculates the power spectral density of EEG signals in segments and constructs a KL divergence feature matrix, the problem of capturing subtle emotional changes in existing technologies is solved, thus achieving accurate emotion recognition.
Patent Information
- Application Number
- CN202610330425.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-18
- Publication Date
- 2026-06-26
AI Technical Summary
Existing EEG emotion recognition technologies struggle to accurately depict subtle emotional changes, especially in their inability to dynamically capture the asymmetry in activation levels between the left and right hemispheres of the brain under complex stimuli.
An emotion recognition method based on two-level KL divergence is adopted. By collecting and preprocessing EEG signals, power spectral density values are calculated in segments to construct power spectral density feature matrices for the left and right brain regions. The energy distribution difference between the left and right brain regions is calculated by KL divergence to construct the first-level and second-level KL divergence feature matrices. Finally, the matrices are input into the classification model for training and recognition.
It achieves accurate identification of subtle emotional changes, deeply mines the emotionally sensitive features in EEG signals, and improves the accuracy of emotion recognition.
Smart Images
Figure CN122272019A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of EEG emotion recognition technology, specifically to an emotion recognition method and system based on bilevel KL divergence. Background Technology
[0002] With the rapid development of neuroscience and brain-computer interface (BCI) technology, emotion recognition based on electroencephalogram (EEG) signals has become a research hotspot in fields such as mental health monitoring, user experience assessment, and neuromarketing. Quantitative assessment of emotional states can not only reveal the cognitive mechanisms of the human brain but also provide a scientific basis for improving psychological regulation methods. Taking everyday social activities like eating hot pot as an example, this type of complex stimulus includes multiple factors such as taste satisfaction, social pleasure, and temperature stimulation. Quantitatively assessing an individual's emotional fluctuations before and after this process is of great significance for studying the positive intervention effects of specific behaviors on mental health.
[0003] However, existing EEG emotion recognition technologies still face many challenges in practical applications. For example, current mainstream methods mostly focus on extracting energy features of a single frequency band (such as power spectral density, PSD) or statistical features of the whole brain. However, according to neurophysiological studies, positive and negative emotions are usually accompanied by an asymmetry in the activation levels of the left and right hemispheres of the brain (especially the frontal lobe). Existing algorithms are insufficient in their ability to dynamically capture this cross-brain region distribution difference, making it difficult to accurately characterize subtle emotional changes. Summary of the Invention
[0004] To accurately capture the asymmetry and dynamic changes in brain regions and features for emotion recognition, this invention proposes an emotion recognition method based on two-level KL divergence, which specifically includes the following steps:
[0005] Multichannel EEG signals of the whole brain were collected at two time points: before and after external sensory stimulation, and the EEG signals were preprocessed.
[0006] The EEG data is divided into multiple segments according to frequency bands. Each segment is used as a calculation unit to obtain the fast Fourier transform of each channel in each segment, and then the power spectral density value of each channel is calculated.
[0007] The whole brain pathway is divided into left brain region pathways and right brain region pathways, and power spectral density feature matrices of the left and right brain regions are constructed respectively.
[0008] KL divergence was used to calculate the differences in energy distribution of each frequency band in the left and right brain regions of the subjects before and after sensory stimulation, and then the first-level KL divergence feature matrix was constructed.
[0009] Calculate the average value of the first-level KL divergence feature matrix of each segment before and after sensory stimulation in each frequency band to obtain the feature matrix before and after sensory stimulation.
[0010] Based on the feature matrices before and after sensory stimulation, the KL divergence of the feature matrices before and after sensory stimulation in the same frequency band is calculated, and then a second-level KL divergence feature matrix is constructed.
[0011] The feature matrix based on sensory stimulation before and after is concatenated with the second-level KL divergence feature matrix and used as the feature vector of the subject to input the classification model for model training, ultimately achieving emotional intelligence recognition.
[0012] Compared with existing technologies, this invention deeply mines the emotionally sensitive features in EEG signals through a two-layer progressive distribution difference calculation, and combines it with a classification network to achieve model training and accurate classification, thereby achieving accurate recognition of subtle emotional changes. Attached Figure Description
[0013] Figure 1 This is a flowchart illustrating an emotion recognition method based on two-level KL divergence according to the present invention.
[0014] Figure 2 This is a flowchart of the two-stage KL divergence calculation of the present invention;
[0015] Figure 3 This is a schematic diagram of an emotion recognition system architecture based on two-level KL divergence according to the present invention. Detailed Implementation
[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0017] This invention proposes an emotion recognition method based on two-level KL divergence, which specifically includes the following steps:
[0018] Multichannel EEG signals of the whole brain were collected at two time points: before and after external sensory stimulation, and the EEG signals were preprocessed.
[0019] The EEG data is divided into multiple segments according to frequency bands. Each segment is used as a calculation unit to obtain the fast Fourier transform of each channel in each segment, and then the power spectral density value of each channel is calculated.
[0020] The whole brain pathway is divided into left brain region pathways and right brain region pathways, and power spectral density feature matrices of the left and right brain regions are constructed respectively.
[0021] KL divergence was used to calculate the differences in energy distribution of each frequency band in the left and right brain regions of the subjects before and after sensory stimulation, and then the first-level KL divergence feature matrix was constructed.
[0022] Calculate the average value of the first-level KL divergence feature matrix of each segment before and after sensory stimulation in each frequency band to obtain the feature matrix before and after sensory stimulation.
[0023] Based on the feature matrices before and after sensory stimulation, the KL divergence of the feature matrices before and after sensory stimulation in the same frequency band is calculated, and then a second-level KL divergence feature matrix is constructed.
[0024] The feature matrix based on sensory stimulation before and after is concatenated with the second-level KL divergence feature matrix and used as the feature vector of the subject to input the classification model for model training, ultimately achieving emotional intelligence recognition.
[0025] In this embodiment, EEG data from two time periods are collected as input, namely, data T1 from a period before external sensory stimulation and data T2 from a period after external sensory stimulation, representing multi-channel EEG signals of the whole brain.
[0026] In this step, EEG leads first need to be installed on the subject's head. The number of leads can be selected according to actual needs, with common options including 8 channels, 16 channels, 32 channels, 64 channels, and 128 channels. The number of EEG leads is denoted as k. The lead positions should accurately correspond to different areas of the brain according to the international 10-20 lead system to ensure comprehensive EEG signal acquisition. The lengths of data T1 (a period before external sensory stimulation) and T2 (a period after external sensory stimulation) should be flexibly set according to experimental requirements.
[0027] As an optional implementation, the acquired multi-channel EEG signals are preprocessed, including removing useless electrode signals and filtering.
[0028] This invention assesses indicators based on the left and right brain regions; therefore, following the international 10-20 lead system, electrodes along the midline are removed, for example: F z P z O z The acquired EEG data of length t is filtered, typically using a Butterworth zero-phase bandpass filter to retain data information from 0 to 250 Hz while removing 50 Hz power frequency interference. The filtered data is then used for subsequent analysis.
[0029] Next, the collected data needs to be segmented according to the preset segment length. The segment length is usually set as a multiple of 2 seconds, such as 2 seconds, 4 seconds, 8 seconds, etc.
[0030] Using a preset segment length T as the calculation unit, the data for this length is: Discrete signal sequences The sampling frequency is (Unit: Hz), perform Discrete Fourier Transform (DFT) on each acquisition channel within each segment, i.e.:
[0031]
[0032] in, It is the imaginary unit. It is a frequency index, and the corresponding physical frequency is... .
[0033] Next, the two-sided power spectrum is calculated:
[0034]
[0035] Then, the two-sided power spectrum is converted into a one-sided power spectrum, as follows:
[0036] DC component ( ):
[0037]
[0038] Positive frequency components ( ,when When it is even; or ,when When the number is odd):
[0039]
[0040] Nyquist frequency (only when When it is even, ):
[0041]
[0042] Finally, the power spectral density of different frequency bands is calculated based on the one-sided power spectrum. The calculation formula is as follows:
[0043]
[0044] in, It refers to the number of points within the frequency band; and The lower limit of the frequency band for which the power spectrum needs to be calculated and frequency upper limit The calculation formula is as follows:
[0045]
[0046] Here, round() rounds the integer to the nearest whole number. .
[0047] In this embodiment, the whole brain channel is divided into left brain region channel and right brain region channel, and power spectral density feature matrices of the left and right brain regions are constructed respectively.
[0048] Specifically, the power spectral density matrices of the left and right brain regions The brain is divided into frequency bands, and then the power spectral density matrices of the left and right brain regions for each frequency band are constructed. and , where n is the number of preset frequency bands. or m represents the number of acquisition channels in the left or right brain region. Let be the power spectral density value of the i-th frequency band on channel m.
[0049] As an optional implementation, the frequency band division may include six frequency bands: 0~4Hz, 4~8Hz, 8~12Hz, 12~32Hz, 32~80Hz and 80~250Hz.
[0050] Then, each submatrix in the power spectral density matrix of the left and right brain regions is transformed into a one-dimensional vector, and this vector is normalized (Softmax) so that the sum of all elements is 1, thus forming a discrete probability distribution. The Softmax calculation formula is as follows:
[0051]
[0052] Thus, the probabilized feature matrix is obtained. and .
[0053] This embodiment uses KL divergence to calculate the differences in energy distribution across frequency bands in the left and right brain regions of the subjects during the T1 and T2 time periods, respectively, and then constructs the first-level KL divergence feature matrix. The two-level KL divergence calculation process is described below. Figure 2 As shown, the calculation process is as follows: Under the same frequency band i, take the feature matrix of the corresponding frequency band. and The first-order KL divergence is calculated to assess the energy difference between the left and right brain regions at the same time and frequency band. The calculation formula is as follows:
[0054]
[0055] in, for or All elements in.
[0056] Thus, the first-order KL divergence matrix is obtained. , .
[0057] Based on the first-level KL divergence feature matrix, the first-level KL divergence feature matrix is superimposed and averaged for time periods T1 and T2 respectively to construct the feature matrices for time periods T1 and T2. , KL divergence is used to calculate the energy distribution differences in each frequency band, and then a second-level KL divergence feature matrix is constructed. The calculation process is as follows:
[0058] The first-level KL divergence matrix By superimposing and averaging the values of time periods T1 and T2, we obtain the feature matrices corresponding to the two time periods T1 and T2 before and after the external sensory stimulation:
[0059] and ;
[0060] based on and Calculate the second-order KL divergence matrix using the following formula:
[0061]
[0062] in, The average of the first-order KL divergence matrices corresponding to frequency band i in time period T1 (before external sensory stimulation) is given. It is the superimposed average of the first-level KL divergence matrix corresponding to frequency band i in time period T2 (before external sensory stimulation).
[0063] Thus, the second-order KL divergence matrix is obtained. , .
[0064] The second-level KL divergence matrix is obtained by calculation, thereby capturing the dynamic transfer pattern of emotional state before and after external stimuli.
[0065] The characteristic matrices for time periods T1 and T2 will be calculated using the first-order KL divergence. , Second-order KL divergence feature matrix The cascaded combinations are used as feature vectors of the subject to input into the classification model for model training, ultimately achieving emotional intelligence recognition.
[0066] The classification models that can be selected for this invention include Support Vector Machine (SVM), K-Nearest Neighbors (KNN), Random Forest, and KAN (Kolmogorov-Arnold Networks).
[0067] This embodiment also discloses an emotion recognition system based on two-level KL divergence, characterized in that, for implementing the emotion recognition method based on two-level KL divergence as described in claim 1, it includes a data acquisition module 1, a data preprocessing module 2, a feature extraction module 3, a two-level KL divergence calculation module 4, a classification model training and recognition module 5, and a result output module 6, wherein:
[0068] Data acquisition module 1 is used to acquire whole-brain electroencephalogram (EEG) data of the subjects;
[0069] Data preprocessing module 2 is used to remove useless electrode signals, filter data using a Butterworth zero-phase bandpass filter, and segment the data.
[0070] Feature extraction module 3 is used to perform FFT transformation and calculate power spectral density values;
[0071] The dual-level KL divergence calculation module 4 evaluates the difference in energy distribution between the left and right brain regions before and after stimulation through KL divergence calculation, including obtaining the first-level KL divergence feature matrix, the feature matrix before and after sensory stimulation, and the second-level KL divergence feature matrix.
[0072] The classification model training and recognition module 5 is used for classification model training and automatic recognition. The classification model takes the concatenated feature matrix before and after sensory stimulation and the second-level KL divergence feature matrix as input and outputs the recognition result. As an optional implementation, the cross-entropy between the predicted result and the real label can be selected as the loss function during training, and the network parameters of the classification model are improved with the goal of minimizing the loss function.
[0073] The result output module 6 is used to classify the EEG signals to be classified based on the completed classification model and output the classification results to an external device for display.
[0074] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A sentiment recognition method based on two-level KL divergence, characterized in that, Specifically, the following steps are included: Multichannel EEG signals of the whole brain were collected at two time points: before and after external sensory stimulation, and the EEG signals were preprocessed. The EEG data is divided into multiple segments according to frequency bands. Each segment is used as a calculation unit to obtain the fast Fourier transform of each channel in each segment, and then the power spectral density value of each channel is calculated. The whole brain pathway is divided into left brain region pathways and right brain region pathways, and power spectral density feature matrices of the left and right brain regions are constructed respectively. KL divergence was used to calculate the differences in energy distribution of each frequency band in the left and right brain regions of the subjects before and after sensory stimulation, and then the first-level KL divergence feature matrix was constructed. Calculate the average value of the first-level KL divergence feature matrix of each segment before and after sensory stimulation in each frequency band to obtain the feature matrix before and after sensory stimulation. Based on the feature matrices before and after sensory stimulation, the KL divergence of the feature matrices before and after sensory stimulation in the same frequency band is calculated, and then a second-level KL divergence feature matrix is constructed. The feature matrix based on sensory stimulation before and after is concatenated with the second-level KL divergence feature matrix and used as the feature vector of the subject to input the classification model for model training, ultimately achieving emotional intelligence recognition.
2. The emotion recognition method based on two-level KL divergence according to claim 1, characterized in that, The process of calculating the power spectral density value for each segment includes: Using a preset segment length T as the calculation unit, the signal of each channel under each segment length is sampled at the sampling frequency to obtain a discrete signal sequence consisting of N sampling points, and then a discrete Fourier transform is performed. Next, the two-sided power spectrum is calculated, and then the two-sided power spectrum is converted into a one-sided power spectrum; Then, based on the divided frequency bands, the lower frequency limit value of each band is obtained. and frequency upper limit Calculate the upper limit and lower limit of the frequency band index; Finally, the power spectral density value of each frequency band is obtained by summing the power spectra within the upper and lower limits of the frequency band index.
3. The emotion recognition method based on two-level KL divergence according to claim 2, characterized in that, If the lower limit of a frequency band is The upper limit of frequency is Then the upper limit of the frequency band index. and lower limit of frequency band index Represented as: Among them, round() rounds the integer to the nearest whole number. For frequency intervals, denoted as .
4. The emotion recognition method based on two-level KL divergence according to claim 2, characterized in that, The process of constructing the power spectral density feature matrices of the left and right brain regions includes: Within each segment, power spectral density matrices for the left and right brain regions are constructed according to frequency bands. , Where n is the preset number of frequency bands, or m represents the number of acquisition channels in the left or right brain region. The power spectral density value of the i-th frequency band on channel m; The submatrix of each frequency band in the power spectral density matrix of the left and right brain regions is transformed into a one-dimensional vector, and the vector is normalized so that the sum of all elements is 1, thus forming a discrete probability distribution. In the same frequency band i, the discrete probability distributions of the left and right hemispheres are taken respectively, the first-level KL divergence is calculated, the difference in energy between the left and right hemispheres at the same time and in the same frequency band is evaluated, and the first-level KL divergence of each frequency band is obtained to form the first-level KL divergence matrix.
5. The emotion recognition method based on two-level KL divergence according to claim 1, characterized in that, Each segment is a multiple of 2 seconds in length.
6. A sentiment recognition method based on two-level KL divergence according to claim 1 or 5, characterized in that, Each segment is 2 seconds, 4 seconds, or 8 seconds long.
7. The emotion recognition method based on two-level KL divergence according to claim 1, characterized in that, When preprocessing EEG signals, Butterworth zero-phase bandpass filters are used to retain data information from 0 to 250 Hz while removing 50 Hz power frequency interference.
8. A sentiment recognition method based on two-level KL divergence according to claim 1 or 7, characterized in that, When dividing the frequency bands, the data is divided into six bands: 0~4Hz, 4~8Hz, 8~12Hz, 12~32Hz, 32~80Hz and 80~250Hz.
9. The emotion recognition method based on two-level KL divergence according to claim 1, characterized in that, The classification model can be a support vector machine model, a k-nearest neighbor algorithm model, a random forest model, or a Kolmogorov-Arnold Networks model.
10. An emotion recognition system based on two-level KL divergence, characterized in that, The method for implementing the sentiment recognition method based on two-level KL divergence as described in claim 1 includes a data acquisition module (1), a data preprocessing module (2), a feature extraction module (3), a two-level KL divergence calculation module (4), a classification model training and recognition module (5), and a result output module (6), wherein: The data acquisition module (1) is used to acquire the whole brain electroencephalogram (EEG) data of the subjects; The data preprocessing module (2) is used to remove useless electrode signals, filter the data using a Butterworth zero-phase bandpass filter, and segment the data. The feature extraction module (3) is used to perform FFT transformation and calculate the power spectral density value; The dual-level KL divergence calculation module (4) evaluates the difference in energy distribution between the left and right brain regions before and after stimulation through KL divergence calculation, including obtaining the first-level KL divergence feature matrix, the feature matrix before and after sensory stimulation, and the second-level KL divergence feature matrix. The classification model training and recognition module (5) is used for classification model training and automatic recognition. The classification model takes the concatenation of the feature matrix before and after sensory stimulation and the second-level KL divergence feature matrix as input and outputs the recognition result. The result output module (6) is used to classify the EEG signals to be classified based on the completed classification model and output the classification results to an external device for display.