Emotion regulation and control method, device and equipment based on electroencephalogram signals and storage medium

Abstract

The invention discloses an emotion regulation and control method, device and equipment based on electroencephalogram signals and a storage medium. The method comprises the steps that the electroencephalogram signals of a target user are obtained; extracting multi-modal traditional features and depth features based on the electroencephalogram signals; the multi-modal traditional features comprise features in a time domain, a frequency domain and a time-frequency domain, and the dynamic features, the frequency domain features and the complexity of the electroencephalogram signals can be represented from different dimensions; the depth feature is a multi-dimensional feature which is obtained through a convolutional neural network and is related to an emotional state, and can represent a complex mode and a potential rule of a situation; determining a target emotion type of the target user by adopting a trained emotion classification model based on the multi-modal traditional features and the depth features; under the condition that the target emotion type is a negative emotion, adjusting the background color and background music of a display interface by adjusting the color of the electroencephalogram signal line to carry out emotion regulation and control. The emotion type can be accurately recognized, and effective emotion regulation measures can be quickly taken.

Description

Technical Field

[0001] The present application relates to the technical field of electroencephalogram (EEG) signal processing, and in particular, to an emotion regulation method, device, equipment and storage medium based on EEG signals. Background Art

[0002] With the continuous development of brain-computer interface technology and affective computing technology, emotion recognition and regulation technology based on electroencephalogram (EEG) signals has received extensive attention in academic research and practical applications.

[0003] In the prior art, emotions are generally recognized by analyzing EEG data, but usually only rough emotion categories can be recognized, and the recognition accuracy is low. When performing emotion regulation, it mainly relies on a single feedback method, such as vision, audition or touch, lacking multi-sensory interaction, making it difficult to effectively cope with complex or intense emotion changes, and usually unable to adjust the feedback in real time to cope with rapidly changing emotion states.

[0004] Therefore, there is an urgent need for an emotion regulation method that can accurately identify emotion types and quickly make effective regulation measures.

[0005] It should be noted that the above statements are only used to provide background technical information related to the present application, and do not necessarily constitute prior art. Summary of the Invention

[0006] In view of the above problems, embodiments of the present application provide an emotion regulation method, device, electronic equipment and storage medium based on EEG signals, which can accurately identify emotion types and quickly make effective emotion regulation measures.

[0007] In a first aspect, an embodiment of the present application provides an emotion regulation method based on EEG signals, the method comprising:

[0008] Obtaining the EEG signals of a target user;

[0009] Based on the EEG signals, extracting multi-modal traditional features and deep features respectively; the multi-modal traditional features include features in the time domain, frequency domain and time-frequency domain, and can characterize the dynamic characteristics, frequency domain characteristics and complexity of EEG signals from different dimensions; the deep features refer to multi-dimensional features related to emotion states obtained through a convolutional neural network, and can characterize complex patterns and potential laws of situations;

[0010] Based on the multi-modal traditional features and the deep features, using a trained emotion classification model to determine the target emotion type of the target user;

[0011] In the case where the target emotion type is a negative emotion, emotion regulation is performed by adjusting the color of the electroencephalogram signal line, the background color of the display interface, and the background music.

[0012] In some alternative embodiments, the extraction of multi-modal traditional features based on the electroencephalogram signal includes:

[0013] Extracting power spectral features based on the electroencephalogram signal; the power spectral features are used to represent the energy distribution of the electroencephalogram signal in the frequency domain and reflect the activity intensity of emotion-related frequency bands;

[0014] Extracting differential entropy features based on the electroencephalogram signal; the differential entropy features are used to characterize the probability distribution of an energy-quantized signal within a specific frequency band;

[0015] Extracting event-related potential features based on the electroencephalogram signal; the event-related potential features are used to reflect the time-locked response of the brain to emotional stimuli;

[0016] Extracting Hjorth parameter features based on the electroencephalogram signal; the Hjorth parameter features are used to describe the dynamic characteristics of the signal, including activity characteristics, mobility characteristics, and complexity characteristics, which respectively describe the overall energy of the signal, the frequency change characteristics, and the dynamic complexity;

[0017] Extracting wavelet energy features based on the electroencephalogram signal; the wavelet energy features are used to describe the energy distribution of the signal in the time-frequency domain and can reflect the activity intensity of emotion-related frequency bands at different times;

[0018] Extracting wavelet entropy features based on the electroencephalogram signal; the wavelet entropy features are the entropy values based on wavelet energy, used to quantify the complexity of the signal and reveal the relationship between the emotional state and the complexity of the electroencephalogram signal.

[0019] In some alternative embodiments, the extraction of power spectral features based on the electroencephalogram signal includes:

[0020] Performing a fast Fourier transform on the electroencephalogram signal to convert the time-domain signal into a frequency-domain signal;

[0021] Calculating the power spectral density of each frequency band based on the signal after the fast Fourier transform.

[0022] In some alternative embodiments, the extraction of Hjorth parameter features based on the electroencephalogram signal includes:

[0023] Determining the activity characteristics based on the variance of the electroencephalogram signal;

[0024] Determining the mobility characteristics of the electroencephalogram signal based on the first derivative of the electroencephalogram signal and the activity characteristics;

[0025] Determine the complexity characteristics of the EEG signal based on the first derivative, second derivative of the EEG signal, and the mobility characteristics.

[0026] In some alternative embodiments, the extracting depth features based on the EEG signal includes:

[0027] Convert the EEG signal into a two-dimensional tensor including the number of channels and the number of time points;

[0028] Perform time convolutional layer processing, spatial convolutional layer processing, activation layer processing, and fully connected layer processing on the two-dimensional tensor in sequence to obtain the depth features.

[0029] In some alternative embodiments, the determining the target emotion type of the target user by using the trained emotion classification model based on the multi-modal traditional features and the depth features includes:

[0030] Fuse the multi-modal traditional features and the depth features to obtain a fused vector;

[0031] Based on the fused vector, perform time series modeling and multi-dimensional analysis by using the trained emotion classification model to generate a probability distribution of various emotion types;

[0032] Determine the emotion type with the maximum probability distribution as the target emotion type of the target user.

[0033] In some alternative embodiments, the emotion regulation by adjusting the color of the EEG signal line, the background color of the display interface, and the background music includes:

[0034] When the target user is in a negative emotion, dynamically adjust the color of the EEG signal line, adjust the background color of the interface through a dynamic animation effect, and adjust the pitch and rhythm of the background music, so that the target user gradually changes from a negative emotion to a positive emotion;

[0035] In the above emotion regulation process, the background color and the background music match the color of the EEG signal line.

[0036] In a second aspect, an embodiment of the present application provides an emotion regulation device based on EEG signals. The emotion regulation device based on EEG signals includes:

[0037] A signal acquisition module, configured to acquire the EEG signal of a target user;

[0038] A feature extraction module, configured to extract multi-modal traditional features and deep features based on the electroencephalogram (EEG) signals respectively; the multi-modal traditional features include features in the time domain, frequency domain, and time-frequency domain, and can characterize the dynamic characteristics, frequency domain characteristics, and complexity of EEG signals from different dimensions; the deep features refer to multi-dimensional features related to emotional states obtained through a convolutional neural network, and can characterize the complex patterns and potential rules of situations;

[0039] An emotion recognition module, configured to determine the target emotion type of the target user by using a trained emotion classification model based on the multi-modal traditional features and the deep features;

[0040] An emotion regulation module, configured to perform emotion regulation by adjusting the color of the EEG signal line, the background color of the display interface, and the background music when the target emotion type is a negative emotion.

[0041] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor runs the computer program to implement the method as described in the first aspect.

[0042] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and the program is executed by a processor to implement the method as described in the first aspect.

[0043] In the present application, based on the obtained EEG signals, multi-modal traditional features and deep features are extracted respectively, and based on the multi-modal traditional features and deep features, a trained emotion classification model is used to determine the target emotion type of the target user. In this way, the emotional fluctuations of the user can be monitored and recognized in real time, and the accuracy and reliability of emotion recognition can be improved. And when the target emotion type is a negative emotion, the color of the EEG signal line can be adjusted, the background color of the display interface, and the background music can be adjusted to timely guide the target user to a positive emotion, thereby effectively solving the key problems such as single adjustment means and lack of real-time dynamic feedback in the prior art. And more accurate and real-time emotion regulation can be provided, reducing the negative impact of emotional fluctuations on decision-making and cognition, and further contributing to the improvement of the long-term mental health of the user.

[0044] The above description is only an overview of the technical solutions of the embodiments of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the description. And in order to make the above and other purposes, features, and advantages of the embodiments of the present application more obvious and understandable, the following specifically gives the specific implementation manners of the present application. Description of the Drawings

[0045] Upon reading the following detailed description of the preferred embodiments, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of illustrating the preferred embodiments and are not considered to be a limitation of the present application. Moreover, in all the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0046] Figure 1 It is a schematic flowchart of a method for emotion regulation based on electroencephalogram (EEG) signals provided by some embodiments of the present application;

[0047] Figure 2 It is a specific flowchart of step S2 provided by some embodiments of the present application;

[0048] Figure 3 It is a more specific flowchart of step S3 provided by some embodiments of the present application;

[0049] Figure 4 It is a schematic diagram of the framework result of an emotion regulation device based on electroencephalogram (EEG) signals provided by some embodiments of the present application;

[0050] Figure 5 It shows a schematic structural diagram of an electronic device provided by an embodiment of the present application. Specific Embodiments

[0051] The embodiments of the technical solution of the present application will be described in detail below with reference to the drawings. The following embodiments are only used to illustrate the technical solution of the present application more clearly, and therefore are only examples and cannot be used to limit the protection scope of the present application.

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the embodiments of the present application belong; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the embodiments of the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above drawings are intended to cover non-exclusive inclusion.

[0053] In the description of the embodiments of the present application, "a plurality" means more than two unless otherwise specifically defined.

[0054] Referring to "embodiments" herein means that a specific feature, structure, or characteristic described in connection with the embodiments can be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0055] In the description of the embodiments of the present application, the term "and / or" is merely an association relationship describing associated objects, indicating that three relationships may exist. For example, A and / or B may represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. Additionally, in this text, the character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0056] The embodiments of the present application propose an emotion regulation method based on electroencephalogram (EEG) signals. This method first extracts multi-modal traditional features and deep features based on EEG signals, and then uses a trained emotion classification model to accurately analyze the extracted multi-modal traditional features and deep features to accurately obtain the emotion type of the target user. When the target user is in a negative emotion, emotion regulation is performed by adjusting the color of the EEG signal line, the background color of the display interface, and the background music. In this way, the emotional fluctuations of the user can be monitored and recognized in real time, improving the accuracy and reliability of emotion recognition. And when the negative emotion of the user is recognized, the color of the EEG signal line, the background color of the display interface, and the adjustment method of the background music are adjusted in a timely manner to guide the user towards a positive emotion, effectively solving key problems such as a single adjustment means and lack of real-time dynamic feedback in the prior art. Moreover, more accurate and real-time emotion regulation can be provided, reducing the negative impact of emotional fluctuations on decision-making and cognition, and thus contributing to the improvement of the long-term mental health of the user.

[0057] The following combines the accompanying drawings to provide a detailed description of the emotion regulation method based on EEG signals provided by the embodiments of the present application. Please refer to Figure 1 , Figure 1 which is a schematic flowchart of the emotion regulation method based on EEG signals provided by the embodiments of the present application. As Figure 1 shown, the emotion regulation method based on EEG signals may include the following steps:

[0058] Step S1, obtaining the EEG signals of the target user.

[0059] Among them, the target user can be any user, which is only used to refer to the user who is subjected to emotion analysis and emotion regulation.

[0060] In this embodiment, the EEG signals (EEG) of the target user can be collected in real time through a portable EEG signal acquisition device. The sampling frequency can be set to 500 Hz or higher to capture the subtle changes in the signal and ensure the time resolution and signal integrity. The acquisition channels can be arranged according to the international 10-20 system to cover the frontal lobe, parietal lobe, and other emotion-related regions, so as to comprehensively obtain the neural activity information related to emotion regulation. During the EEG signal acquisition process, the EEG waveform of the target user can be recorded in real time to ensure stable data transmission without loss. At the same time, a high-impedance amplifier is used to reduce the influence of external environmental interference on the signal.

[0061] The EEG signals collected by the above-mentioned portable EEG signal acquisition device can be called the original EEG signals. These original EEG signals may be affected by artifacts and noise interference. Therefore, the original EEG signals can be preprocessed to optimize the signal-to-noise ratio and extract effective information before performing the feature extraction process in the subsequent step S2.

[0062] Specifically, the following preprocessing process can be carried out: First, for the 50Hz (or 60Hz) power frequency interference problem, a notch filter is used to remove power line noise. When designing the notch filter, ensure that it has no significant impact on the signals in the target frequency band to avoid unnecessary attenuation of the spectrum of the effective EEG signals. Then, a band-pass filter in the range of 0.5 - 45Hz is used to retain the frequency components related to emotions and filter out low-frequency drift (<0.5Hz) and high-frequency noise (>45Hz), such as electrode contact noise, electromyogram interference, etc. The band-pass filter usually adopts a Butterworth filter to achieve a smooth frequency response characteristic and ensure the fidelity of the signal. Next, the EEG signals are decomposed by independent component analysis (ICA). After separating the original signals into independent components, the artifact signals, such as electrooculogram (EOG), electromyogram (EMG), and electrocardiogram (ECG) interference, are identified and removed. The identification of artifact components can be based on their specific time-domain or frequency-domain characteristics (such as the electrooculogram signal has the characteristics of low frequency and high amplitude). After removing the artifacts, the remaining components are reconstructed into clean EEG signals. Subsequently, the denoised EEG signals are normalized so that they are normalized to a unified range (such as a mean of 0 and a standard deviation of 1). Normalization helps to eliminate the signal amplitude differences between different users and acquisition environments, facilitating the stable training and inference of subsequent feature extraction and classification models. Finally, according to the experimental tasks or data analysis requirements, the preprocessed continuous signals are divided into time periods of fixed length (such as 1-second or 2-second windows), and an overlapping sliding window method (such as 50% overlap) can be adopted to increase the data sample size and improve the generalization ability of the emotion recognition model.

[0063] After the above process operations, the generated EEG signals have a high signal-to-noise ratio and at the same time retain rich emotion-related information. This provides a reliable data basis for subsequent multi-modal traditional feature extraction and deep feature extraction.

[0064] Step S2: Extract multi-modal traditional features and deep features based on the EEG signals respectively;

[0065] Among them, the multi-modal traditional features include features in the time domain, frequency domain, and time-frequency domain, which can characterize the dynamic characteristics, frequency domain characteristics, and complexity of EEG signals from different dimensions; the deep features refer to multi-dimensional features related to emotional states obtained through a convolutional neural network, which can characterize the complex patterns and potential laws of emotions.

[0066] In this embodiment, the extraction of multi-modal traditional features provides a comprehensive and rich feature basis for emotion recognition. The extraction of multi-modal traditional features is a key process in the emotion recognition process based on EEG signals, aiming to extract useful information from EEG signals in different dimensions, including time-domain, frequency-domain, and time-frequency domain features. These features can comprehensively reflect the neural activity patterns related to emotions and provide diverse input data for subsequent emotion classification models.

[0067] In the process of extracting multi-modal traditional features, six types of features, namely power spectrum, differential entropy, event-related potential (ERP), Hjorth parameters, wavelet energy, and wavelet entropy, can be specifically extracted from the preprocessed EEG signals, so as to comprehensively analyze the characteristics of EEG signals in the time-domain, frequency-domain, and time-frequency domain. Accordingly, the above extraction of multi-modal traditional features based on EEG signals can specifically include: extracting power spectrum features, differential entropy features, event-related potential features, Hjorth parameter features, wavelet energy features, and wavelet entropy features based on EEG signals.

[0068] Among them, the power spectrum features are used to represent the energy distribution of EEG signals in the frequency domain and reflect the activity intensity of emotion-related frequency bands; the differential entropy features are used to characterize the probability distribution of an energy-quantized signal within a specific frequency band; the event-related potential features are used to reflect the time-locked response of the brain to emotional stimuli; the Hjorth parameter features are used to describe the dynamic characteristics of the signal, including activity characteristics, mobility characteristics, and complexity characteristics, which respectively describe the overall energy of the signal, the frequency change characteristics, and the dynamic complexity; the wavelet energy features are used to describe the energy distribution of the signal in the time-frequency domain and can reflect the activity intensity of emotion-related frequency bands at different times; the wavelet entropy features are the entropy values based on wavelet energy and are used to quantify the complexity of the signal and reveal the relationship between the emotional state and the complexity of EEG signals.

[0069] Through the extraction of the above six types of features, the dynamic characteristics, frequency-domain characteristics, and complexity of EEG signals can be comprehensively characterized from different dimensions, providing rich and comprehensive feature inputs for subsequent emotion classification.

[0070] Specifically, the process of extracting power spectrum features based on EEG signals can include the following processing: performing a fast Fourier transform on the EEG signals to convert the time-domain signals into frequency-domain signals; calculating the power spectral density of each frequency band based on the signals after the fast Fourier transform. In this way, the frequency-domain features that can reflect the activity intensity of emotion-related frequency bands, that is, the power spectrum features, can be extracted.

[0071] Such as Figure 2As shown, the process of extracting Hjorth parameter features based on EEG signals may specifically include the following processing: Step S21, determining the activity feature based on the variance of the EEG signal; Step S22, determining the mobility feature of the EEG signal based on the first derivative and the activity feature of the EEG signal; Step S23, determining the complexity feature of the EEG signal based on the first derivative, the second derivative and the mobility feature of the EEG signal. In this way, the Hjorth parameter features for respectively describing the overall energy, frequency change characteristics and dynamic complexity of the signal can be accurately extracted, providing differential information in terms of signal morphology for different emotional states.

[0072] The following is a detailed description of the above-mentioned multi-modal traditional feature extraction:

[0073] Power spectrum feature extraction: Power spectrum features are the energy distribution of EEG signals in the frequency domain and can reflect the activity intensity in emotion-related frequency bands. For example, alpha waves may be related to relaxation, and beta waves may be related to anxiety. The specific calculation method is as follows: First, use the fast Fourier transform (FFT) to convert the time-domain signal into a frequency-domain signal, then square and average the transformed signal to obtain the power spectral density (PSD), and finally extract the average power spectra of five frequency bands, namely delta (1 - 4 Hz), theta (4 - 8 Hz), alpha (8 - 13 Hz), beta (13 - 30 Hz) and gamma (30 - 45 Hz), as the energy distribution of each frequency band. The calculation formula of the fast Fourier transform can be expressed as:

[0074]

[0075] where x(t) is the preprocessed EEG signal, f is the frequency, and N is the number of sampling points of the signal. The calculation formula of the power spectral density (PSD) is:

[0076]

[0077] where f is the frequency and N is the number of sampling points of the signal.

[0078] Differential entropy feature extraction: Differential entropy is a feature based on information theory that can quantify the probability distribution of a signal in a specific frequency band, can capture the change in the signal distribution under emotion induction, and has strong emotion discrimination ability. The specific calculation method is: First, decompose the preprocessed EEG signal into multiple time windows, and then calculate the differential entropy values of different frequency bands within each time window. The calculation formula of differential entropy is:

[0079]

[0080] where σ 2 is the variance of the EEG signal, and H(x) represents the differential entropy of the signal.

[0081] Event-related potential (ERP) feature extraction: ERP features can reflect the time-locked response of the brain to emotional stimuli. Extract event-related potentials (ERP) according to the experimental task markers, and calculate the amplitudes and latencies of ERP components (such as P300 and N200) within a specific time window. The calculation formula for the EPR amplitude is:

[0082]

[0083] where t s is the occurrence time of the stimulus event, and T is the length of the time window. The latency of ERP, that is, the difference between the time point when the signal reaches the peak and the occurrence time of the stimulus event.

[0084] Hjorth parameter feature extraction: Hjorth parameters are statistical features used to describe the dynamic characteristics of signals, including activity, mobility, and complexity, which respectively describe the overall energy of the signal, the frequency change characteristics, and the dynamic complexity, providing information on the morphological differences of signals for different emotional states. The specific calculations are as follows:

[0085] 1) Activity: Defined as the variance of the signal, representing the overall energy level of the signal, and the calculation formula is as follows:

[0086] Activity = Var(x(t))

[0087] 2) Mobility: Defined as the ratio of the standard deviation of the first derivative of the signal to the standard deviation of the signal itself, and the calculation formula is as follows:

[0088]

[0089] 3) Complexity: Defined as the ratio of the change between the second derivative and the first derivative of the signal, measuring the degree of change of the signal, and the calculation formula is as follows:

[0090]

[0091] Wavelet energy feature extraction: Wavelet energy describes the energy distribution of a signal in the time-frequency domain and can reflect the activity intensity of emotion-related frequency bands at different times. The specific calculation method is to decompose the EEG signal into wavelet coefficients of different scales through discrete wavelet transform (DWT), and then calculate the sum of the squares of the wavelet coefficients at each decomposition scale as the wavelet energy. The signal decomposition formula based on DWT is:

[0092]

[0093] where c j,k is the wavelet coefficient, and ψ j,k (t) is the wavelet basis function. Therefore, the calculation formula for wavelet energy is:

[0094]

[0095] Wavelet entropy feature extraction: Wavelet entropy is the entropy value based on wavelet energy, which is used to quantify the complexity of signals and reveal the relationship between emotional states and the complexity of EEG signals. The specific calculation method is to first normalize the energy of each scale, calculate the energy probability p j , and then calculate the wavelet entropy H of the signal based on the Shannon entropy formula. The calculation formula is:

[0096]

[0097] In some alternative embodiments, the extraction of deep features based on EEG signals may include the following processing: converting the EEG signal into a two-dimensional tensor including the number of channels and the number of time points; sequentially performing a temporal convolutional layer processing, a spatial convolutional layer processing, an activation layer processing, and a fully connected layer processing on the two-dimensional tensor to obtain deep features.

[0098] In this embodiment, a convolutional neural network is used to extract deep features from the preprocessed EEG signals, mining the complex patterns and potential laws related to emotional states in the signals, and providing a high-dimensional feature representation for emotion classification.

[0099] The following is the specific extraction process of deep features:

[0100] 1) Data input: The preprocessed EEG signals are organized in a two-dimensional tensor format where C represents the number of channels and T represents the number of time points. The signals are input into the deep learning network in this form to facilitate capturing their spatio-temporal characteristics.

[0101] 2) Temporal convolutional layer: Through one-dimensional convolutional operations, focus on the short-term dynamic features in the time dimension. The convolutional kernel size is set to 1×k t , and it slides along the time direction to capture local time patterns. The calculation formula for the output feature map of temporal convolution is:

[0102]

[0103] where, w i,j is the convolutional kernel weight and b is the bias term.

[0104] 3) Spatial convolutional layer: The features output by the temporal convolution are further input into the deep spatial convolutional layer. The convolutional kernel size of this layer is set to C×1, covering all channels, and only performing convolutional operations in the channel dimension to capture the co-activity patterns between channels. The calculation formula is:

[0105]

[0106] where, wi where a is the channel weight and b is the bias term.

[0107] 4) Activation layer: Through a non-linear activation function (such as ReLU or ELU), perform a non-linear transformation on the convolutional output to enhance the model's ability to express complex patterns. The activation function formula is:

[0108] ReLU(x) = max(0, x)

[0109] or

[0110]

[0111] where α is a hyperparameter used to control the stretchability in the negative value region.

[0112] 5) Fully connected layer: After completing the spatio-temporal feature extraction, flatten the feature tensor and input it into the fully connected layer for high-dimensional feature integration. Through linear transformation, map the high-dimensional features to a feature vector of a fixed length:

[0113] Y dense = W · Y flatten + b where W is the weight matrix of the fully connected layer, b is the bias term, and Y flatten represents flattening the tensor into a one-dimensional vector.

[0114] Through the above process, the convolutional neural network realizes a comprehensive modeling of the EEG signals, especially the deep feature mining in the time and space dimensions, so as to obtain deep features. These features can capture the complex patterns in the signals, provide high-quality and multi-dimensional input data for the emotion classification task, and significantly improve the classification performance.

[0115] Step S3, based on the multi-modal traditional features and deep features, use the trained emotion classification model to determine the target emotion type of the target user;

[0116] Among them, the emotion classification model can be but is not limited to the Transformer network.

[0117] In this embodiment, by fusing traditional features and deep features and combining with the Transformer network to achieve emotion classification, it fully utilizes multi-dimensional information and time dynamic characteristics, greatly improving the accuracy and robustness of emotion recognition. And it fully utilizes the complementary advantages of traditional features and deep learning features, effectively improving the classification performance of emotion recognition.

[0118] Specifically, such as Figure 3As shown in the figure, the above step S3 may include the following specific steps: Step S31, fuse the multi-modal traditional features and deep features to obtain a fused vector; Step S32, based on the fused vector, use the trained emotion classification model to perform time series modeling and multi-dimensional analysis to generate the probability distributions of various emotion types; Step S33, determine the emotion type with the largest probability distribution as the target emotion type of the target user.

[0119] In this embodiment, the traditional features and deep features are effectively fused, and the Transformer model is combined for efficient emotion classification. The feature fusion step ensures the full utilization of various emotion-related information, and the powerful modeling ability of the Transformer can capture the dynamic changes of the emotional state over time. Finally, the emotion classification model can accurately identify the emotional state of the target user.

[0120] The following is the specific implementation process of step S3:

[0121] Feature fusion: The goal of feature fusion is to combine the features extracted from different methods to form a comprehensive feature vector containing multi-dimensional emotion information, including power spectrum, differential entropy, event-related potential (ERP), Hjorth parameters, wavelet energy, and wavelet entropy, etc. These features provide multi-dimensional static information about emotions. The spatio-temporal dynamic features in the EEG signals are extracted by a convolutional neural network (CNN) to generate high-dimensional deep features. In order to make full use of this information, the traditional features and deep features are fused. Common fusion methods include concatenation fusion and weighted fusion.

[0122] 1.1) Concatenation fusion: Directly concatenate the traditional features and deep learning features in the feature dimension to generate a new feature vector containing all the information:

[0123] F fused =[F traditionla ,F deep =[f1,f2,...,f n ,d1,d2,...,d m

[0124] where F traditionla =[f1,f2,...,f n represents the traditional feature set, and each f i represents a traditional emotion-related feature. F deep =[d1,d2,...,d m represents the deep feature set, and each d i represents the spatio-temporal features extracted by the convolutional neural network.

[0125] ​1.2) Weighted Fusion: Different weights are assigned to traditional features and deep features according to feature importance to generate weighted fusion features:

[0126] F fused = αF traditionla + βF deep

[0127] where α and β are weight coefficients, which can be automatically adjusted by methods such as cross-validation to optimize the performance of emotion recognition.

[0128] Through the above method, the fused feature set not only retains the interpretability of traditional features but also contains the high-dimensional abstract information of deep features.

[0129] Emotion Classification Model: The vector F after feature fusion fused is input into the Transformer network for emotion classification, which can accurately classify the emotion state into multiple categories (e.g., sadness, anxiety, despair, happiness, excitement, satisfaction, etc.). Transformer has powerful sequence modeling capabilities and can capture the dynamic changes of emotion states. Due to its powerful self-attention mechanism and sequence modeling capabilities, the Transformer network has significant advantages in capturing the temporal dependencies and complex patterns in emotion signals. The following is the specific workflow of the Transformer network in emotion classification:

[0130] 2.1) Input Processing and Positional Encoding: The fused feature vector is input into the Transformer encoder as sequence data. Since the Transformer itself does not have the concept of sequence order, it is necessary to introduce temporal order information through positional encoding (PositionalEncoding, PE). The positional encoding formula is:

[0131]

[0132] where pos is the position, i is the dimension index, and d is the feature dimension size

[0133] 2.2) Self-attention Mechanism: The self-attention mechanism is the core of the Transformer. The Transformer uses the self-attention mechanism to capture the dynamic relationships between emotion features. It calculates the relationships between the features at each position in the input sequence to perform weighted summation and outputs a new feature representation. The calculation formula of the self-attention mechanism is:

[0134]

[0135] where Q, K, and V are the query, key, and value matrices respectively, and d kis the dimension of the key vector. The self-attention mechanism can dynamically adjust the weights of each feature according to the context information, capturing the subtle changes in emotional fluctuations.

[0136] 2.3) Multi-Head Attention: To enhance the model's feature capture ability, Transformer adopts the multi-head attention mechanism to learn different emotional features in parallel from multiple subspaces. Each attention head calculates independently, and the outputs of multiple attention heads are concatenated and then fed into the fully connected layer.

[0137] 2.4) Fully Connected Layer: The output of multi-head attention is mapped to the emotional category space through the linear transformation of one or more fully connected layers, and the probability distribution of each emotional category is generated through the softmax activation function. The emotional category predicted by the model is the category with the highest probability.

[0138] Classification Results: The output of the emotion classification model is a multi-dimensional probability distribution, representing the probability that the input features belong to each emotional category. By setting a threshold or selecting the category with the highest probability, the classification result is finally obtained, indicating the user's current emotional state (e.g., sad, anxious, desperate, happy, excited, satisfied, etc.). Suppose the emotional categories recognized by the model include six emotional states: sad, anxious, desperate, happy, excited, and satisfied, then the output form is a vector:

[0139] P = [P 悲伤 ,P 焦虑 ,P 绝望 ,P 快乐 ,P 兴奋 ,P 满足

[0140] where P represents the predicted probability of this category. The emotional state is determined by the category with the largest probability value. For example, when P 快乐 is the largest, the current emotion is recognized as "happy".

[0141] Step S4, in the case where the target emotion type is a negative emotion, emotional regulation is performed by adjusting the color of the EEG signal line, the background color of the display interface, and the background music.

[0142] In this embodiment, the user can be guided in real time from negative emotions to positive emotions through the combination of the interface, animation, and sound effects, helping the user to adjust and stabilize emotions. When the user's negative emotion is recognized, the negative emotion can be alleviated and the emotion can be promoted to change in a positive direction through dynamic interface color changes, animation effects, sound effect feedback, etc.

[0143] ​In an optional embodiment, the above step S4 may include the following processing procedures: when the target user is in a negative mood, by dynamically adjusting the color of the electroencephalogram (EEG) signal line, adjusting the interface background color through a dynamic animation effect, and adjusting the tone and rhythm of the background music, so that the target user gradually changes from a negative mood to a positive mood; during the above-mentioned emotion regulation process, the background color and the background music match the color of the EEG signal line.

[0144] In this embodiment, in view of the fact that when the target user is in a negative mood, the color of the EEG signal line can be in a specific color, so the negative mood of the target user can be intervened and alleviated by dynamically adjusting the color of the EEG signal line, adjusting the background color through a gradient animation effect, and adjusting the background sound effect, thereby promoting the target user's emotion to develop in a more positive direction and enhancing the user's emotion regulation ability.

[0145] During the above-mentioned emotion regulation process, both the background color and the background music match the color of the EEG signal line, which can help the user better restore concentration after the emotion is effectively regulated and avoid the interference of emotions. And through precise sensory feedback, the user can quickly enter a more stable emotional state, thereby improving the attention concentration and work efficiency.

[0146] The following is an example to illustrate the above emotion regulation process:

[0147] Regulate Sadness → Happiness

[0148] After detecting the user's sad mood, the emotion regulation mechanism is activated. The color of the EEG signal line gradually transitions from dark blue (#003366) to light blue (#99CCFF). To further guide the emotional change, the background color of the interface is adjusted through dynamic animation effects. The background color gradually transitions from dark blue (#003366) to soft light blue (#99CCFF) through a gradient animation, creating a peaceful and soothing atmosphere. This color transformation helps the user relax both physically and mentally. The background music also gradually adjusts along with the emotional change. Initially, the music presents a low-pitched piano piece with a low frequency and a steady rhythm. As the mood picks up, the background music gradually transitions to a more upbeat electronic music, and the pitch and rhythm gradually change from a steady bass to a bright and lively melody. The gradual change in music not only helps soothe sadness but also drives the user into a more positive emotional state through the change in audio pitch. When it is recognized that the emotion is gradually turning positive, the background color transitions again. The background color changes from light blue (#99CCFF) to warm gold (#FFCC00), causing the user's mood to slowly rise from calm to positive. At this time, the waveform of the target user's brain waves becomes more regular, and the background music also changes accordingly. The low pitch gradually changes to a bright and pleasant melody, and the rhythm of the music gradually speeds up. This change helps enhance the sense of the rising mood, enabling the user to experience a pleasant and relaxing emotional change. As the emotion further recovers, the changes in the background color and brain waves accelerate, entering a brighter and more vibrant color tone. The background color gradually changes from warm gold (#FFCC00) to a brighter orange (#FF6600), guiding the user's mood towards a more positive and excited state. The rhythm of the background music speeds up synchronously, changing from a lively melody to a more dynamic electronic dance music, with the rhythm gradually increasing. The rhythm of the sound effects speeds up, quickly guiding the user into a more excited and positive emotional state. This rapid change in rhythm and melody helps the user completely emerge from sadness and enter a state full of vitality, excitement, and joy. As the emotion rapidly rises, the color and waveform changes of the EEG signal line accelerate. The color of the EEG signal line gradually changes from gold (#FFCC00) to vibrant orange (#FF6600), and the waveform becomes more stable and regular, reflecting the rapid rise of the emotion. Through this gradually accelerating change in color and waveform, the user's emotion ultimately enters a state full of energy, excitement, and joy. The background music reaches its climax, the sound effects become pleasant and dynamic, the melody turns bright and energetic, driving the user into a relaxed, happy, and vibrant state. At this stage, the brain wave frequency remains at a relatively high level (such as alpha waves, beta waves), showing a highly active and positive emotional state, and the user's emotion has completely changed to happiness.

[0149] Regulate Anxiety → Excitement

[0150] After detecting the user's anxiety, the emotion regulation mechanism is immediately activated. First, the color of the EEG signal line gradually transitions from dark red (#8B0000) to warm orange (#FFA500). This color change helps release and relieve the user's emotions. The interface background color also gradually transitions from dark red (#8B0000) to soft warm orange (#FFA500) through an animation effect. This color change helps create a more gentle and soothing atmosphere, thus reducing the user's sense of tension. The background music gradually adjusts as the emotion relaxes, transitioning from low and dull sounds (such as slow bass piano music) to a more gentle and lighter rhythm. The gradual change in the sound effects drives the user to gradually relax, helping the emotion to gradually return to a calm state from a tense state. When it is detected that the anxiety has started to ease, the waveform of the brain waves also becomes more regular, indicating further emotional stability and recovery. As the brain waves recover, the color of the EEG signal line gradually transitions from warm orange (#FFA500) to a brighter orange-red (#FF4500), helping the user enter a more positive state. As the emotion further rises, the waveform of the brain waves becomes more regular and energetic, with the frequency further increasing. As the emotion improves, the color of the EEG signal line gradually becomes brighter and more energetic, accelerating the transition from warm orange (#FFA500) to a brighter and more energetic orange-red (#FF4500). This accelerated color change drives the user's emotion towards a positive state. The rhythm of the background music accelerates synchronously, transitioning from a gentle melody to more rhythmic electronic music, and the rhythm of the sound effects gradually increases, quickly driving up the user's emotion. Through the accelerated rhythm and strong rhythm of the electronic music, the user's emotion is guided into an excited state, enhancing the user's vitality and motivation. Through this series of progressive feedback mechanisms, the user's emotion finally completely transforms from an anxious state to an excited and positive state. The brain wave frequency reaches high beta waves or gamma waves (20 - 100Hz), and the waveform is more stable and regular. The color of the EEG signal line accelerates the transition from orange-red (#FF4500) to a more vibrant bright orange (#FF6347), and along with the continuous increase in the brain activity frequency, it drives the user into a state full of vitality and dynamism. The melody of the background music quickly turns to energetic electronic dance music or exciting drum beats (such as rhythmic electronic dance music or big drum rhythms), and through the fast rhythm and exciting sound effects, it helps the user completely release the anxiety and enter an excited state full of passion and motivation.

[0151] Regulate Despair → Contentment

[0152] After detecting the user's desperate mood, the emotion regulation mechanism is activated. First, the color of the EEG signal line gradually changes, slowly transitioning from a deep gray (#808080) to a gentle beige (#F5F5DC), guiding the user's mood from negative to restored. To further facilitate the mood recovery, the background color of the interface gradually changes from dark gray (#808080) to soft beige (#F5F5DC) through an animation effect. The beige background color symbolizes warmth and comfort, helping to relieve the user's inner depression and creating a soothing and healing atmosphere to assist the mood in gradually recovering from the low point. The sound effects of the background music also adjust according to the mood changes. Initially, the music presents a low-pitched string sound (such as a slow string melody), with a low frequency and a slow rhythm. As the mood gradually recovers, the background music transitions from the low-pitched string sound to a more lively guitar sound, the melody gradually becomes brighter and more pleasant, and the pitch gradually increases to help the mood gradually rise. When it is detected that the user's mood is gradually recovering, the waveform of the brainwaves also becomes more stable and regular. As the target user's mood recovers, the color of the EEG signal line gradually changes from beige (#F5F5DC) to a brighter gold (#FFD700). Gold symbolizes positive and warm emotions, indicating the rise and enhancement of the mood, and helping the user to feel psychological recovery and joy. As the mood further warms up, the color of the EEG signal line gradually turns from the warm beige (#F5F5DC) to the vibrant gold (#FFD700), accelerating the rise of the user's mood. Gold helps the user enter a more positive and energetic psychological state. As the mood gradually rises, the rhythm of the background music gradually speeds up. The music melody changes from a gentle guitar sound to a more lively and rhythmic melody, guiding the user into a pleasant emotional state, driving the mood to rise and gradually transforming from the low point to a more positive state. As the mood fully recovers, the waveform of the brainwaves becomes more regular and active, with frequencies reaching higher beta waves and gamma waves (30 - 100 Hz), showing positive, pleasant, and energetic emotions. As the mood fully rises, the melody of the background music becomes more dynamic and powerful. The rhythm of the sound effects accelerates, quickly guiding the user into a vibrant, positive, and pleasant emotional state.

[0153] In summary, the emotion regulation method based on EEG signals provided in this embodiment extracts multi-modal traditional features and deep features respectively based on the acquired EEG signals, and determines the target emotion type of the target user by using the trained emotion classification model based on the multi-modal traditional features and deep features. In this way, the emotional fluctuations of the user can be monitored and recognized in real time, and the accuracy and reliability of emotion recognition can be improved. And when the target emotion type is a negative emotion, the color of the EEG signal line can be adjusted, the background color of the display interface can be adjusted, and the background music can be adjusted to timely guide the target user to a positive emotion, thus effectively solving the key problems such as single adjustment means and lack of real-time dynamic feedback in the prior art. And it can provide more accurate and real-time emotion regulation, reduce the negative impact of emotional fluctuations on decision-making and cognition, and thus contribute to the improvement of the long-term mental health of the user.

[0154] Based on the same concept as the above emotion regulation method based on EEG signals, an embodiment of the present application also provides an emotion regulation device based on EEG signals for implementing the above emotion regulation method based on EEG signals, as Figure 4 shown. The emotion regulation device based on EEG signals includes:

[0155] A signal acquisition module for acquiring the EEG signals of the target user;

[0156] A feature extraction module for respectively extracting multi-modal traditional features and deep features based on the EEG signals; the multi-modal traditional features include features in the time domain, frequency domain, and time-frequency domain, and can characterize the dynamic characteristics, frequency domain characteristics, and complexity of the EEG signals from different dimensions; the deep features refer to multi-dimensional features related to the emotional state obtained through a convolutional neural network, and can characterize the complex patterns and potential laws of the situation;

[0157] An emotion recognition module for determining the target emotion type of the target user by using the trained emotion classification model based on the multi-modal traditional features and deep features;

[0158] An emotion regulation module for performing emotion regulation by adjusting the color of the EEG signal line, adjusting the background color of the display interface, and background music when the target emotion type is a negative emotion.

[0159] It can be understood that the emotion regulation device based on EEG signals provided in this embodiment is used to execute the above emotion regulation method based on EEG signals, so it can at least achieve the beneficial effects that the above emotion regulation method based on EEG signals can achieve, and the various embodiments of the above emotion regulation method based on EEG signals are also applicable to the emotion regulation device based on EEG signals, which will not be elaborated here.

[0160] Based on the same concept as the above emotion regulation method based on EEG signals, an embodiment of the present application also provides

[0161] Embodiments of the present application also provide an electronic device to execute the above-described emotion regulation method based on electroencephalogram signals. Please refer to Figure 5 , which shows a schematic diagram of an electronic device provided by some embodiments of the present application. As Figure 5 shown, the electronic device 5 includes: a processor 501, a memory 502, a bus 503, and a communication interface 504. The processor 501, the communication interface 504, and the memory 502 are connected through the bus 503; a computer program that can run on the processor 501 is stored in the memory 502. When the processor 501 runs the computer program, it executes the emotion regulation method based on electroencephalogram signals provided by any of the foregoing embodiments of the present application.

[0162] Among them, the memory 502 may include a high-speed random access memory (RAM: Random Access Memory), and may also include a non-volatile memory, such as at least one disk memory. Through at least one communication interface 504 (which can be wired or wireless), a communication connection is established between this device network element and at least one other network element, and the Internet, wide area network, local area network, metropolitan area network, etc. can be used.

[0163] The bus 503 may be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. Among them, the memory 502 is used to store programs. After receiving an execution instruction, the processor 501 executes the program. The emotion regulation method based on electroencephalogram signals disclosed in any of the foregoing embodiments of the present application can be applied to the processor 501 or implemented by the processor 501.

[0164] The processor 501 may be an integrated circuit chip with signal processing capabilities. In the implementation process, the steps of the above method can be completed by the integrated logic circuit of the hardware in the processor 501 or the instructions in the form of software. The above-mentioned processor 501 may be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by the combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register, etc. This storage medium is located in the memory 502, and the processor 501 reads the information in the memory 502 and combines its hardware to complete the steps of the above method.

[0165] The electronic device provided in the embodiments of the present application and the method for emotion regulation based on electroencephalogram signals provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by it.

[0166] The embodiments of the present application also provide a computer-readable storage medium corresponding to the method for emotion regulation based on electroencephalogram signals provided in the foregoing embodiments. A computer program (i.e., a program product) is stored thereon. When the computer program is run by a processor, it will execute the method for emotion regulation based on electroencephalogram signals provided in any of the foregoing embodiments.

[0167] It should be noted that the computer-readable storage medium may include, but is not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, optical disc or other optical and magnetic storage media, etc., which will not be elaborated here one by one.

[0168] The computer-readable storage medium provided in the embodiments of the present application and the method for emotion regulation based on electroencephalogram signals provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0169] The embodiments of the present application further provide a computer program product corresponding to the method for emotion regulation based on electroencephalogram signals provided in the foregoing embodiments, including a computer program, which is executed by a processor to implement the above-mentioned method for emotion regulation based on electroencephalogram signals.

[0170] The computer program product provided by the embodiments of the present application and the method for emotion regulation based on electroencephalogram signals provided by the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the method implemented when the computer program is executed by a processor.

[0171] It can be understood that the descriptions of the foregoing embodiments tend to emphasize the differences between the embodiments. The similarities or differences among them can be referred to each other. For the sake of brevity, they will not be elaborated herein.

[0172] Those skilled in the art can understand that in the above method of the specific embodiments, the writing order of each step does not mean a strict execution order and does not impose any limitation on the implementation process. The specific execution order of each step should be determined according to its function and possible internal logic.

[0173] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application, and they should all be covered within the scope of the claims and the description of the present application. In particular, as long as there is no structural conflict, the technical features mentioned in each embodiment can be combined in any way. The present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. An emotion regulation method based on electroencephalogram signals, characterized in that, The method includes: Obtaining the electroencephalogram (EEG) signal of the target user; Respectively extracting multimodal traditional features and deep features based on the EEG signal; the multimodal traditional features include features in the time domain, frequency domain, and time-frequency domain, and can characterize the dynamic characteristics, frequency domain characteristics, and complexity of the EEG signal from different dimensions; the deep features refer to multi-dimensional features related to the emotional state obtained through a convolutional neural network, and can characterize the complex patterns and potential laws of the situation; Based on the multimodal traditional features and the deep features, using a trained emotion classification model to determine the target emotion type of the target user; In the case where the target emotion type is a negative emotion, performing emotion regulation by adjusting the color of the EEG signal line, the background color of the display interface, and the background music.

2. The method according to claim 1, wherein The extracting of multimodal traditional features based on the EEG signal includes: Extracting power spectral features based on the EEG signal; the power spectral features are used to represent the energy distribution of the EEG signal in the frequency domain and reflect the activity intensity of the emotion-related frequency bands; Extracting differential entropy features based on the EEG signal; the differential entropy features are used to characterize the probability distribution of the energy-quantized signal within a specific frequency band; Extracting event-related potential features based on the EEG signal; the event-related potential features are used to reflect the time-locked response of the brain to emotional stimuli; Extracting Hjorth parameter features based on the EEG signal; the Hjorth parameter features are used to describe the dynamic characteristics of the signal, including activity features, mobility features, and complexity features, which respectively describe the overall energy of the signal, the frequency change characteristics, and the dynamic complexity; Extracting wavelet energy features based on the EEG signal; the wavelet energy features are used to describe the energy distribution of the signal in the time-frequency domain and can reflect the activity intensity of the emotion-related frequency bands at different times; Extracting wavelet entropy features based on the EEG signal; the wavelet entropy features are the entropy values based on wavelet energy and are used to quantify the complexity of the signal and reveal the relationship between the emotional state and the complexity of the EEG signal.

3. The method according to claim 2, characterized in that The extracting of power spectral features based on the EEG signal includes: Performing a fast Fourier transform on the EEG signal to convert the time-domain signal into a frequency-domain signal; Calculating the power spectral density of each frequency band based on the signal after the fast Fourier transform.

4. The method according to claim 2, wherein The extracting of Hjorth parameter features based on the EEG signal includes: Determining the activity feature based on the variance of the EEG signal; Determining the mobility feature of the EEG signal based on the first derivative of the EEG signal and the activity feature; Determining the complexity feature of the EEG signal based on the first derivative, second derivative of the EEG signal, and the mobility feature.

5. The method according to claim 1, characterized in that The extracting of deep features based on the EEG signal includes: Converting the EEG signal into a two-dimensional tensor including the number of channels and the number of time points; Successively performing time convolutional layer processing, spatial convolutional layer processing, activation layer processing, and fully connected layer processing on the two-dimensional tensor to obtain the deep features.

6. The method according to claim 1, wherein The using of a trained emotion classification model to determine the target emotion type of the target user based on the multimodal traditional features and the deep features includes: Fuse the multi-modal traditional features and the deep features to obtain a fused vector; Based on the fused vector, use the trained emotion classification model for time series modeling and multi-dimensional analysis to generate the probability distributions of various emotion types; Determine the emotion type with the largest probability distribution as the target emotion type of the target user.

7. The method according to claim 1, wherein The emotion regulation by adjusting the color of the electroencephalogram signal line, the background color of the display interface, and the background music includes: When the target user is in a negative emotion, dynamically adjust the color of the electroencephalogram signal line, and adjust the background color of the interface through a dynamic animation effect, and adjust the tone and rhythm of the background music, so that the target user gradually changes from a negative emotion to a positive emotion; During the above emotion regulation process, the background color and the background music match the color of the electroencephalogram signal line.

8. An emotion regulation device based on electroencephalogram signals, characterized in that, The emotion regulation device based on electroencephalogram signals includes: A signal acquisition module for acquiring the electroencephalogram signals of a target user; A feature extraction module for respectively extracting multi-modal traditional features and deep features based on the electroencephalogram signals; the multi-modal traditional features include features in the time domain, frequency domain, and time-frequency domain, and can characterize the dynamic characteristics, frequency domain characteristics, and complexity of electroencephalogram signals from different dimensions; the deep features refer to multi-dimensional features related to the emotional state obtained through a convolutional neural network, and can characterize the complex patterns and potential laws of the situation; An emotion recognition module for determining the target emotion type of the target user based on the multi-modal traditional features and the deep features by using a trained emotion classification model; An emotion regulation module for performing emotion regulation by adjusting the color of the electroencephalogram signal line, the background color of the display interface, and the background music when the target emotion type is a negative emotion.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor runs the computer program to implement the method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the method according to any one of claims 1-7.

Images