Emotion prediction method and device based on electroencephalogram signals

Abstract

The application provides an emotion prediction method and device based on electroencephalogram signals, which comprises the following steps: obtaining electroencephalogram signals of a target user; performing time-frequency analysis on the electroencephalogram signals to obtain target electroencephalogram characteristic signals, and extracting electroencephalogram characteristics of different frequency bands from the target electroencephalogram characteristic signals to obtain multi-dimensional electroencephalogram characteristics; calibrating a resting emotional state baseline according to the multi-dimensional electroencephalogram characteristics to obtain a calibrated emotional state baseline, and extracting features corresponding to the calibrated emotional state baseline to obtain calibrated electroencephalogram characteristics; inputting the calibrated electroencephalogram characteristics into an emotion prediction model to perform emotion prediction by using the emotion prediction model, and outputting an emotion prediction result; and performing real-time emotion monitoring and feedback based on the emotion prediction result. The application solves the problem of low emotion recognition accuracy caused by ignoring data individual differences and non-stationary characteristics in the emotion recognition process in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electroencephalogram signal analysis and processing, in particular to an emotion prediction method based on electroencephalogram signals, an emotion prediction device based on electroencephalogram signals, a computer readable storage medium and an emotion prediction system. BACKGROUND

[0002] In recent years, with the development of affective computing and neuroscience, emotion recognition technology based on electroencephalogram (EEG) signals has gradually become an important research direction in the fields of mental health monitoring, intelligent human-computer interaction, etc. As physiological data reflecting brain activity, electroencephalogram signals contain rich emotional state information and can exhibit unique advantages in non-invasiveness and real-time performance. However, due to the high individual variability, non-stationarity and susceptibility to noise interference of electroencephalogram signals, traditional emotion recognition methods are difficult to achieve high-precision and personalized emotion prediction. In addition, existing emotion prediction systems often lack real-time emotion fluctuation monitoring and feedback mechanisms, making it difficult to meet the demand for dynamic emotion monitoring in complex application scenarios. Therefore, designing an efficient, accurate and self-adaptive emotion prediction method has become a technical problem to be solved in the field of emotion recognition. SUMMARY

[0003] The main purpose of the present application is to provide an emotion prediction method based on electroencephalogram signals, an emotion prediction device based on electroencephalogram signals, a computer readable storage medium and an emotion prediction system, to at least solve the problem of low emotion recognition accuracy caused by ignoring data individual differences and non-stationary characteristics in the emotion recognition process in the prior art.

[0004] In order to achieve the above-mentioned purpose, according to one aspect of the present application, an emotion prediction method based on electroencephalogram signals is provided, comprising: acquiring electroencephalogram signals of a target user; performing time-frequency analysis on the electroencephalogram signals to obtain a target electroencephalogram feature signal, and extracting electroencephalogram features of different frequency bands from the target electroencephalogram feature signal to obtain multi-dimensional electroencephalogram features, the target electroencephalogram feature signal being an electroencephalogram signal related to emotion; calibrating a resting emotional state baseline according to the multi-dimensional electroencephalogram features to obtain a calibrated emotional state baseline, and extracting features corresponding to the calibrated emotional state baseline to obtain calibrated electroencephalogram features, the resting emotional state baseline being a baseline formed by the characteristics of the brain activity state of the target user when not subjected to emotional fluctuations or external stimuli; inputting the calibrated electroencephalogram features into an emotion prediction model to perform emotion prediction using the emotion prediction model, and outputting an emotion prediction result, the emotion prediction model being a converged model obtained by iteratively training a predetermined model using a plurality of sets of training data using a neural network, each set of training data including sample electroencephalogram features and an emotional state corresponding to the sample electroencephalogram features; and performing real-time emotion monitoring and feedback based on the emotion prediction result.

[0005] Optionally, the electroencephalogram signal is subjected to time-frequency analysis to obtain a target electroencephalogram feature signal, including: decomposing the electroencephalogram signal by using wavelet transform to generate a first electroencephalogram feature signal, the first electroencephalogram feature signal being a global electroencephalogram signal of multiple scales and multiple frequency bands; decomposing the first electroencephalogram feature signal into multiple groups of intrinsic mode functions by using an empirical mode decomposition method to obtain a characteristic signal corresponding to an emotion of different frequency components in each intrinsic mode function, thereby obtaining the target electroencephalogram feature signal.

[0006] Optionally, electroencephalogram features of different frequency bands are extracted from the target electroencephalogram feature signal to obtain multi-dimensional electroencephalogram features, including: generating a time-frequency spectrum according to the target electroencephalogram feature signal; and extracting features of the time-frequency spectrum by using a convolutional neural network to generate the multi-dimensional electroencephalogram features.

[0007] Optionally, before the multi-dimensional electroencephalogram features are calibrated according to a resting emotional state baseline to obtain calibrated electroencephalogram features, the method further includes: acquiring resting electroencephalogram signal data under the condition that the target user is in a resting state, the resting state being a brain activity state of the target user when not subjected to emotional fluctuations or external stimuli; extracting a characteristic frequency band corresponding to the resting electroencephalogram signal data by analyzing the resting electroencephalogram signal data; and determining a resting emotional state baseline of the target user according to the characteristic frequency band.

[0008] Optionally, the resting emotional state baseline is calibrated according to the multi-dimensional electroencephalogram features to obtain a calibrated emotional state baseline, including: segmenting the multi-dimensional electroencephalogram features by using a sliding window method to extract a first electroencephalogram feature from each sliding window; comparing the first electroencephalogram feature with the resting emotional state baseline; and updating the resting emotional state baseline by weighting to obtain the calibrated emotional state baseline.

[0009] Optionally, before the calibration electroencephalogram feature is input into the emotion prediction model to utilize the emotion prediction model to perform emotion prediction and output an emotion prediction result, the method further comprises: determining a model structure of the emotion prediction model, the model structure comprising an input layer, an intermediate convolutional layer, and an output layer, the intermediate convolutional layer comprising a convolutional neural network layer, a long short-term memory network layer, a convolutional layer, and a recurrent layer, the convolutional neural network layer being configured to extract an electroencephalogram feature signal related to emotion, the long short-term memory network layer being configured to capture changes in the electroencephalogram feature signal in a time sequence, the convolutional layer being configured to receive the electroencephalogram feature signal of the convolutional neural network layer and perform feature processing on the electroencephalogram feature signal to obtain an emotion feature, and the recurrent layer being configured to receive changes in the electroencephalogram feature signal of the output of the long short-term memory network layer in a time sequence and identify changes in the emotion feature in the time sequence; constructing an initial emotion prediction model according to the model structure; and iteratively training the initial emotion prediction model using a plurality of sets of training data by means of a neural network to obtain the emotion prediction model.

[0010] Optionally, based on the emotion prediction result, real-time emotion monitoring and feedback are performed, including: in a closed-loop feedback system, generating a feedback signal based on the emotion prediction result, and recording a history of changes in the emotional state of the target user according to the feedback signal.

[0011] According to another aspect of the present application, an emotion prediction device based on electroencephalogram signals is provided, the device comprising: an acquisition unit configured to acquire electroencephalogram signals of a target user; an analysis unit configured to perform time-frequency analysis on the electroencephalogram signals to obtain a target electroencephalogram feature signal, and extract electroencephalogram features of different frequency bands from the target electroencephalogram feature signal to obtain multi-dimensional electroencephalogram features, the target electroencephalogram feature signal being an electroencephalogram signal related to emotion; a calibration unit configured to calibrate a resting emotional state baseline according to the multi-dimensional electroencephalogram features to obtain a calibrated emotional state baseline, and extract features corresponding to the calibrated emotional state baseline to obtain calibration electroencephalogram features, the resting emotional state baseline being a baseline formed by features of a brain activity state of the target user when not subjected to emotional fluctuations or external stimuli; an input-output unit configured to input the calibration electroencephalogram features into an emotion prediction model to utilize the emotion prediction model to perform emotion prediction and output an emotion prediction result, the emotion prediction model being a converged model obtained by iteratively training a predetermined model using a plurality of sets of training data by means of a neural network, each set of the training data comprising sample electroencephalogram features and an emotional state corresponding to the sample electroencephalogram features; and a monitoring and feedback unit configured to perform real-time emotion monitoring and feedback based on the emotion prediction result.

[0012] According to still another aspect of the present application, a computer readable storage medium is provided, the computer readable storage medium comprising a stored program, wherein the computer readable storage medium is caused to perform any of the above-mentioned emotion prediction methods based on electroencephalogram signals when the program is run.

[0013] According to still another aspect of the present application, an emotion prediction system is provided, comprising one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs comprise instructions for performing any of the above-mentioned emotion prediction methods based on electroencephalogram signals.

[0014] By applying the technical solution of the present application, the emotion-related electroencephalogram features are extracted through multi-level time-frequency analysis, the personalized emotion baseline calibration method is used to cope with physiological differences of different users, and the personalized adaptation of the emotional state is realized. The deep learning model, i.e., the above-mentioned emotion prediction model, is used to perform time series analysis on the extracted features to improve the accuracy of emotion prediction, and further through the closed-loop feedback system, the emotional fluctuation is monitored in real time, and support is provided for emotion regulation and mental health. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 is a hardware structure block diagram of a mobile terminal for performing an emotion prediction method based on electroencephalogram signals according to an embodiment of the present application;

[0016] Figure 2 is a flowchart of an emotion prediction method based on electroencephalogram signals according to an embodiment of the present application;

[0017] Figure 3 is a structure block diagram of an emotion prediction device based on electroencephalogram signals according to an embodiment of the present application.

[0018] Among them, the above-mentioned drawings include the following reference signs:

[0019] 102, processor; 104, memory; 106, transmission device; 108, input and output device. DETAILED DESCRIPTION

[0020] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0021] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] As described in the background section, the prior art ignores individual differences and non-stationary features in the emotion recognition process, resulting in low accuracy in emotion recognition. To address the above problems, embodiments of this application provide a method for performing emotion prediction based on electroencephalogram (EEG) signals, an emotion prediction device based on EEG signals, a computer-readable storage medium, and an emotion prediction system.

[0024] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0025] The methods and embodiments provided in this application can be executed on a mobile terminal, a computer terminal, or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for an emotion prediction method based on electroencephalogram (EEG) signals, according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the sameFigure 1 different configurations.

[0026] The memory 104 can be used to store computer programs, such as software programs of application software and modules, such as a computer program corresponding to the emotion prediction method based on electroencephalogram signals in the embodiments of the present application. The processor 102 can execute various functional applications and data processing, i.e., implement the above method, by running the computer program stored in the memory 104. The memory 104 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory remotely arranged with respect to the processor 102, and the remote memory can be connected to the mobile terminal through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The specific examples of the network can include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet in a wireless manner.

[0027] In the embodiments, an emotion prediction method based on electroencephalogram signals running on a mobile terminal, a computer terminal, or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown.

[0028] Figure 2 is a flowchart of the emotion prediction method based on electroencephalogram signals according to the embodiments of the present application. As shown in Figure 2 the method includes the following steps:

[0029] In step S201, the electroencephalogram signals of a target user are acquired.

[0030] Specifically, by analyzing the acquired electroencephalogram of the target user, the pattern and characteristics of the brain activity can be understood, and the emotional state can be inferred. The electroencephalogram can provide objective data about the user's emotions, which is more accurate and objective than traditional self-reporting methods. By analyzing the electroencephalogram, the brain activity patterns associated with different emotional states, such as happiness, stress, and anxiety, can be identified. This allows for more accurate prediction of the target user's emotional state.

[0031] In step S202, time-frequency analysis is performed on the electroencephalogram to obtain a target electroencephalogram feature signal, and electroencephalogram features in different frequency bands are extracted from the target electroencephalogram feature signal to obtain multi-dimensional electroencephalogram features. The target electroencephalogram feature signal is an electroencephalogram signal related to emotions.

[0032] Specifically, by performing time-frequency analysis on the electroencephalogram, extracting the target electroencephalogram feature signal and further extracting the electroencephalogram features in different frequency bands, multi-dimensional electroencephalogram features are obtained, thereby realizing the extraction and analysis of the electroencephalogram signal related to emotions. This can effectively capture the features in the electroencephalogram signal related to emotions, providing more accurate and comprehensive information for subsequent emotion prediction. By analyzing multi-dimensional electroencephalogram features, the emotional state of the subject can be more comprehensively understood, thereby realizing accurate prediction of emotions. Through analysis of the electroencephalogram, the relationship between emotions and brain activity can also be explored.

[0033] In step S203, the resting emotional state baseline is calibrated based on the multi-dimensional electroencephalogram features to obtain a calibrated emotional state baseline, and the features corresponding to the calibrated emotional state baseline are extracted to obtain calibrated electroencephalogram features. The resting emotional state baseline is a baseline formed by the characteristics of the brain activity state of the target user when there is no emotional fluctuation or external stimulation.

[0034] Specifically, based on the extracted multi-dimensional electroencephalogram features, individual emotional state baseline calibration is performed. By calibrating the resting emotional state baseline, the emotional characteristics of the target user can be more accurately extracted, thereby improving the accuracy and reliability of emotion prediction. The calibrated electroencephalogram features obtained by calibrating the emotional state baseline can better reflect the emotional state of the target user, thereby improving the accuracy and stability of emotion prediction. By comparing the differences between the calibrated emotional state baseline and the actual emotional state, the emotional state of the target user can be more accurately identified and predicted, providing a more effective basis for further emotional intervention and regulation.

[0035] In step S204, the calibrated electroencephalogram features are input into an emotion prediction model to perform emotion prediction using the emotion prediction model, and an emotion prediction result is output. The emotion prediction model is a converged model obtained by iteratively training a predetermined model using multiple sets of training data with a neural network. Each set of training data includes sample electroencephalogram features and the emotional state corresponding to the sample electroencephalogram features.

[0036] Specifically, the calibrated electroencephalogram features are input into the emotion prediction model for emotion prediction, the training data are iteratively trained by a neural network to obtain a converged model, and an emotion prediction model based on the neural network is constructed, so as to realize the prediction of emotions, more accurately predict the emotional state of individuals, and help individuals better understand and manage their emotions in practical applications, and improve the effect of emotion regulation.

[0037] Step S205, based on the emotion prediction result, real-time emotion monitoring and feedback are performed.

[0038] Specifically, by monitoring the electroencephalogram signals of the testee, the system can capture their emotional state in real time, and according to the prediction result, the system can give feedback, such as reminding the testee to take relaxation methods when the emotional fluctuation is large, or providing encouragement and support when the emotion is low. Real-time emotion monitoring and feedback can help the testee better understand their emotional changes and take timely measures to regulate emotions, thereby improving emotional management ability and mental health level, helping the testee better understand and control their emotions, and improving life quality and work efficiency.

[0039] In this embodiment, the emotion-related electroencephalogram features are extracted through multi-level time-frequency analysis, and the individualized emotion baseline calibration method is used to cope with physiological differences of different users, so as to realize the individualized adaptation of emotional state. A deep learning model, i.e., the above-mentioned emotion prediction model, is used to perform time series analysis on the extracted features to improve the accuracy of emotion prediction, and further through a closed-loop feedback system to monitor the emotional fluctuation in real time, to provide support for emotion regulation and mental health.

[0040] Specifically, the electroencephalogram (EEG) can be used to obtain the electroencephalogram signals of the target user. Electroencephalogram is a technology that records brain electrical activity by placing electrodes on the scalp. Since the electroencephalogram signals are affected by many factors, such as electrode position, hair, muscle activity, etc., professional processing and correction are needed in the data acquisition and processing process to ensure accurate emotion prediction results.

[0041] Specifically, time-frequency analysis is a common signal processing technique for analyzing the characteristics of signal changes in time and frequency. By performing Fourier transform on the electroencephalogram signal, the signal can be converted from time domain to frequency domain, thereby obtaining the energy distribution of the signal at different frequencies, which helps to understand the frequency characteristics of brain activity under different emotional states, such as the enhancement of specific frequency components under anxiety state. It can also be used to detect event-related potentials (ERP) in electroencephalogram signals, which is a brain electrical activity related to specific stimulus events. By analyzing the time-frequency characteristics of ERP, the response pattern of the brain to stimulation under different emotional states can be more accurately captured, thereby realizing the prediction and classification of emotions.

[0042] In the implementation process, the time-frequency analysis of the electroencephalogram signal is performed to obtain the target electroencephalogram feature signal, which can be achieved by the following steps: decomposing the electroencephalogram signal by wavelet transform to generate a first electroencephalogram feature signal, which is a global electroencephalogram signal with multiple scales and multiple frequency bands; and decomposing the first electroencephalogram feature signal into a plurality of intrinsic mode functions by using the empirical mode decomposition method to obtain the emotional corresponding feature signal of different frequency components in each intrinsic mode function, thereby obtaining the target electroencephalogram feature signal.

[0043] In this scheme, the target electroencephalogram feature signal is obtained by time-frequency analysis, which includes a global electroencephalogram signal with multiple scales and multiple frequency bands, and emotional corresponding feature signals of different frequency components. Through the decomposition and processing of the electroencephalogram signal, the features related to emotions in brain activity can be more accurately captured, thereby realizing the prediction of emotions, which can provide more accurate and reliable data support for emotion recognition and emotion monitoring, and help to further study the relationship between emotions and electroencephalogram signals.

[0044] Specifically, the electroencephalogram signal data is first decomposed by wavelet transform to generate a multi-scale, multi-frequency band electroencephalogram feature signal (i.e., a first electroencephalogram feature signal), thereby extracting a global feature signal. The electroencephalogram signal (i.e., the first electroencephalogram feature signal) is decomposed into a group of intrinsic mode functions (IMFs) (i.e., a plurality of intrinsic mode functions) by using the empirical mode decomposition (EMD) method, thereby obtaining emotional corresponding features of different frequency components in each IMF, i.e., extracting each local feature signal, and obtaining electroencephalogram features of different frequency bands, thereby obtaining the target electroencephalogram feature signal.

[0045] It should be noted that the wavelet transform described above is a signal processing technique that can decompose a signal into frequency components of different scales. The electroencephalogram signal is processed using wavelet transform to extract features related to emotions, that is, the collected electroencephalogram signal is preprocessed, such as noise removal, filtering and other operations. The electroencephalogram signal is decomposed into components of different frequencies using wavelet transform, different wavelet basis functions can be selected to achieve different decomposition effects, and the energy or other features of each frequency component are calculated to reflect the activity of the electroencephalogram signal in different frequency ranges.

[0046] It should be noted that the empirical mode decomposition (EMD) described above is a signal processing method that can decompose non-stationary and nonlinear signals into multiple intrinsic mode functions (IMF). The electroencephalogram signal is decomposed into multiple IMFs using the EMD method, and the spectral features and energy distribution of each IMF are analyzed to obtain emotion-related information in the electroencephalogram signal.

[0047] Specifically, the electroencephalogram signal is first decomposed by EMD to obtain multiple IMF components, then the frequency spectrum of each IMF is analyzed, and its frequency distribution and frequency intensity are calculated. At the same time, the energy distribution of each IMF can also be calculated to understand the signal energy size in different frequency ranges. Finally, combining the frequency spectrum and energy information of all IMFs, an emotion prediction model can be established to predict the emotional state of the subject by analyzing the IMF features in the electroencephalogram signal.

[0048] In the specific implementation process, the brain electrical characteristics of different frequency bands are extracted from the target brain electrical characteristic signal to obtain multi-dimensional brain electrical characteristics, which can be achieved by the following steps: generating a time-frequency spectrogram according to the target brain electrical characteristic signal; and extracting features of the time-frequency spectrogram using a convolutional neural network to generate multi-dimensional brain electrical characteristics.

[0049] In this scheme, by extracting brain electrical characteristics of different frequency bands, generating a time-frequency spectrogram, and using a convolutional neural network to extract features of the time-frequency spectrogram, multi-dimensional brain electrical characteristics are obtained, which can more comprehensively capture information in the electroencephalogram signal, improve the accuracy and reliability of emotion prediction, and use multi-dimensional brain electrical characteristics for emotion prediction to more accurately analyze the emotional state of individuals, helping to understand the mechanism of emotion regulation and the mechanism of emotional disorders. At the same time, through the application of deep learning algorithms, a large amount of electroencephalogram signal data can be effectively processed, improving the efficiency and accuracy of emotion prediction.

[0050] Specifically, a time-frequency spectrogram is generated for the electroencephalogram signal data (i.e., the target brain electrical characteristic signal) processed by time-frequency analysis, and a convolutional neural network (CNN) is used to extract features of the time-frequency spectrogram to generate multi-dimensional brain electrical characteristic signals for emotion prediction.

[0051] It should be noted that the above-mentioned convolutional neural network (Convolutional Neural Network, CNN) is a deep learning model widely used in image recognition and speech recognition, etc. It can extract features from electroencephalogram data to help identify and predict emotional states. Through multiple convolutional layers and pooling layers, different levels of feature information are extracted, and through fully connected layers and softmax layers, the prediction results of emotional states are output. In the training process, the parameters of CNN can be optimized through the back propagation algorithm, so that it can more accurately predict the emotional state.

[0052] In the specific implementation process, before the multi-dimensional electroencephalogram features are calibrated according to the resting emotional state baseline to obtain the calibrated electroencephalogram features, the above method further includes the following steps: acquiring resting electroencephalogram signal data of the target user in a resting state, the resting state being the brain activity state of the target user without emotional fluctuations or external stimuli; extracting the feature frequency band corresponding to the resting electroencephalogram signal data by analyzing the resting electroencephalogram signal data; and determining the resting emotional state baseline of the target user according to the feature frequency band.

[0053] In this scheme, acquiring electroencephalogram signal data of the target user in a resting state can help determine the brain activity state of the user without emotional fluctuations or external stimuli. By analyzing the resting electroencephalogram signal data and extracting the corresponding feature frequency band, the resting emotional state baseline of the user can be determined. According to the baseline, the emotional state of the user can be more accurately calibrated, thereby improving the accuracy and reliability of emotion prediction, and better understanding and analyzing the brain activity of the user, providing stronger basis and support for emotion prediction.

[0054] Specifically, the electroencephalogram signal data of the user in a resting state is collected, the data is analyzed, the feature frequency band is extracted, and the initial resting emotional baseline parameter of the user is determined, wherein the resting state refers to the brain activity state of the user without significant emotional fluctuations or external stimuli.

[0055] In the specific implementation process, the resting emotional state baseline is calibrated according to the multi-dimensional electroencephalogram features to obtain the calibrated emotional state baseline, which can be realized by the following steps: segmenting the multi-dimensional electroencephalogram features using a sliding window method, extracting first electroencephalogram features from each sliding window; and comparing the first electroencephalogram features with the resting emotional state baseline, and updating the resting emotional state baseline by weighting to obtain the calibrated emotional state baseline.

[0056] In the scheme, the accuracy and stability of emotion prediction can be improved by calibrating the baseline of emotional state, and by using multi-dimensional EEG features for segmentation processing and extracting first EEG features, which are compared with and weightedly updated by the resting emotional state baseline, the emotional state changes of individuals can be better reflected, and the accuracy of emotion prediction can be improved, making the prediction result more reliable and effective.

[0057] Specifically, the subsequent collected real-time EEG signals are segmented by using a sliding window method, new EEG features are extracted in each sliding window, and are compared with the resting emotional baseline, wherein the time length and step length of the sliding window are set according to the specific emotional monitoring requirements, and the feature data generated by each window is used to weightly update the resting baseline parameters.

[0058] It should be noted that the above sliding window method is a commonly used signal processing technique for analyzing local features in continuous signals, extracting time domain and frequency domain features of EEG signals, and inputting these features into an emotion classifier for emotion prediction. First, the continuous EEG signals are divided into fixed length windows, then the features of the EEG signals in each window are calculated, which can include time domain features (such as mean, variance, skewness, etc.) and frequency domain features (such as power spectral density, frequency band energy, etc.), which can be used to train an emotion classifier for predicting the emotional state of the subject.

[0059] In the specific implementation process, before the calibrated EEG features are input into the emotion prediction model to use the emotion prediction model for emotion prediction and output the emotion prediction result, the above method further includes the following steps: determining the model structure of the emotion prediction model, the model structure including an input layer, an intermediate convolutional layer and an output layer, the intermediate convolutional layer including a convolutional neural network layer, a long short-term memory network layer, a convolutional layer and a recurrent layer, the convolutional neural network layer being used to extract EEG feature signals related to emotions, the long short-term memory network layer being used to capture changes in EEG feature signals in time series, the convolutional layer being used to receive EEG feature signals of the convolutional neural network layer and perform feature processing on the EEG feature signals to obtain emotion features, and the recurrent layer receiving changes in EEG feature signals in time series of the output of the long short-term memory network layer and identifying changes in emotion features in time series; constructing an initial emotion prediction model according to the model structure; using a plurality of sets of training data to iteratively train the initial emotion prediction model using a neural network to obtain the emotion prediction model.

[0060] In the scheme, the model structure of the emotion prediction model is determined, including an input layer, an intermediate convolutional layer, and an output layer, the intermediate convolutional layer includes a convolutional neural network layer, a long short-term memory network layer, a convolutional layer, and a recurrent layer, which can extract the electroencephalogram feature signals related to emotions and capture the changes of the electroencephalogram feature signals in time series, and finally obtain the emotion features. Through iterative training, an optimized emotion prediction model is obtained, which can more accurately predict the emotional state of the individual, and can help researchers and clinicians better understand and predict the emotional state of the individual, providing an important reference for emotion management and treatment.

[0061] Specifically, a deep learning model is introduced as a baseline adjustment mechanism to dynamically adjust the baseline parameters of the emotional state according to the real-time electroencephalogram signal fluctuation information of the user. The emotion prediction model includes a convolutional neural network CNN and a long short-term memory network LSTM, wherein the CNN is used to extract the electroencephalogram feature signals corresponding to the emotions, and the LSTM is used to capture the changes of the electroencephalogram signal features in time series. Meanwhile, the neural network structure contains a multi-layer fusion model, wherein the convolutional layer is used to extract local emotional features, and the recurrent layer is used to identify the time series changes of the emotional features.

[0062] It should be noted that the above-mentioned long short-term memory network (Long Short-Term Memory, LSTM) is a commonly used recurrent neural network model, which is particularly suitable for processing sequence data and can be used to capture long-term dependencies in electroencephalogram signal sequences, thereby more accurately predicting the emotional state of the subject. By introducing a gating unit to control the flow of information, the problems of gradient vanishing and gradient explosion in traditional recurrent neural networks are effectively avoided.

[0063] In the specific implementation process, based on the emotion prediction result, real-time emotion monitoring and feedback are performed, which can be realized by the following steps: generating a feedback signal based on the emotion prediction result in a closed-loop feedback system, and recording the emotional state change history of the target user according to the feedback signal.

[0064] In the scheme, the emotional state of the user can be monitored in real time, and feedback signals can be generated according to the prediction results to help the user adjust emotions. By recording the emotional state change history of the user, the emotional change pattern of the user can be better understood, helping the user to better manage emotions, and the user can more effectively cope with emotional fluctuations, improving the user's emotional management ability, thereby improving the user's quality of life and work efficiency.

[0065] Specifically, the closed-loop feedback system generates a feedback signal based on the emotion fluctuation detection result, and records the emotional state change history of the user.

[0066] The technical scheme provided by the above embodiment of the present application realizes personalized and dynamic monitoring of emotional states through advanced time-frequency analysis technology and adaptive emotion baseline calibration. The neural network structure of the present application is aimed at feature extraction and time-dependent processing of electroencephalogram signals, and builds a more accurate emotion prediction model. In addition, the real-time tracking and data recording of emotional states are realized by the design of a closed-loop feedback system, so that it not only has high-precision emotion prediction capability, but also can meet the emotional monitoring needs in complex scenarios. This method effectively improves the stability and applicability of emotion prediction, and expands the application of electroencephalogram signals in the fields of emotion recognition and intelligent interaction.

[0067] As can be seen from the above, in the embodiment of the present application, a kind of emotion prediction method based on electroencephalogram signal is proposed for the limitation of traditional emotion prediction method, to solve the problems of data individual difference, non-stationary feature processing and real-time emotional feedback in the process of emotion recognition. Through multi-level time-frequency analysis, the electroencephalogram features related to emotion are extracted. Based on the individualized emotion baseline calibration method, the physiological differences of different users are dealt with, and the individualization of emotional state is realized. In addition, a deep learning model combining convolutional neural network (CNN) and long short-term memory network (LSTM) is used to perform time series analysis on the extracted features to improve the accuracy of emotion prediction. Further, through the closed-loop feedback system, the emotional fluctuation is monitored in real time, which provides support for emotion regulation and mental health.

[0068] In order to enable those skilled in the art to more clearly understand the technical solutions of the present application, the implementation process of the emotion prediction method based on electroencephalogram signal of the present application will be described in detail below in conjunction with specific embodiments.

[0069] The present embodiment relates to a specific emotion prediction method based on electroencephalogram signal, comprising the following steps: data acquisition module, data processing module, closed-loop feedback system and emotion monitoring module.

[0070] 1) The data acquisition module is used to acquire the electroencephalogram signal of the user.

[0071] 2) The data processing module is used to carry the algorithm model.

[0072] 3) The closed-loop feedback system is used to continuously receive the real-time data stream provided by the emotion detection module and analyze the emotional fluctuation.

[0073] 4) The emotion monitoring module is used to identify the change of emotional features in electroencephalogram signal and generate feedback signal.

[0074] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in a different order.

[0075] The embodiment of the present application also provides an emotion prediction device based on electroencephalogram signals. It should be noted that the emotion prediction device based on electroencephalogram signals of the embodiment of the present application can be used to execute the emotion prediction method based on electroencephalogram signals provided by the embodiment of the present application. The device is used to realize the above-mentioned embodiment and preferred embodiment, and will not be described here. As used below, the term "module" can be a combination of software and / or hardware that realizes a predetermined function. Although the device described in the following embodiment is preferably realized in software, hardware, or a combination of software and hardware is also possible and is contemplated.

[0076] The following describes the emotion prediction device based on electroencephalogram signals provided by the embodiment of the present application.

[0077] Figure 3 is a structural block diagram of the emotion prediction device based on electroencephalogram signals according to the embodiment of the present application. As shown in Figure 3 , the device comprises an acquisition unit 301, an analysis unit 302, a calibration unit 303, an input and output unit 304, and a monitoring feedback unit 305.

[0078] The acquisition unit 301 is configured to acquire electroencephalogram signals of a target user.

[0079] The analysis unit 302 is configured to perform time-frequency analysis on the electroencephalogram signals to obtain a target electroencephalogram feature signal, and extract electroencephalogram features of different frequency bands from the target electroencephalogram feature signal to obtain multi-dimensional electroencephalogram features. The target electroencephalogram feature signal is an electroencephalogram signal related to emotion.

[0080] The calibration unit 303 is configured to calibrate a resting emotional state baseline according to the multi-dimensional electroencephalogram features to obtain a calibrated emotional state baseline, and extract features corresponding to the calibrated emotional state baseline to obtain a calibrated electroencephalogram feature. The resting emotional state baseline is a baseline formed by the characteristics of the brain activity state of the target user when he or she is not affected by emotional fluctuations or external stimuli.

[0081] The input and output unit 304 is configured to input the calibrated electroencephalogram feature into an emotion prediction model to perform emotion prediction using the emotion prediction model, and output an obtained emotion prediction result. The emotion prediction model is a converged model obtained by iteratively training a predetermined model using a neural network with a plurality of training data sets. Each training data set includes a sample electroencephalogram feature and a sample electroencephalogram feature corresponding emotional state.

[0082] The monitoring feedback unit 305 is configured to perform real-time emotion monitoring and feedback based on the emotion prediction result.

[0083] In this embodiment, emotion-related electroencephalogram features are extracted through multi-level time-frequency analysis, and a personalized emotion baseline calibration method is used to cope with physiological differences of different users, so as to realize personalized adaptation of emotional state. A deep learning model, i.e., the above-mentioned emotion prediction model, is used to perform time series analysis on the extracted features to improve the accuracy of emotion prediction, and further through a closed-loop feedback system, real-time monitoring of emotional fluctuations is realized to provide support for emotion regulation and mental health.

[0084] In the specific implementation process, the analysis unit includes a first decomposition module and a second decomposition module. The first decomposition module is configured to decompose the electroencephalogram signal by using wavelet transform to generate a first electroencephalogram feature signal, which is a global electroencephalogram signal of multiple scales and multiple frequency bands. The second decomposition module is configured to decompose the first electroencephalogram feature signal into multiple groups of intrinsic mode functions by using an empirical mode decomposition method to obtain an emotion corresponding feature signal of different frequency components in each intrinsic mode function, thereby obtaining a target electroencephalogram feature signal.

[0085] In this scheme, the target electroencephalogram feature signal is obtained through time-frequency analysis, which includes a global electroencephalogram signal of multiple scales and multiple frequency bands and an emotion corresponding feature signal of different frequency components. Through decomposition and processing of the electroencephalogram signal, the features related to emotion in brain activity can be more accurately captured, thereby realizing prediction of emotion and providing more accurate and reliable data support for emotion recognition and emotion monitoring, which helps to further study the relationship between emotion and electroencephalogram signal.

[0086] In the specific implementation process, the analysis unit includes a first generation module and a second generation module. The first generation module is configured to generate a time-frequency spectrum based on the target electroencephalogram feature signal. The second generation module is configured to extract features of the time-frequency spectrum by using a convolutional neural network to generate multi-dimensional electroencephalogram features.

[0087] In this scheme, by extracting electroencephalogram features of different frequency bands, generating a time-frequency spectrum, and extracting features of the time-frequency spectrum by using a convolutional neural network, multi-dimensional electroencephalogram features are obtained, which can more comprehensively capture information in the electroencephalogram signal, improve the accuracy and reliability of emotion prediction, and use multi-dimensional electroencephalogram features for emotion prediction to more accurately analyze the emotional state of individuals, helping to understand the mechanism of emotion regulation and the mechanism of emotional disorders. At the same time, through the application of deep learning algorithms, a large amount of electroencephalogram signal data can be effectively processed, and the efficiency and accuracy of emotion prediction can be improved.

[0088] In the implementation process, the device further includes a second acquisition unit, a second analysis unit and a first determination unit. The second acquisition unit is configured to acquire the resting EEG signal data when the target user is in a resting state before the multi-dimensional EEG features are calibrated according to the resting emotional state baseline, and the resting state is the brain activity state of the target user when the target user is not affected by emotional fluctuations or external stimuli. The second analysis unit is configured to extract the characteristic frequency band corresponding to the resting EEG signal data by analyzing the resting EEG signal data. The first determination unit is configured to determine the resting emotional state baseline of the target user according to the characteristic frequency band.

[0089] In this scheme, acquiring the EEG signal data of the target user in the resting state can help determine the brain activity state of the user when the user is not affected by emotional fluctuations or external stimuli. By analyzing the resting EEG signal data and extracting the corresponding characteristic frequency band, the resting emotional state baseline of the user can be determined. According to the baseline, the emotional state of the user can be calibrated more accurately, thereby improving the accuracy and reliability of emotion prediction, and better understanding and analyzing the brain activity of the user, providing stronger basis and support for emotion prediction.

[0090] In the implementation process, the calibration unit includes a processing module and a comparison module. The processing module is configured to segment the multi-dimensional EEG features by using a sliding window method, and extract the first EEG features from each sliding window. The comparison module is configured to compare the first EEG features with the resting emotional state baseline, and update the resting emotional state baseline by weighting, to obtain the calibrated emotional state baseline.

[0091] In this scheme, the accuracy and stability of emotion prediction can be improved by calibrating the emotional state baseline. By segmenting and processing the multi-dimensional EEG features to extract the first EEG features, and comparing and updating the resting emotional state baseline by weighting, the emotional state changes of individuals can be better reflected, and the accuracy of emotion prediction can be improved, making the prediction results more reliable and effective.

[0092] In the implementation process, the device further includes a second determination unit, a construction unit and a training unit. The second determination unit is configured to determine a model structure of the emotion prediction model before inputting the calibrated electroencephalogram features into the emotion prediction model to perform emotion prediction by using the emotion prediction model and outputting an obtained emotion prediction result. The model structure includes an input layer, an intermediate convolutional layer and an output layer. The intermediate convolutional layer includes a convolutional neural network layer, a long short-term memory network layer, a convolutional layer and a recurrent layer. The convolutional neural network layer is configured to extract electroencephalogram features signals related to emotions. The long short-term memory network layer is configured to capture changes in the electroencephalogram feature signals in a time sequence. The convolutional layer is configured to receive the electroencephalogram feature signals of the convolutional neural network layer and perform feature processing on the electroencephalogram feature signals to obtain emotion features. The recurrent layer receives changes in the electroencephalogram feature signals of the output of the long short-term memory network layer in a time sequence and identifies changes in the emotion features in a time sequence. The construction unit is configured to construct an initial emotion prediction model according to the model structure. The training unit is configured to use a plurality of sets of training data to perform iterative training on the initial emotion prediction model by using a neural network to obtain the emotion prediction model.

[0093] In the scheme, the model structure of the emotion prediction model includes an input layer, an intermediate convolutional layer and an output layer. The intermediate convolutional layer includes a convolutional neural network layer, a long short-term memory network layer, a convolutional layer and a recurrent layer. The emotion prediction model can extract electroencephalogram feature signals related to emotions and capture changes in the electroencephalogram feature signals in a time sequence to finally obtain emotion features. Through iterative training, an optimized emotion prediction model is obtained, which can more accurately predict the emotional state of an individual and help researchers and clinicians better understand and predict the emotional state of an individual, providing an important reference for emotion management and treatment.

[0094] In the implementation process, the monitoring feedback unit includes a third generation module. The third generation module is configured to generate a feedback signal based on the emotion prediction result in the closed-loop feedback system, to record the history of changes in the emotional state of the target user according to the feedback signal.

[0095] In the scheme, the emotional state of the user can be monitored in real time, and the feedback signal can be generated according to the prediction result to help the user adjust the emotion. By recording the history of changes in the emotional state of the user, the emotional change pattern of the user can be better understood, the user can better manage the emotion, the user can more effectively cope with emotional fluctuations, the emotional management ability of the user can be improved, and thus the life quality and work efficiency of the user can be improved.

[0096] The emotion prediction device based on the electroencephalogram signal includes a processor and a memory, and the acquisition unit, the analysis unit, the calibration unit, the input / output unit, and the monitoring feedback unit are all stored in the memory as program units, and the corresponding functions are realized by the processor executing the program units stored in the memory. The modules are all located in the same processor, or the modules are located in different processors in any combination.

[0097] The processor includes a core, and the core calls the corresponding program unit in the memory. The core can be one or more, and the problem of low emotion recognition accuracy caused by ignoring the data individual difference and non-stationary characteristics in the emotion recognition process in the prior art can be solved by adjusting the core parameters.

[0098] The memory can include a non-permanent memory in a computer readable medium, a random access memory (RAM), and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM), and the memory includes at least one memory chip.

[0099] The embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium includes a stored program, wherein the computer readable storage medium controls a device in which the computer readable storage medium is located to execute the emotion prediction method based on the electroencephalogram signal when the program runs.

[0100] Specifically, the emotion prediction method based on the electroencephalogram signal includes:

[0101] In step S201, the electroencephalogram signal of a target user is acquired.

[0102] Specifically, by analyzing the acquired electroencephalogram signal of the target user, the mode and characteristics of the brain activity can be understood, so as to infer the emotional state, the electroencephalogram signal can provide objective data about the user's emotion, which is more accurate and objective than the traditional self-reporting method, by analyzing the electroencephalogram signal, the brain activity mode related to different emotional states can be identified, for example, the emotional states of joy, stress and anxiety will show different characteristics in the brain, and the emotional state of the target user can be more accurately predicted.

[0103] In step S202, the electroencephalogram signal is subjected to time-frequency analysis to obtain a target electroencephalogram feature signal, and electroencephalogram features of different frequency bands are extracted from the target electroencephalogram feature signal to obtain multi-dimensional electroencephalogram features, and the target electroencephalogram feature signal is an electroencephalogram signal related to emotion.

[0104] Specifically, by performing time-frequency analysis on the electroencephalogram signal, the target electroencephalogram feature signal is extracted and the electroencephalogram features of different frequency bands are further extracted to obtain multi-dimensional electroencephalogram features, thereby realizing the extraction and analysis of the electroencephalogram signal related to emotion, which can effectively capture the features related to emotion in the electroencephalogram signal, provide more accurate and comprehensive information for subsequent emotion prediction, and through the analysis of multi-dimensional electroencephalogram features, the emotional state of the subject can be more comprehensively understood, thereby realizing accurate prediction of emotion, and through the analysis of electroencephalogram signal, the relationship between emotion and brain activity can be explored.

[0105] In step S203, the resting emotional state baseline is calibrated according to the multi-dimensional electroencephalogram features to obtain a calibrated emotional state baseline, and the features corresponding to the calibrated emotional state baseline are extracted to obtain calibrated electroencephalogram features. The resting emotional state baseline is a baseline formed by the features of the brain activity state of the target user when he or she is not affected by emotional fluctuations or external stimuli.

[0106] Specifically, based on the extracted multi-dimensional electroencephalogram features, individual emotional state baseline calibration is performed, and through calibration of the resting emotional state baseline, the emotional features of the target user can be more accurately extracted, thereby improving the accuracy and reliability of emotion prediction. The calibrated electroencephalogram features obtained by calibrating the emotional state baseline can better reflect the emotional state of the target user, thereby improving the accuracy and stability of emotion prediction. By comparing the differences between the calibrated emotional state baseline and the actual emotional state, the emotional state of the target user can be more accurately identified and predicted, thereby providing a more effective basis for further emotional intervention and regulation.

[0107] In step S204, the calibrated electroencephalogram features are input into an emotion prediction model to perform emotion prediction using the emotion prediction model, and an emotion prediction result is output. The emotion prediction model is a converged model obtained by iteratively training a predetermined model using a plurality of sets of training data using a neural network. Each set of training data includes sample electroencephalogram features and the emotional state corresponding to the sample electroencephalogram features.

[0108] Specifically, the calibrated electroencephalogram features are input into the emotion prediction model for emotion prediction, a converged model is obtained by iteratively training the training data using a neural network, and an emotion prediction model based on a neural network is constructed, thereby realizing prediction of emotion, which can more accurately predict the emotional state of an individual, help the individual better understand and manage his or her emotion in actual application, and improve the effect of emotion regulation.

[0109] In step S205, real-time emotion monitoring and feedback are performed based on the emotion prediction result.

[0110] Specifically, by monitoring the brain electrical signals of the testee, the system can capture their emotional state in real time, and according to the prediction result, the system can give feedback accordingly, such as reminding the testee to take a relaxation method when the emotional fluctuation is large, or providing encouragement and support when the emotion is low. Real-time emotional monitoring and feedback can help the testee better understand their own emotional changes and take timely measures to regulate their emotions, thereby improving emotional management ability and mental health level, helping the testee better understand and control their own emotions, and improving life quality and work efficiency.

[0111] Optionally, the electroencephalogram signal is subjected to time-frequency analysis to obtain a target electroencephalogram feature signal, including: decomposing the electroencephalogram signal by using wavelet transform to generate a first electroencephalogram feature signal, the first electroencephalogram feature signal being a global electroencephalogram signal of multiple scales and multiple frequency bands; and decomposing the first electroencephalogram feature signal into multiple groups of intrinsic mode functions by using an empirical mode decomposition method to obtain a feature signal corresponding to an emotion of different frequency components in each intrinsic mode function, thereby obtaining the target electroencephalogram feature signal.

[0112] Optionally, electroencephalogram features of different frequency bands are extracted from the target electroencephalogram feature signal to obtain multi-dimensional electroencephalogram features, including: generating a time-frequency spectrum based on the target electroencephalogram feature signal; and extracting features of the time-frequency spectrum by using a convolutional neural network to generate the multi-dimensional electroencephalogram features.

[0113] Optionally, before the multi-dimensional electroencephalogram features are calibrated based on the resting emotional state baseline to obtain calibrated electroencephalogram features, the method further includes: acquiring resting electroencephalogram signal data under the condition that the target user is in a resting state, the resting state being a brain activity state of the target user when not subjected to emotional fluctuations or external stimuli; extracting a characteristic frequency band corresponding to the resting electroencephalogram signal data by analyzing the resting electroencephalogram signal data; and determining a resting emotional state baseline of the target user based on the characteristic frequency band.

[0114] Optionally, the resting emotional state baseline is calibrated based on the multi-dimensional electroencephalogram features to obtain a calibrated emotional state baseline, including: segmenting the multi-dimensional electroencephalogram features by using a sliding window method to extract a first electroencephalogram feature from each sliding window; comparing the first electroencephalogram feature with the resting emotional state baseline; and updating the resting emotional state baseline by weighting to obtain the calibrated emotional state baseline.

[0115] Optionally, before inputting the calibrated electroencephalogram features into the emotion prediction model to utilize the emotion prediction model to perform emotion prediction and output an emotion prediction result, the method further comprises: determining a model structure of the emotion prediction model, the model structure comprising an input layer, an intermediate convolutional layer, and an output layer, the intermediate convolutional layer comprising a convolutional neural network layer, a long short-term memory network layer, a convolutional layer, and a recurrent layer, the convolutional neural network layer being configured to extract electroencephalogram features signals related to emotions, the long short-term memory network layer being configured to capture changes in the electroencephalogram features signals in a time sequence, the convolutional layer being configured to receive the electroencephalogram features signals of the convolutional neural network layer and perform feature processing on the electroencephalogram features signals to obtain emotion features, and the recurrent layer being configured to receive changes in the electroencephalogram features signals of the output of the long short-term memory network layer in a time sequence and identify changes in the emotion features in a time sequence; constructing an initial emotion prediction model according to the model structure; and iteratively training the initial emotion prediction model using a plurality of sets of training data by using a neural network to obtain the emotion prediction model.

[0116] Optionally, based on the emotion prediction result, real-time emotion monitoring and feedback are performed, comprising: generating a feedback signal based on the emotion prediction result in a closed-loop feedback system, and recording a history of changes in the emotional state of the target user according to the feedback signal.

[0117] The embodiments of the present application provide an emotion prediction system, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs comprise an emotion prediction method based on electroencephalogram signals according to any of the above.

[0118] The embodiments of the present application provide a processor for running a program, wherein the program performs the above-mentioned emotion prediction method based on electroencephalogram signals when running.

[0119] Obviously, those skilled in the art should understand that each module or each step of the present application described above can be realized by a general computing device, which can be concentrated on a single computing device or distributed on a network composed of multiple computing devices, and can be realized by program codes executable by a computing device, so that they can be stored in a storage device and executed by a computing device, and in some cases, the steps shown or described can be executed in different orders, or they can be manufactured into individual integrated circuit modules, or multiple modules or steps among them can be manufactured into a single integrated circuit module. Thus, the present application is not limited to any specific combination of hardware and software.

[0120] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0121] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0122] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0123] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0124] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0125] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0126] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0127] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, so that a process, method, article or apparatus that includes a list of elements does not only include those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0128] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Those skilled in the art can make various changes and modifications to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An emotion prediction method based on electroencephalogram signals, characterized in that, The method comprises the following steps: obtaining the brain electrical signal of the target user; performing time-frequency analysis on the brain electrical signal to obtain a target brain electrical characteristic signal, and extracting brain electrical characteristics of different frequency bands from the target brain electrical characteristic signal to obtain multi-dimensional brain electrical characteristics, wherein the target brain electrical characteristic signal is an emotion-related brain electrical signal; calibrating a resting emotional state baseline according to the multi-dimensional brain electrical characteristics to obtain a calibrated emotional state baseline, and extracting the characteristics corresponding to the calibrated emotional state baseline to obtain a calibrated brain electrical characteristic, wherein the resting emotional state baseline is a baseline formed by the characteristics of the brain activity state of the target user when the target user is not affected by emotional fluctuations or external stimuli; inputting the calibrated brain electrical characteristic into an emotion prediction model to perform emotion prediction using the emotion prediction model, and outputting an emotion prediction result, wherein the emotion prediction model is a converged model obtained by iteratively training a predetermined model using a plurality of sets of training data by a neural network, each set of training data comprising sample brain electrical characteristics and an emotional state corresponding to the sample brain electrical characteristics; based on the emotion prediction result, performing real-time emotion monitoring and feedback.

2. The method of claim 1, wherein, The method comprises the following steps: performing time-frequency analysis on the brain electrical signal to obtain a target brain electrical characteristic signal, comprising: decomposing the brain electrical signal using wavelet transform to generate a first brain electrical characteristic signal, wherein the first brain electrical characteristic signal is a global brain electrical signal with multiple scales and multiple frequency bands; 3. The method of claim 1, wherein, decomposing the first brain electrical characteristic signal into a plurality of intrinsic mode functions using an empirical mode decomposition method to obtain emotion corresponding characteristic signals of different frequency components in each intrinsic mode function, thereby obtaining the target brain electrical characteristic signal. extracting brain electrical characteristics of different frequency bands from the target brain electrical characteristic signal to obtain multi-dimensional brain electrical characteristics, comprising: generating a time-frequency spectrum from the target brain electrical characteristic signal; 4. The method of claim 1, wherein, extracting features from the time-frequency spectrum using a convolutional neural network to generate the multi-dimensional brain electrical characteristics. Before calibrating the multi-dimensional brain electrical characteristics according to the resting emotional state baseline to obtain the calibrated brain electrical characteristic, the method further comprises: obtaining resting brain electrical signal data when the target user is in a resting state, wherein the resting state is the brain activity state of the target user when the target user is not affected by emotional fluctuations or external stimuli; extracting characteristic frequency bands corresponding to the resting brain electrical signal data by analyzing the resting brain electrical signal data; 5. The method of claim 1, wherein, determining the resting emotional state baseline of the target user according to the characteristic frequency bands. Calibrating the resting emotional state baseline according to the multi-dimensional brain electrical characteristics to obtain a calibrated emotional state baseline, comprising: segmenting the multi-dimensional brain electrical characteristics using a sliding window method to extract first brain electrical characteristics from each sliding window; 6. The method of claim 1, wherein, and comparing the first brain electrical characteristics with the resting emotional state baseline, and updating the resting emotional state baseline by weighting to obtain the calibrated emotional state baseline. Before inputting the calibrated brain electrical characteristic into an emotion prediction model to perform emotion prediction using the emotion prediction model, and outputting an emotion prediction result, the method further comprises: determine a model structure of the emotion prediction model, the model structure comprising an input layer, an intermediate convolutional layer, and an output layer, the intermediate convolutional layer comprising a convolutional neural network layer, a long short-term memory network layer, a convolutional layer, and a recurrent layer, the convolutional neural network layer being configured to extract electroencephalogram feature signals related to emotions, the long short-term memory network layer being configured to capture changes in the electroencephalogram feature signals in a time sequence, the convolutional layer being configured to receive the electroencephalogram feature signals of the convolutional neural network layer and perform feature processing on the electroencephalogram feature signals to obtain emotion features, and the recurrent layer being configured to receive changes in the electroencephalogram feature signals of the output of the long short-term memory network layer in a time sequence and identify changes in the emotion features in the time sequence; construct an initial emotion prediction model according to the model structure; perform iterative training on the initial emotion prediction model using a plurality of sets of training data by means of a neural network to obtain the emotion prediction model.

7. The method of claim 1, wherein, based on the emotion prediction result, perform real-time emotion monitoring and feedback, including: in a closed-loop feedback system, generate a feedback signal based on the emotion prediction result, and record the history of changes in the emotional state of the target user according to the feedback signal.

8. An emotion prediction device based on electroencephalogram signals, characterized by, The device comprises: an acquisition unit configured to acquire electroencephalogram signals of a target user; an analysis unit configured to perform time-frequency analysis on the electroencephalogram signals to obtain target electroencephalogram feature signals, and extract electroencephalogram features of different frequency bands from the target electroencephalogram feature signals to obtain multi-dimensional electroencephalogram features, the target electroencephalogram feature signals being electroencephalogram signals related to emotions; a calibration unit configured to calibrate a resting emotional state baseline according to the multi-dimensional electroencephalogram features to obtain a calibrated emotional state baseline, and extract features corresponding to the calibrated emotional state baseline to obtain calibrated electroencephalogram features, the resting emotional state baseline being a baseline formed by features of a brain activity state of the target user when the target user is not subjected to emotional fluctuations or external stimuli; an input / output unit configured to input the calibrated electroencephalogram features into an emotion prediction model to perform emotion prediction using the emotion prediction model, and output an emotion prediction result, the emotion prediction model being a converged model obtained by performing iterative training on a predetermined model using a plurality of sets of training data by means of a neural network, each set of the training data comprising sample electroencephalogram features and an emotional state corresponding to the sample electroencephalogram features; a monitoring and feedback unit configured to perform real-time emotion monitoring and feedback based on the emotion prediction result.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium comprises a stored program, wherein the program controls the device in which the computer-readable storage medium is located to perform the method of any one of claims 1 to 7 when the program is executed.

10. An emotion prediction system characterized by, comprise: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs comprise instructions for performing the method of any one of claims 1 to 7.

Images