Emotion prediction system, method, device and medium

Abstract

The present application proposes an emotion prediction system, method, device and medium. A vector quantized variational autoencoder extracts temporally continuous differential entropy features from EEG signals, maps them to a latent space, obtains feature vectors, and performs vector quantization to obtain codebook features. The codebook features are concatenated with the feature vectors to obtain dynamic emotion deep features. A flexible actor-critic model captures the temporal dependencies of the dynamic emotion deep features to obtain prediction deep features. The prediction deep features are processed to obtain emotion prediction values. Based on the set reward function, the flexible actor-critic model can not only consider prediction accuracy, but also whether the change trend of the predicted values ​​in the previous and next time steps is consistent with the change trend of the true values. In this way, the modeling of dynamic emotions that change continuously in time is achieved, and the regression prediction of continuous emotional states is achieved, thereby promoting the application of emotion recognition in brain-computer interfaces and human-computer interactions.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an emotion prediction system, method, device and medium. Background Art

[0002] Human emotion is a continuous and dynamic process characterized by the complex interaction between the human body and the external environment. Identifying task-related emotional segments from continuous EEG signals is a major challenge.

[0003] Electroencephalography (EEG) provides a direct, objective, and scientific basis for assessing emotional states and is a valuable tool in emotion recognition research. In recent years, the potential of EEG-based emotion recognition has attracted increasing attention from researchers.

[0004] By analyzing EEG data, researchers can identify and classify different emotional states, promoting a deeper understanding of human emotions. Existing technologies have proposed a multimodal emotion dataset with continuous labels corresponding to emotions. However, these labels are only used to select training data for emotion classification. There is a lack of continuous modeling of dynamic emotions and a lack of methods for predicting and regressing continuous emotional states, which in turn limits the application of emotion recognition in brain-computer interfaces and human-computer interaction. Summary of the Invention

[0005] The present application proposes an emotion prediction system, method, device and medium, which can solve one of the problems existing in the background technology.

[0006] To achieve the above objectives, this application adopts the following technical solutions:

[0007] In a first aspect, an emotion prediction system is provided, the prediction system comprising:

[0008] A vector quantized variational autoencoder is used to obtain an EEG signal; extract temporally continuous differential entropy features of the EEG signal; map the differential entropy features to a latent space to obtain a feature vector; vector quantize the feature vector to obtain a codebook feature; and concatenate the feature vector with the codebook feature to obtain a dynamic emotion depth feature; and

[0009] A flexible actor-critic model under a Markov decision framework, with the goals of maximizing reward and maximizing entropy, includes: an actor network and a critic network. The actor network is used to capture the temporal dependencies of the dynamic emotional deep features to obtain predictive deep features; an action strategy is generated based on the current state, and the critic network is used to evaluate the value of the state-action pair. The reward function of the flexible actor-critic model includes: a first part for evaluating the accuracy of the current prediction, and a second part for evaluating the consistency between the changing trend of the predicted value and the changing trend of the true value in the previous and next time steps.

[0010] Based on the above technical solution, the emotion prediction system includes a vector quantized variational autoencoder and a flexible actor-critic model. The vector quantized variational autoencoder extracts the time-continuous differential entropy features of the EEG signal, maps it to the latent space, obtains the feature vector, and performs vector quantization to obtain the codebook feature. The codebook feature is spliced ​​with the feature vector to obtain the dynamic emotion deep feature. Afterwards, the flexible actor-critic model captures the temporal dependency of the dynamic emotion deep feature to obtain the prediction deep feature. By processing the prediction deep feature, the emotion prediction value is obtained. Based on the set reward function, the flexible actor-critic model can not only consider the prediction accuracy, but also consider whether the change trend of the predicted value before and after the time step is consistent with the change trend of the true value. In this way, the modeling of dynamic emotions that change continuously in time is realized, and the regression prediction of continuous emotional states is realized, thereby promoting the application of emotion recognition in brain-computer interface and human-computer interaction.

[0011] In a possible design manner of the first aspect, the vector quantization variational autoencoder is specifically used to:

[0012] discretizing the feature vector into discrete vectors; and

[0013] The codebook features are determined by using the discrete vectors and the codewords and indices in the codebook using a nearest neighbor search.

[0014] In a possible design manner of the first aspect, the loss function adopted by the vector quantization variational autoencoder includes: a reconstruction loss term and a quantization loss term.

[0015] In a possible design manner of the first aspect, the vector quantized variational autoencoder introduces a self-attention mechanism when concatenating the feature vector and the codebook feature.

[0016] Since some semantic information of individual differences may be lost during the vector quantization process, based on the above technical solution and using the self-attention mechanism, we can effectively highlight important features, enhance the semantic information of features, and further improve the accuracy of emotion prediction.

[0017] In a possible design of the first aspect, the actor network includes:

[0018] A long short-term memory network is used to capture the temporal dependencies of the dynamic emotion deep features to obtain prediction deep features; and

[0019] The fully connected layer is used to generate an action strategy based on the current state.

[0020] In a possible design of the first aspect, the second part Constructed by the following function:

[0021]

[0022] in, Indicates the change in the true value, Indicates the change in the predicted value, and else indicates other situations.

[0023] Based on the above technical solution, the design of the second part of the reward function enables the intelligent agent to capture the transition relationship of EEG emotional state during the process of exploration and utilization, thereby ensuring accurate prediction of emotional state.

[0024] In a second aspect, a training method for an emotion prediction system is provided. The training method is for training the emotion prediction system as described above, and the training method includes:

[0025] Obtaining a training data set, the training data set including: EEG signals and true emotion values; and

[0026] The emotion prediction system is trained using the training data set.

[0027] In a third aspect, a mood prediction method is provided, the prediction method comprising:

[0028] Obtaining current EEG signals; and

[0029] The emotion prediction system as described above is used to process the current EEG signal to obtain an emotion prediction result.

[0030] In a fourth aspect, an electronic device is provided, comprising: a processor, and a memory coupled to the processor, the memory being used to store a computer program; the processor being used to execute the computer program stored in the memory, so that the electronic device performs the training method described in the second aspect, or performs the prediction method described in the third aspect.

[0031] In a fifth aspect, a computer-readable storage medium is provided, comprising a computer program or instructions, which, when executed on a computer, enables the computer to execute the training method described in the second aspect, or the prediction method described in the third aspect.

[0032] In a sixth aspect, a computer program product is provided, comprising: a computer program or instructions, which, when executed on a computer, causes the computer to execute the training method described in the second aspect, or to execute the prediction method described in the third aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0034] Figure 1 This is a schematic diagram of the structure of the emotion prediction system provided in Example 1 of the present application;

[0035] Figure 2 This is a framework diagram of a dynamic emotion prediction model provided in Example 2 of this application;

[0036] Figure 3 This is the VQ-VAE model framework diagram provided in Example 2 of this application. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0038] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, used in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0039] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0040] Example 1

[0041] like Figure 1 As shown, this embodiment provides an emotion prediction system 100, which includes:

[0042] A vector quantized variational autoencoder 101 is configured to obtain an EEG signal; extract temporally continuous differential entropy features of the EEG signal; map the differential entropy features to a latent space to obtain a feature vector; perform vector quantization on the feature vector to obtain a codebook feature; and concatenate the feature vector with the codebook feature to obtain a dynamic emotion depth feature; and

[0043] A flexible actor-critic model 102 under a Markov decision framework, with the goal of maximizing reward and maximizing entropy, includes: an actor network 201 and a critic network 202. The actor network 201 is used to capture the temporal dependencies of the dynamic emotional deep features to obtain predictive deep features; an action strategy is generated based on the current state, and the critic network 202 is used to evaluate the value of the state-action pair. The reward function of the flexible actor-critic model includes: a first part for evaluating the accuracy of the current prediction, and a second part for evaluating the consistency between the trend of changes in the predicted value and the trend of changes in the true value in the previous and next time steps.

[0044] Specifically, the Vector Quantized Variational Autoencoder (VQ-VAE) is an improvement on the Variational Autoencoder (VAE). It maps the continuous encoder output to a discrete latent space by introducing a vector quantization (VQ) layer. This discretized representation helps capture some modalities, such as language, reasoning, and planning, while also making the generated data higher quality.

[0045] VQ-VAE consists of the following core components: a vector quantizer, an encoder, and a decoder. The encoder first encodes the input into latent vectors, then passes through the vector quantizer to map these vectors into a discrete latent space, and finally reconstructs the latent space through the decoder.

[0046] Electroencephalogram (EEG) signals are a comprehensive reflection of the electrophysiological activity of brain nerve cells in the cerebral cortex or on the surface of the scalp. In engineering applications, EEG signals can be used to implement brain-computer interfaces (BCIs). By effectively extracting and processing EEG signals, we can achieve specific goals, such as the continuous dynamic emotion prediction in this embodiment, by leveraging the differences in EEG responses to different sensory, motor, or cognitive activities.

[0047] The differential entropy feature of EEG signals is a feature used to analyze EEG signals, mainly used to describe the complexity and irregularity of EEG signals. Differential entropy is a generalization of Shannon entropy for continuous variables.

[0048] The extraction of differential entropy features can be achieved mainly through the steps of importing relevant libraries, defining functions for calculating differential entropy, defining functions for calculating power spectral density, and extracting differential entropy features.

[0049] Differential entropy features are widely used in EEG signal analysis, particularly in emotion recognition. For example, on the DEAP dataset, by segmenting EEG signals in the frequency domain and extracting their differential entropy features, combined with continuous convolutional neural networks (CNNs), support vector machines (SVMs), and multi-layer perceptrons (MLPs), high-precision emotion recognition can be achieved, with an accuracy rate typically exceeding 90%. Of course, the processing of differential entropy features in this embodiment differs from the above example.

[0050] In this embodiment, after mapping the differential entropy features to the latent space to obtain a feature vector, VQ-VAE further discretizes the feature vector into a discrete vector, and uses the nearest neighbor search based on the discrete vector and the codewords and indexes in the codebook to determine the codebook features.

[0051] Specifically, in the latent space, the continuous feature vector is discretized into a discrete vector. By minimizing the difference between the discrete vector and the codeword in the codebook, an index for determining the codebook feature is obtained, and then the codeword corresponding to the index is determined as the codebook feature.

[0052] The loss function used by VQ-VAE includes reconstruction loss and quantization loss. The reconstruction loss is used to measure the difference between the encoded input and the decoded output, while the quantization loss is used to measure the vector quantization loss.

[0053] When VQ-VAE concatenates feature vectors and codebook features, it can introduce a self-attention mechanism to achieve dynamic weighting of feature vectors and codebook features.

[0054] Markov decision making (MDP) refers to a method of making decisions using the Markov transition matrix. It is a probabilistic decision-making technique. Its basic principle is that the decision maker cannot know the probability of a particular natural state occurring in the near future. However, if the decision maker knows the probability distribution changes between natural states (i.e., the transition matrix), they can use the transition matrix to calculate the stable probability of each natural state occurring in the future environment. The optimal solution can then be selected using the expected value decision method or deterministic decision-making techniques. An MDP is constructed based on a set of interacting objects: an agent and an environment. Its elements include state, action, and reward. In an MDP simulation, the agent perceives the current system state and takes actions in the environment according to the strategy, thereby changing the state of the environment and receiving rewards. The accumulated rewards over time are called returns.

[0055] The Soft Actor-Critic (SAC) model addresses reinforcement learning in both discrete and continuous action spaces, employing an off-policy reinforcement learning algorithm. The core idea of ​​the SAC model is to add a "flexible" objective, namely the maximum entropy objective, to the standard actor-critic model, thereby maximizing not only the reward but also the entropy.

[0056] The SAC model optimizes the objective function through the policy gradient method.

[0057] The SAC model includes: Actor network and Critic network.

[0058] The actor network consists of a long short-term memory (LSTM) network and a fully connected (FC) layer. The LSTM network is used to capture the temporal dependencies of dynamic emotion deep features to generate predictive deep features. The fully connected layer is used to process the predictive deep features and generate action strategies based on the current state.

[0059] The critic network is used to evaluate the value of state-action pairs. The SAC algorithm typically includes two critic networks, each outputting a value estimate for a given state and action. These two value estimates are used to calculate the target value and update the critic network parameters by minimizing the Bellman error.

[0060] In the SAC model, you can also set a target network, such as a target actor network and a target critic network. The target network is a delayed copy used to slow down fluctuations during training.

[0061] In this embodiment, the reward function of the SAC model includes: a first part for evaluating the accuracy of the current prediction, and a second part for evaluating the consistency between the changing trend of the predicted value and the changing trend of the true value in the previous and next time steps.

[0062] Specifically, the first part can be defined by the predicted value and the true value of the current time step, and the second part can be defined by the change in the predicted value of the previous and next time steps and the change in the true value of the previous and next time steps.

[0063] In this way, the reward function guides the agent to capture changes in emotional state and accurately predict emotions during its continuous sequential decision-making.

[0064] This embodiment further provides a training method for an emotion prediction system, wherein the training method is for training the emotion prediction system as described above, and the training method includes:

[0065] Obtaining a training data set, the training data set including: EEG signals and true emotion values; and

[0066] The emotion prediction system is trained using the training data set.

[0067] This embodiment also provides a mood prediction method, which includes:

[0068] Obtaining current EEG signals; and

[0069] The emotion prediction system as described above is used to process the current EEG signal to obtain an emotion prediction result.

[0070] The above training method and prediction method are similar to those of the prediction system and will not be repeated here.

[0071] Example 2

[0072] This embodiment provides a method for predicting emotions from continuous EEG signals using deep reinforcement learning based on a vector quantized variational autoencoder (VQ-VAE).

[0073] 1. Framework Description

[0074] First, if Figure 2 As shown in Figure 2, our proposed dynamic emotion prediction model includes the following modules or sub-models:

[0075] (1) Based on VQ-VAE emotional state representation, by vector quantizing the EEG signal features, we can obtain the potential emotional state representation from a global perspective and compress the data distribution at the same time; through nearest neighbor search, we can obtain the index of the corresponding codebook vector, and then obtain the embedded representation of the corresponding codebook vector through the index to obtain the global semantic feature vector corresponding to the EEG feature.

[0076] (2) Self-Attention mechanism, which is used to concatenate the latent space features of the VQ-VAE encoder with its corresponding codebook vector, and use the self-attention mechanism for dynamic weighting to obtain deep features related to the prediction task.

[0077] (3) Based on the Soft Actor-Critic (SAC) strategy optimization, we optimized the model using the SAC reinforcement learning algorithm based on the Actor-Critic framework. We carefully designed a reward function to guide the agent to effectively predict the emotional state score based on changes in EEG emotional state. The actor is the actor mentioned above, and the critic is the critic mentioned above.

[0078] Secondly, we propose a training method for a dynamic emotion prediction model (framework):

[0079] The training method consists of two phases:

[0080] (1) Initial training stage (unsupervised learning stage), such as Figure 3 As shown, the initial training phase includes the following steps:

[0081] Step 1: Map the EEG signal into the latent space through the encoder.

[0082] Step 2: The EEG features in the latent space are embedded into codebook vectors through clustering quantization. The index of the corresponding codebook vector is obtained through nearest neighbor search, and then the embedding representation of the corresponding codebook vector is obtained through the index to obtain the global semantic feature vector corresponding to the EEG feature.

[0083] Step 3: The decoder restores the embedded representation of the codebook vector to the original data.

[0084] Step 4: Optimize the model by reconstruction loss and quantization loss.

[0085] (2) Reinforcement learning stage, such as Figure 2 As shown in Figure 2, the reinforcement learning phase includes the following steps:

[0086] Step 1: Using only the VQ-VAE encoder and codebook pre-trained with unsupervised learning, and freezing the encoder and codebook, the EEG features are input into the encoder and mapped into the latent space. The EEG features in the latent space are then indexed using a nearest neighbor search to obtain the corresponding codebook vector, which is then used to obtain the embedded representation of the corresponding codebook vector. The latent space features are concatenated with their corresponding codebook vectors and dynamically weighted using a self-attention mechanism to obtain deep features relevant to the prediction task.

[0087] Step 2: Use LSTM to obtain the state transition relationship of deep features and obtain deep time series features.

[0088] Step 3: Optimize the strategy based on SAC and continuously guide the agent to solve the optimal strategy through the reward function we designed.

[0089] In reinforcement learning, experience replay is a method to improve learning efficiency by storing the states, actions, rewards, and next states (i.e., experiences) that an agent experiences during training.

[0090] The training of the entire model is divided into two stages:

[0091] 1. In the first stage, VQ-VAE is used for unsupervised learning. The EEG signal is reconstructed using an encoder and a decoder, and the continuous latent space is discretized into a codebook, thereby extracting the deep features of the EEG in an unsupervised manner.

[0092] In the second phase, we conduct reinforcement learning training. During this training process, we only use the encoder and codebook in the VQ-VAE to extract EEG features. To alleviate quantization issues, we use the self-attention mechanism. In the second phase, the decoder is not required; the encoder is used only for feature extraction. SA Fusion is a separate module that requires training in the second phase.

[0093] In this section, we will introduce a deep reinforcement learning method for continuous EEG emotion prediction based on VQ-VAE. During the entire model training process, we use the differential entropy features of EEG signals as the input of the model. The overall framework of dynamic emotion prediction is shown in the figure below. Figure 2 As shown in Figure 2, it is mainly divided into two stages: (1) emotional state representation stage based on VQ-VAE unsupervised learning; (2) reinforcement learning stage based on emotional state representation.

[0094] During the unsupervised learning emotional state representation phase, the subject's EEG differential entropy features are vector quantized and reconstructed using VQ-VAE to obtain the subject's deep EEG emotional state. VQ-VAE maps the data into a latent space through an encoder and discretizes and compresses it into a codebook for representation. All semantic features of the EEG data are represented by the codebook vectors, which we define as the latent emotional state. This approach extracts deep features while reducing redundancy in the data distribution, resulting in a semantic representation of the latent emotional state that remains constant over time.

[0095] During the reinforcement learning phase, the model predicts emotion scores based on unsupervised latent emotional states. We pass the EEG signals through an encoder to obtain deep features, and then use nearest neighbor search to obtain corresponding codebook vectors (codewords). To enhance the semantic information of individual differences in emotional states, we concatenate the codebook vectors with their features and dynamically weight them using a self-attention mechanism to obtain deep features relevant to the prediction task. We also capture their temporal dependencies through an LSTM to obtain deep features corresponding to dynamic emotions. Finally, we optimize the model using a soft actor-critic (SAC) reinforcement learning algorithm based on an actor-critic framework. We carefully design a reward function to guide the agent to effectively predict emotional state scores based on changes in EEG emotional states. During the model optimization phase, the actor explores the action space through trial and error, locally learning the interactive relationship between the video-level EEG sample feature distribution and the global emotional state. At the end of each interaction round, the critic scores the actor's decision-making, looking for areas for strategy improvement. Through continuous iteration, the actor ultimately learns the optimal strategy for accurately predicting the subject's emotional state.

[0096] 2. Method Introduction

[0097] 2.1 Markov Decision Process

[0098] We define the process of predicting the subject's emotional state score from continuous EEG signals as a Markov decision process. That is, a Markov decision process (MDP). MDP is defined as a tuple , where S represents the state space, A represents the action space, and A=[-1,1] corresponds to the possible actions that the agent may take. Transition probability function Describes the probability of transitioning from one state to another given a specific action. γ∈[0,1] is a decay factor that determines the agent's trade-off between short-term and long-term benefits during decision-making. Reward function Assign numerical rewards based on state-action-state transitions to provide feedback for the agent's decisions.

[0099] 2.2 Representation of emotional state

[0100] In order to obtain the semantic representation of the potential emotional state of all subjects in the EEG data, the entire data distribution is vector-quantized and compressed into a codebook. Figure 1As shown in Figure 2. From a static perspective, considering the global data distribution of the signal, we believe that each cluster of EEG signal features within this data distribution represents a potential emotional state. We use VQ-VAE to vector quantize the EEG signal features, mapping them into a latent space through an encoder and embedding them into codebook vectors through cluster quantization. Each codeword is quantized to its corresponding codeword by performing a nearest neighbor search in the codebook. The decoder then attempts to restore the codeword in the latent space to the original data as closely as possible.

[0101] Specifically, we use VQ-VAE to transform the differential entropy features of EEG signals Input encoder, input EEG signal into encoder and map it into continuous latent space to get feature vector ; In the latent space, we discretize the feature vectors of a set of neighbors into a vector in the codebook , that is, the code word, and t is the time sequence.

[0102] (1)

[0103] Each time, the nearest neighbor search is performed in the codebook to obtain the index i of the codebook vector corresponding to the EEG feature, and the nearest codebook vector is obtained through the index and used as the semantic feature of the current EEG feature.

[0104] (2)

[0105] (3)

[0106] (4)

[0107] The EEG signal input is x, and the decoder restores the output as , sg represents the stop gradient, and β is the weight coefficient.

[0108] In the process of vector quantization of EEG signals, we use the reconstruction loss and quantify the loss Optimize. The reconstruction loss combines the EEG signal input x with the decoder to restore the output Calculate mean square error; quantization error in the codebook optimization process It consists of two parts: the former is the vector quantization loss, which is used to measure the difference between the encoded feature and the codebook vector, and aims to minimize the distance between each encoded feature and the codebook vector; the latter is the commitment loss, whose goal is to encourage the encoder's output to be as close as possible to its nearest codebook vector.

[0109] By vector quantizing features in the latent space, we obtain deep EEG signal features while removing redundant EEG data and compressing the global data distribution into a codebook vector for representation. We believe that the codebook vector represents a certain underlying emotional state to a certain extent. The semantic information of this emotional state is static and does not change over time, just as a person's emotional state of happiness or sadness has no direct relationship to the subject's current moment in time.

[0110] The VQ-VAE-based emotional state representation loses some semantic information related to individual differences through vector quantization. To enhance the semantic information of features and our state representation, we introduced a self-attention mechanism. This concatenates the feature vectors in the latent space with the codewords in the latent space and implements dynamic weighting to obtain dynamic deep features. The global feature template (codeword) is weighted concatenated with the current subject's personalized EEG representation (feature vector), effectively highlighting important features and assigning higher weights to features that are closer to the true emotion score, resulting in a dynamic emotion deep feature. We calculate the attention weight A of the current feature fusion using the following formula:

[0111] (5)

[0112] Where Q is the query vector, K is the key vector, V is the value vector, A is the attention weight matrix, and d is the vector dimension. The weighting through the self-attention mechanism can effectively highlight important features.

[0113] Finally, we use LSTM to capture the temporal dependencies of dynamic emotion deep features, thereby capturing the dynamic relationships between different potential emotional states. LSTM uses a forget gate, input gate, and output gate to complete the temporal feature dependencies. The calculation formula is as follows:

[0114] Forget Gate The input gate determines what information to discard from the unit. and Composition, then unit status The information update feature of the combination of the forget gate and the input gate is and Composition, decide what information to output. LSTM will be based on the time series features of the previous step and the current input After processing, the time series characteristics of the current time step are output .

[0115] (6)

[0116] in is the output of the forget gate, and are the weight and bias constant terms respectively, and σ is the activation function;

[0117] (7)

[0118] in is the output of the input gate, and are the weight and bias constant terms respectively;

[0119] (8)

[0120] in is the output of the candidate memory, and are the weight and bias constant terms respectively, and tanh is the activation function;

[0121] (9)

[0122] in is the output of the current memory, is the output of the memory in the previous step;

[0123] (10)

[0124] in is the output of the output gate, and are the weight and bias constant terms respectively;

[0125] (11)

[0126] in is the temporal feature of the current time step. By capturing the temporal dependencies of EEG signal sequence features through LSTM and obtaining the transition relationships between states, we can enhance the temporal semantic information of dynamic emotional depth features, which is beneficial to the accuracy and stability of sequential decision-making in the agent's perception state.

[0127] 2.3 Reward Function Design

[0128] To guide the agent to accurately predict emotional state scores based on changes in EEG emotional state transitions, we designed two process reward functions: the MAE reward function and the Delta reward function. For trial-level EEG signals, the reward function guides the agent to capture changes in the subject's emotional state and accurately predict their emotional state score during the continuous sequential decision-making process.

[0129] MAE reward function: This reward function calculates the square error between the agent's current predicted emotion score and the labeled continuous score to effectively ensure the accuracy of the prediction.

[0130] (12)

[0131] in is the true label at the current time step, is the agent’s predicted value at the current time step, is an exponential function .

[0132] Delta reward function: This reward function compares the consistency of the changing trend of the state prediction value with the changing trend of the actual value of the previous and next true labels at each time step.

[0133] (13)

[0134] (14)

[0135] Through the guidance of the reward function, the intelligent agent can capture the transition relationship of EEG emotional state during the process of exploration and utilization, thereby accurately predicting the emotional state score.

[0136] 2.4 Optimization Process

[0137] We use the SAC reinforcement learning algorithm to train our model. Throughout the process, the real-time continuous EEG emotional state of the subject is predicted at the trial-level, and the model learns the optimal strategy through trial and error in the continuous action space. Solving the optimal strategy in the continuous action space faces problems such as large exploration space and low sample efficiency. In order to make the optimization process more efficient, we use the SAC reinforcement learning algorithm to optimize the model. By introducing the constraint of entropy maximization, the intelligent agent can always maintain exploratory power in the face of random dynamic environments, preventing the model from falling into local optimality, thereby improving the robustness and generalization ability of the model. The SAC algorithm is an off-policy method based on actor-critic, and its cumulative expectation under the maximum entropy mechanism is expressed as:

[0138] (15)

[0139] Where: π and are the current strategy and optimal strategy of the agent respectively; For the agent in state The action to be performed is The reward obtained by the agent; is the state-action trajectory distribution formed by the strategy π; is the state-action trajectory distribution from the current state space Mapped to the distribution strategy on the action space; H is the strategy The larger the entropy value, the more exploration of the environment is done, which prevents the strategy from converging to the local optimum. α is the temperature coefficient of the action entropy, which is used to determine the weight ratio of entropy and reward.

[0140] The Q-value function improved based on entropy is defined as follows:

[0141] (16)

[0142] Where γ is the reward discount factor; V(s) is the state value function, and its calculation expression is

[0143] (17)

[0144] Combine the Bellman operator with the current Q-value function for iterative update:

[0145] (18)

[0146] in is the Bellman operator under strategy π; is the value function of the kth iteration, and eventually Q will converge to the soft Q value function under the fixed strategy π.

[0147] The agent strategy is updated by minimizing the KL divergence:

[0148] (19)

[0149] in is the KL divergence; is the set of strategy distributions; For the old strategy The Q-value function under ; is the normalization factor constant.

[0150] In the SAC algorithm, a neural network is used to fit the Q-value function and the policy function, and the Bellman residual is minimized in the form of mean square error to update the Q-value network parameters:

[0151] (20)

[0152] Where θ is the parameter of the Q value network, is the parameter of the target Q value network, are the parameters of the policy network; 、 and is the updated function; D is the experience pool.

[0153] Transform the above formula (20) to obtain the optimization goal of the actor, which is expressed as

[0154] (twenty one)

[0155] The actor outputs the policy entropy, where a is the temperature coefficient; it is adaptively updated during training by minimizing J(a):

[0156] (twenty two)

[0157] in The number of dimensions of the actions output by the policy network.

[0158] During training, the model is divided into two parts: the actor and the critic. The actor continuously interacts with the latent space of emotional states during decision-making, iteratively searching for the optimal strategy based on a reward mechanism. The critic guides this exploration process by estimating the cumulative expected reward for each state and action, thereby assisting the actor in finding the optimal strategy.

[0159] In this method, we conducted a cross-subject experiment on the SEED database and experimented with our proposed method on this database. The training method of our proposed method uses the experimental protocol of leave-one-subject cross-validation, retaining the original temporal sequence of the subject's EEG signal features, and using the entire video-level EEG signal of each subject as training data. We use supervised rewards to guide the agent to accurately predict the subject's emotional state score. We will perform sequential reasoning on the test set to obtain its emotional state score. Here we use MSE to evaluate the relationship between the predicted emotional state score and the true label, and the true label range is [-1,1]. The calculation formula of MSE is:

[0160] (twenty three)

[0161] Where n is the total number of video-level EEG signal samples, is the true value of the i-th sample, is the model prediction value of the i-th sample.

[0162] Table 1 Results of the EEG emotional state regression prediction experiment with leave-one-out cross-validation in the SEED database

[0163]

[0164] According to the results in Table 1, in the SEED database, the proposed method achieved a mean square error (MSE) of 0.037 in the leave-one-subject-out cross-validation experiment for predicting emotional states. This is a 64% improvement in regression prediction performance compared to the current SOTA method, EmoTDMF, which achieved an MSE of 0.10 under the same experimental protocol. To a certain extent, our proposed model can better capture the dynamic changes in emotional states and accurately and stably complete the prediction and regression tasks.

[0165] In addition, to further verify the effectiveness of the proposed method, we compared the results of modeling using different training methods on the SEED database. Here we use MSE, MAE and As an evaluation indicator, the specific calculation formula of MAE is as follows:

[0166] (twenty four)

[0167] Where n is the total number of video-level EEG signal samples, is the true value of the i-th sample, is the model prediction value of the i-th sample.

[0168] Table 2. Performance of regression prediction using leave-one-subject-out cross-validation on the SEED dataset under different training methods of the proposed model.

[0169]

[0170] As shown in Table 2, the experimental results of leave-one-out cross-validation of different models with different training methods are compared. Here we use MSE, MAE and Three evaluation indicators; Currently, SOTA EmoTDMF uses the supervised training method SFT, and the results of MSE and MAE have reached 0.10 and 0.24, respectively. Reached 0.59; Our method uses the reinforcement learning RL training method, and the MSE result reached 0.036 and MAE reached 0.14. It reached 0.62. Compared with the baseline of our proposed method which uses supervised learning SFT, the MSE result reached 0.25 and the MAE reached 0.39. The above experimental results show that our proposed method uses reinforcement learning to define the process of model prediction of emotional state as a Markov decision process. This enables the model to better learn the relationship between EEG feature state transitions, capture subtle dynamic changes in emotions, and optimize the model's sequential prediction capabilities. The regression prediction results show significant improvements in mean square error and absolute error compared to existing methods.

[0171] References:

[0172] [1] Zhou, X., Liang, Z., Ye, W., Xue, J., Liu, H., Zhang, M.,&Zhang,Z. (2024, April). EmoTVR: A Hybrid Model to Estimate Continuous-Time andContinuous-Level Emotion from Electroencephalography. In ICASSP 2024-2024IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP) (pp. 2021-2025). IEEE

[0173] [2] Yang, Z., Dong, W., Li, X., Huang, M., Sun, Y.,&Shi, G. (2023). Vector quantization with self-attention for quality-independent representation learning. In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition (pp. 24438-24448).

[0174] In summary, this embodiment mainly embodies the following technical points and advantages:

[0175] 1. VQ-VAE is used to vector quantize EEG signal features and encode them into codebook vectors. This allows us to learn the underlying emotional states within these EEG signal features and compress the data distribution. This method has the beneficial effect of learning the more essential semantic information of EEG signals and encoding the underlying emotional states into codebook vectors, resulting in a more robust and computationally efficient representation.

[0176] 2. The self-attention mechanism is used to realize feature splicing before and after quantization to enhance the semantic information of individual differences. The beneficial effect is that the self-attention mechanism can effectively highlight important features related to the prediction regression task.

[0177] 3. The SAC algorithm is used for reinforcement learning training. The beneficial effect is that it can solve the optimal strategy in the continuous action space more efficiently and stably. Reinforcement learning can better learn the relationship between EEG feature state transitions, capture subtle dynamic changes in emotions, and optimize the model's ability to make sequential predictions.

[0178] An embodiment of the present application also provides an electronic device, comprising: a processor, and a memory coupled to the processor, wherein the memory is used to store a computer program; and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method described in any one of the above embodiments.

[0179] The electronic device may be a computing device such as a desktop computer, a notebook computer, a PDA, a cloud server, etc. The electronic device may include, but is not limited to, a processor and a memory.

[0180] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor. The processor is the control center of the electronic device, connecting various parts of the entire device using various interfaces and lines.

[0181] The memory may be used to store the computer program, and the processor implements various functions of the electronic device by running or executing the computer program stored in the memory and calling the data stored in the memory.

[0182] The memory may primarily include a program storage area and a data storage area. The program storage area may store an operating system, at least one application required for a function, and the like; the data storage area may store data generated based on the use of the mobile phone. Furthermore, the memory may include high-speed random access memory (RAM) and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0183] The embodiment of the present application also provides a storage medium, which is a computer-readable storage medium. The computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, it can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunications signal, and a flexible component distribution medium.

[0184] An embodiment of the present application further provides a computer program product, including: a computer program or instructions, which, when executed on a computer, causes the computer to execute any of the above-mentioned possible implementation methods.

[0185] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications are also considered to be within the scope of protection of the present application.

Claims

1. An emotion prediction system, characterized in that: The prediction system includes: A vector quantized variational autoencoder is used to obtain an EEG signal; extract temporally continuous differential entropy features of the EEG signal; map the differential entropy features to a latent space to obtain a feature vector; vector quantize the feature vector to obtain a codebook feature; and concatenate the feature vector with the codebook feature to obtain a dynamic emotion depth feature; and A flexible actor-critic model within a Markov decision framework, with the goal of maximizing reward and entropy, includes an actor network and a critic network. The actor network is used to capture the temporal dependencies of the dynamic emotional deep features to obtain predictive deep features. An action strategy is generated based on the current state, and the critic network is used to evaluate the value of state-action pairs. The reward function of the flexible actor-critic model includes a first part for evaluating the accuracy of the current prediction and a second part for evaluating the consistency between the trend of changes in the predicted value and the trend of changes in the true value in the previous and next time steps. The first part is , the second part is , in, is the true label at the current time step, is the agent’s predicted value at the current time step, is an exponential function , t is the time sequence, ΔY true Indicates the change in true value, ΔY pred Indicates the change in the predicted value, and else indicates other situations.

2. The prediction system according to claim 1, wherein: The vector quantization variational autoencoder is specifically used for: discretizing the feature vector into discrete vectors; and The codebook features are determined by using the discrete vectors and the codewords and indices in the codebook using a nearest neighbor search.

3. The prediction system according to claim 1, wherein: The loss function used by the vector quantization variational autoencoder includes: a reconstruction loss term and a quantization loss term.

4. The prediction system according to claim 1, wherein: The vector quantized variational autoencoder introduces a self-attention mechanism when splicing the feature vector and the codebook feature.

5. The prediction system according to claim 1, wherein: The actor network includes: A long short-term memory network is used to capture the temporal dependencies of the dynamic emotion deep features to obtain prediction deep features; and The fully connected layer is used to generate an action strategy based on the current state.

6. A training method for an emotion prediction system, characterized in that: The training method is for training the emotion prediction system according to any one of claims 1 to 5, and the training method includes: Obtaining a training data set, the training data set including: EEG signals and true emotion values; and The emotion prediction system is trained using the training data set.

7. A method for predicting emotions, characterized in that: The prediction method comprises: Obtaining current EEG signals; and Using the emotion prediction system according to any one of claims 1 to 5, the current EEG signal is processed to obtain an emotion prediction result.

8. An electronic device, characterized in that: The electronic device includes: a processor, and a memory coupled to the processor, The memory is used to store computer programs; and The processor is configured to execute the computer program stored in the memory, so that the electronic device executes the training method according to claim 6 or the prediction method according to claim 7.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a computer program or instructions, which, when executed on a computer, causes the computer to execute the training method according to claim 6 or the prediction method according to claim 7.

Images