Continuous dynamic emotion prediction methods, training methods and equipment

Abstract

This application proposes a continuous dynamic emotion prediction method, training method, and device. It learns a spatiotemporal codebook through a vector quantization variational self-encoding / decoding module, enabling better learning of the causal and spatiotemporal semantics of dynamic emotions from EEG signal data. Employing Transformer for long-term temporal modeling allows for better learning of temporal dependencies and contextual relationships, enabling the model to learn a more generalizable and dynamic emotion latent space that better captures dynamic trends. Through a reward mechanism and reinforcement learning, the model's temporal prediction and regression capabilities are further optimized. The model not only dynamically adapts to the dynamics of human emotions but also accurately predicts the subject's current emotion, ensuring both accuracy and effectiveness.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a continuous dynamic emotion prediction method, training method and device. Background Technology

[0002] Human emotion is a continuous and dynamic process characterized by complex interactions between the human body and the external environment. Identifying task-relevant emotional segments from continuous electroencephalogram (EEG) signals is a significant challenge. Electroencephalography (EEG) provides a direct, objective, and scientific method for assessing emotional states, making it a valuable tool in emotion recognition research. In recent years, the potential of EEG-based emotion recognition has attracted increasing attention from researchers. A major shortcoming of existing research is its reliance on static labeling methods, i.e., assigning a single, fixed label to the entire EEG segment. This approach cannot capture the dynamic characteristics of human emotions in EEG-induced experiments because emotional states are inherently dynamic and constantly change with internal cognitive processes and external stimuli.

[0003] Furthermore, current research in the field of emotion recognition based on EEG signals only focuses on static emotion labels, limiting the study of continuous dynamic emotion recognition. However, accurate prediction, identification, and tracking of both static and dynamic emotion states are crucial for improving brain-computer interfaces and human-computer interaction. By analyzing EEG data, researchers can identify and classify different emotion states, promoting a deeper understanding of human emotions. This study proposes a multimodal emotion dataset with continuous labels corresponding to emotions; however, these labels are only used for selecting training data for emotion classification, lacking continuous modeling of dynamic emotions; and it also lacks methods for predicting and regressing continuous emotion states.

[0004] Therefore, developing an artificial intelligence (AI) method that can dynamically adapt to the dynamics of human emotions and accurately predict the current emotions of subjects is crucial for improving the accuracy and effectiveness of emotion recognition models. Summary of the Invention

[0005] This application proposes a continuous dynamic emotion prediction method, training method, and device, which can solve one of the problems existing in the background technology.

[0006] To achieve the above objectives, this application adopts the following technical solution: Firstly, a training method for a continuous dynamic sentiment prediction model is provided. The continuous dynamic sentiment prediction model includes: a vector quantization variational autoencoder module, a Transformer encoding module, and a regression head based on an actor-critic framework. The vector quantization variational autoencoder module includes: a vector quantization variational autoencoder, a quantization module, and a decoder. The Transformer encoding module includes: a preprocessing module and a Transformer encoder. The training method for the continuous dynamic sentiment prediction model includes: In the unsupervised learning phase, the vector quantization variational autoencoder is trained to obtain a latent space codebook containing EEG emotional states. In the unsupervised learning phase, the vector quantization variational autoencoder extracts spatiotemporal differential entropy features with causal relationships from the EEG signals. The quantization module quantizes the spatiotemporal differential entropy features into quantization vectors and finds the codebook vector with the shortest distance to the quantization vector based on the latent space codebook. The decoder decodes the codebook vector to obtain reconstructed EEG emotional data. The codebook vector corresponds to discrete labels with spatiotemporal semantics. The first loss function of the unsupervised learning phase is constructed using the EEG signals and the reconstructed EEG emotional data. During the dynamic model training phase, the parameters of the vector quantization variational auto-decoder module are frozen, and the Transformer encoder is trained to predict discrete labels. During the dynamic model training phase, the preprocessing module preprocesses the EEG signals to obtain EEG temporal data, and maps the EEG temporal data to a dynamic emotion latent space with the same dimension as the codebook vector to obtain temporal input features. Learnable positional codes are embedded in the temporal input features. The Transformer encoder captures the dependencies between the temporal input features at different time steps through a self-attention mechanism and obtains the dynamic emotion latent space representation at each time step to finally obtain a continuous dynamic emotion representation. The second loss function in the dynamic model training phase is cross-entropy loss. Furthermore, during the reinforcement learning phase, the Transformer encoder is fine-tuned using reinforcement learning rewards and a soft-actor critic algorithm.

[0007] Based on the above technical solution, the model learns the spatiotemporal codebook through a vector quantization variational self-encoding and decoding module, which better learns the causal and spatiotemporal semantics of dynamic emotions from EEG signal data. By using Transformer for long-term time-series modeling, the model learns temporal dependencies and contextual relationships better, enabling it to learn a dynamic emotional latent space with stronger generalization and better capture of dynamic trends. Through a reward mechanism and reinforcement learning, the model's temporal prediction and regression capabilities are further optimized. The model can not only dynamically adapt to the dynamics of human emotions but also accurately predict the current emotions of the subjects, ensuring both accuracy and effectiveness.

[0008] In one possible design of the first aspect, the vector quantization variational autoencoder comprises, in sequence: a spatial encoder for obtaining spatial features at each time step using a sliding window, and a temporal encoder for capturing the causal dynamic evolution of the spatial features to obtain the spatiotemporal differential entropy features. The decoder includes a causal temporal convolutional decoder and a spatial decoder connected in sequence.

[0009] In one possible design approach of the first aspect, the first loss function L is: in, To reconstruct the loss, To quantify the loss, x represents the EEG signal. To reconstruct the EEG emotional data, For quantization vectors, is the codebook vector, and sg is the stopping gradient calculation.

[0010] In one possible design of the first aspect, during the training phase of the dynamic model, the preprocessing module is further configured to: flatten the EEG signal to obtain the EEG time series data, and use a mask at the time step level with a probability of 50% to set the corresponding time step of the masked EEG time series data to 0, thereby obtaining the masked EEG time series data.

[0011] In one possible design approach of the first aspect, the continuous dynamic emotion representation for: Where A is the coefficient matrix of the continuous codebook vector combination. , For batch size, For timing length, Let E be the number of codebook vectors, and E be the latent space codebook. D is the dimension of the codebook vector. This is the residual term.

[0012] In one possible design of the first aspect, the reward function for the reinforcement learning phase includes: a regression prediction reward function, a temporal smoothing reward function, and a mean-aligned reward function, wherein the regression prediction reward function... for: The time-series smoothing reward function for: The mean-aligned reward function for: in, It is the current time step's real label. This is the predicted value at the current time step. This is the predicted value from the previous time step. This is the true mean. To predict the mean, k1, k2, and k3 are coefficients.

[0013] In one possible design approach of the first aspect, the actor-critic framework includes: an actor and a critic, in which the actor continuously interacts with the dynamic emotional latent space during the decision-making process, iteratively seeking the optimal strategy based on a reward mechanism, and the critic guides the exploration process by estimating the cumulative expected reward of each state and action, thereby assisting the actor in finding the optimal strategy.

[0014] Secondly, a continuous dynamic emotion prediction method is provided, including: Obtaining EEG signals; Furthermore, the EEG signals are processed using the emotion prediction model trained as described above to obtain prediction results.

[0015] In one possible design of the second aspect, the continuous dynamic emotion prediction method further includes: The prediction results of the regression head are smoothed, and key time points are extracted as dynamic emotion keyframes.

[0016] Thirdly, an electronic device is provided, comprising: a processor, and a memory coupled to the processor, the memory for storing a computer program; the processor for executing the computer program stored in the memory such that the electronic device performs a training method for the continuous dynamic emotion prediction model as in any possible implementation of the first aspect, or performs a continuous dynamic emotion prediction method as in any possible implementation of the second aspect. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the training method for the continuous dynamic emotion prediction model provided in Embodiment 1 of this application; Figure 2 This is a flowchart of the continuous dynamic emotion method provided in Embodiment 1 of this application; Figure 3 This is a schematic diagram of the unsupervised learning process for long-term dynamic emotion modeling based on Transformer provided in Embodiment 2 of this application; Figure 4 This is a schematic diagram of the reinforcement learning stage model training framework provided in Embodiment 2 of this application; Figure 5 This is the spatiotemporal features and codebook vector TSNE diagram of the VQ-VAE training process provided in Embodiment 2 of this application; Figure 6 This is a schematic diagram illustrating the changes of different emotions in the latent space during the training process, as provided in Embodiment 2 of this application. Figure 7 This is a heatmap of cosine similarity features of sadness at different time steps in the dynamic emotional latent space during the training process, provided in Embodiment 2 of this application. Figure 8 This is a schematic diagram of the keyframe recognition of happy emotions by the model provided in Embodiment 2 of this application for time-series prediction. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0020] It should be noted that although functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification and the above-mentioned figures are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0022] Example 1 like Figure 1 As shown, this embodiment provides a training method for a continuous dynamic sentiment prediction model. The continuous dynamic sentiment prediction model includes: a vector quantization variational autoencoder / decoder module, a Transformer encoding module, and a regression head based on an actor-critic framework. The vector quantization variational autoencoder / decoder module includes: a vector quantization variational autoencoder (VQ-VAE), a quantization module, and a decoder. The Transformer encoding module includes: a preprocessing module and a Transformer encoder. The training method for the continuous dynamic sentiment prediction model includes: Step S101, Unsupervised learning stage: The vector quantization variational autoencoder is trained to obtain a latent space codebook containing EEG emotional states. In the unsupervised learning stage, the vector quantization variational autoencoder extracts spatiotemporal differential entropy features with causal relationships from the EEG signals. The quantization module quantizes the spatiotemporal differential entropy features into quantization vectors and finds the codebook vector with the shortest distance to the quantization vector based on the latent space codebook. The decoder decodes the codebook vector to obtain reconstructed EEG emotional data. The codebook vector corresponds to discrete labels with spatiotemporal semantics. The first loss function of the unsupervised learning stage is constructed using the EEG signals and the reconstructed EEG emotional data. Step S102, Dynamic Model Training Stage: Freeze the parameters of the vector quantization variational auto-decoder module, train the Transformer encoder, and predict discrete labels. In the dynamic model training stage, the preprocessing module preprocesses the EEG signals to obtain EEG temporal data, and maps the EEG temporal data to a dynamic emotion latent space with the same dimension as the codebook vector to obtain temporal input features. Learnable positional codes are embedded in the temporal input features. The Transformer encoder captures the dependencies between the temporal input features at different time steps through a self-attention mechanism and obtains the dynamic emotion latent space representation at each time step to finally obtain a continuous dynamic emotion representation. The second loss function in the dynamic model training stage is cross-entropy loss. In step S103, the reinforcement learning stage, the Transformer encoder is fine-tuned through reinforcement learning rewards and a soft actor critic algorithm.

[0023] It should be noted that during the training phase of the dynamics model, only the vector quantization variational auto-encoder module and the Transformer encoder module are needed to form the training model. (See reference...) Figure 3 The reinforcement learning and actual prediction phases only require the Transformer encoding module and a regression head based on the actor-critic framework to form the model. (See reference...) Figure 4 .

[0024] The continuous dynamic emotion prediction model trained by the above training method can predict the dynamic emotion latent space representation of each time step in the dynamic emotion latent space, that is, the discrete label corresponding to the codebook vector. Then, the continuous dynamic emotion representation is composed of several continuous time step dynamic emotion latent space representations, which accurately reflects the continuous dynamic emotion of the subject.

[0025] Specifically, the vector quantization variational autoencoder may include, in sequence, a spatial encoder for obtaining spatial features at each time step using a sliding window, and a temporal encoder for capturing the causal dynamic evolution of the spatial features to obtain the spatiotemporal differential entropy features.

[0026] Among them, the spatial encoder can adopt a convolutional neural network. The convolutional neural network treats the EEG differential entropy features (number of channels × number of frequency bands) of each time step in the input sliding window as an image, extracts features from its spatial structure, and finally obtains the spatial features of each time step under the sliding window.

[0027] The temporal encoder can use a causal temporal convolutional network to process the above spatial features temporally. At each time step, it relies on past data to capture the relationships between temporal sequences. Through dilated convolution, it further expands the receptive field of each time step to capture longer temporal historical dependencies, thus obtaining the final spatiotemporal differential entropy features with causal relationships.

[0028] The quantization module quantizes each spatiotemporal feature within the sliding window. Specifically, the quantization module obtains the distance matrix between the spatiotemporal differential entropy feature and each codebook vector at each time step by using nearest neighbor search and Euclidean distance. Then, based on the distance matrix, the codebook vector with the shortest distance corresponding to the spatiotemporal differential entropy feature at each time step is directly used to replace the spatiotemporal differential entropy feature at the current time step.

[0029] The structure and function of the decoder are basically the same as those of the vector quantization variational autoencoder. It may include a causal temporal convolutional decoder and a spatial decoder connected in sequence. The decoder can also perform deconvolution on both the temporal and spatial levels through the quantized codebook vector to obtain the final restored EEG emotion data.

[0030] In one possible implementation, the first loss function L is: in, To reconstruct the loss, To quantify the loss, x represents the EEG signal. To reconstruct the EEG emotional data, For quantization vectors, This is the codebook vector.

[0031] In one possible implementation, during the training phase of the dynamic model, the preprocessing module is further used to: flatten the EEG signals to obtain the EEG time-series data, and apply a 50% probability time-step mask to the corresponding time steps of the masked EEG time-series data, setting them to zero to obtain the masked EEG time-series data. Specifically, the masking is performed on the features before inputting them into the Transformer network. Each time step has a 50% probability of being selected by the mask, and then all the time-step features of the mask are set to zero. This allows the model to "guess" the codebook vector corresponding to noisy data based on the context, thus learning a more robust representation.

[0032] In one possible implementation, the continuous dynamic emotion representation for: Where A is the coefficient matrix of the continuous codebook vector combination. , For batch size, For timing length, Let E be the number of codebook vectors, and E be the latent space codebook. D is the dimension of the codebook vector. This is the residual term.

[0033] In one possible implementation, the reward function for the reinforcement learning phase includes: a regression prediction reward function, a temporal smoothing reward function, and a mean-aligned reward function, wherein the regression prediction reward function... for: The time-series smoothing reward function for: The mean-aligned reward function for: in, It is the current time step's real label. This is the predicted value at the current time step. This is the predicted value from the previous time step. This is the true mean. To predict the mean, k1, k2, and k3 are coefficients.

[0034] In one possible implementation, the actor-critic framework includes an actor and a critic. The actor continuously interacts with the dynamic emotional latent space during the decision-making process, iteratively searching for the optimal strategy based on a reward mechanism. The critic guides the exploration process by estimating the cumulative expected reward for each state and action, thereby assisting the actor in finding the optimal strategy.

[0035] like Figure 2 As shown, this embodiment also provides a continuous dynamic emotion prediction method, including: Step S201: Obtain EEG signals; And, in step S202, the EEG signal is processed using the emotion prediction model trained as described above to obtain the prediction result.

[0036] In one possible implementation, the continuous dynamic emotion prediction method further includes: The prediction results of the regression head are smoothed, and key time points are extracted as dynamic emotion keyframes.

[0037] Specifically, the predicted series of values ​​are first stored in an array, with the array index representing a time point. The keyframe is determined by judging the changes in the predicted continuous values.

[0038] This embodiment learns the spatiotemporal codebook through a vector quantization variational self-encoding and decoding module, enabling better learning of the causal and spatiotemporal semantics of dynamic emotions from EEG signal data. Employing Transformer for long-term temporal modeling allows for better learning of temporal dependencies and contextual relationships, enabling the model to learn a more generalizable and dynamic emotional latent space that better captures dynamic trends. Through a reward mechanism and reinforcement learning, the model's temporal prediction and regression capabilities are further optimized. The model not only dynamically adapts to the dynamics of human emotions but also accurately predicts the subject's current emotions, ensuring both accuracy and effectiveness.

[0039] Example 2 This embodiment proposes an emotion keyframe recognition framework or network based on a neurodynamic model, a deep reinforcement learning-based continuous EEG signal emotion prediction method based on VQ-VAE, and a model training method. The entire modeling process utilizes EEG differential entropy features.

[0040] (a) The emotion keyframe recognition framework or network in this embodiment: This emotion keyframe recognition framework or network includes the following modules: (1) Unsupervised learning based on VQ-VAE causal spatiotemporal codebook: learning the representational relationships of spatial and temporal features from a large amount of EEG emotion data. For a sliding window containing T time steps (length approximately 4~8), firstly, the spatial features corresponding to each time step are learned individually through a convolutional neural network; secondly, the temporal causal relationship is learned through a temporal convolutional network to obtain the extracted spatiotemporal features. Vector quantization is performed based on the spatiotemporal features, and a typical EEG emotion state dictionary is learned from a large amount of spatiotemporal features. Finally, a set of fixed emotion vectors is used to represent the typical state of the entire dataset. Such a codebook can generate discrete labels representing semantic information for the EEG data of each time step in an unsupervised manner. Finally, the original input EEG emotion data is restored using a spatiotemporal decoder based on the quantized codebook vectors.

[0041] In other words, based on the causal spatiotemporal codebook of VQ-VAE, a causal temporal encoder and a spatial encoder were designed. By vector quantization of the spatiotemporal features of EEG signals, a latent space codebook containing typical emotional states of EEG signals was learned through reconstruction and quantization. This codebook was used as a word segmenter for EEG signals. The codebook was used to generate discrete tags with semantic information for EEG signals.

[0042] (2) Transformer-based long-term dynamic emotion modeling is used to learn the neurodynamic model of data distribution under long-term representation of EEG emotion data. This module includes the following sub-modules: three parts, projection layer, position encoding and Transformer long-term modeling. First, the first part maps the original EEG differential entropy features to the latent space of dynamic emotion through a fully connected layer. In this latent space, we assume that any EEG emotion can be represented by a fixed set of EEG emotion vectors by linear combination plus zero mean residual. The second part is position encoding. In this latent space, a set of learnable position encoding vectors represents the temporal state of different positions. The features mapped to the emotion latent space are added with position encoding to obtain the temporal feature input features at the time step level. The third part learns the dynamic emotion latent space through Transformer. Through the spatiotemporal codebook learned in the previous step, i.e. a set of fixed vectors, the EEG emotion at each time step is represented by a set of fixed emotion vectors through cross-entropy loss, thus completing the pre-training of the dynamic emotion latent space.

[0043] In other words, the dynamic model based on the Transformer encoder is used in this stage. The Transformer encoder is used to model long-term temporal dependencies, enhance the model's overall trend of temporal changes, and complete unsupervised training through random masking. It is defined as a potential neuro-emotional dynamic model.

[0044] (3) The sequence decision-making process is optimized based on the Soft Actor-Critic (SAC) reinforcement learning algorithm to further optimize the temporal prediction regression capability based on the dynamic emotion latent space. The SAC reinforcement learning algorithm enables the model to complete the accurate emotion prediction optimization process under the feedback of the three reward functions proposed in the design.

[0045] In other words, emotion decoding based on deep reinforcement learning utilizes the dynamic emotion latent space features of the pre-trained Transformer Encoder neurodynamic model, and uses reinforcement learning to fine-tune the model to achieve accurate emotion prediction.

[0046] (4) Identification of EEG keyframes based on continuous emotion prediction: Based on such continuous emotion prediction values, the local extreme points of the predicted continuous values ​​are used as keyframes, and the effectiveness and accuracy of the identified keyframes are further verified by visualization.

[0047] In other words, the system used EEG signals to decode emotions and identified keyframes based on the dynamic changes in emotions.

[0048] (II) This embodiment proposes a training method for the dynamic emotion prediction model (framework) described in (I) above: This training method consists of three phases: 1. The training phase (unsupervised learning phase) of the causal spatiotemporal codebook based on VQ-VAE, such as... Figure 3 As shown: The causal spacetime codebook training phase includes the following steps: Step S301: The EEG signal input spatial encoder independently extracts the spatial features of each time step within the sliding window.

[0049] This step specifically includes the following sub-steps: Step (1): Using a convolutional neural network, the EEG differential entropy features (number of channels × number of frequency bands) of each time step in the input sliding window are regarded as an image. By using the convolutional network, features are extracted from its spatial structure, and finally the spatial features of each time step under the sliding window are obtained.

[0050] Step 302: Spatial features are input into the temporal encoder, which captures the causal dynamic evolution relationship of spatial features through a temporal convolutional network to obtain spatiotemporal features.

[0051] This step specifically includes the following sub-steps: Step (1): The spatial features obtained in Step 1 are processed temporally using a causal temporal convolutional network. At each time step, the relationship between them is captured by relying only on their past data. The receptive field of each time step is further expanded by dilated convolution to capture longer temporal historical dependencies and obtain the final spatiotemporal features.

[0052] Step 303: Spatiotemporal features are quantized into corresponding codebook vectors using codebook vectors.

[0053] This step specifically includes the following sub-steps: Step (1): Using the spatiotemporal features obtained in step 2, quantize each spatiotemporal feature within the sliding window. That is, by using nearest neighbor search based on Euclidean distance, calculate the distance between the feature and a set of codebooks with fixed vectors, and obtain the distance matrix between the feature and each codebook vector at each time step.

[0054] Step (2): Using the distance matrix obtained in step (1), the codebook vector with the shortest distance corresponding to the spatiotemporal feature at each time step is directly used to replace the spatiotemporal feature at the current time step. Finally, the spatiotemporal features of the sliding window are converted into the corresponding codebook vector through quantization.

[0055] Step 304: The spatiotemporal decoder reconstructs the EEG signal from the codebook vector.

[0056] This step specifically includes the following sub-steps: Step (1): Using the sliding window formed by the quantized spatiotemporal features obtained in step 3, the temporal structure of the original data is restored through a causal temporal convolutional decoder, resulting in a set of spatial features with sliding window lengths.

[0057] Step (2): Using the spatial features of a set of sliding window lengths obtained in step (1), deconvolution upsampling is performed through a spatial decoder to obtain the original EEG data. Finally, the quantized codebook vector is deconvolved in both time and space to obtain the restored EEG data.

[0058] Step (3): The reconstruction loss is obtained by the mean square error between the EEG data restored by the spatiotemporal decoder in step (2) and the original EEG data; the mean square error between the spatiotemporal latent space features obtained in step 2 and the quantized spatiotemporal features obtained in step 3 is calculated by pass-through estimation; the model completes the final training process through two loss functions.

[0059] Through the training phase (unsupervised learning phase) of the causal spatiotemporal codebook based on VQ-VAE, a pre-trained VQ-VAE encoder, decoder, and codebook based on causal spatiotemporal convolution can be obtained. The spatiotemporal codebook contains a fixed set of vectors that can represent typical EEG emotional states.

[0060] 2. The training phase of the dynamic model based on the Transformer encoder, such as... Figure 3 As shown: The training phase of a dynamic model based on the Transformer Encoder includes the following steps: Step S305: Use the codebook pre-trained in Phase 1 to generate discrete labels based on features.

[0061] This step specifically includes the following sub-steps: Step (1): Use the spatiotemporal encoder learned during the training phase of the causal spatiotemporal codebook to extract spatiotemporal features for each time step in the input sliding window, and obtain the causal spatiotemporal features at each time step.

[0062] Step (2): Quantize the spatiotemporal features of causality obtained in step (1) using a codebook to obtain the codebook vector ID after quantization of each spatiotemporal feature, where the codebook vector ID represents a discrete label with semantics.

[0063] Ultimately, discrete labels with spatiotemporal semantics were obtained at each time step.

[0064] The first stage completes codebook learning, acquiring a fixed set of vectors. The second stage inputs the raw EEG signal features into the encoder to obtain spatiotemporal features. These features are then compared with vectors in the codebook, and the ID of the most similar codebook vector is used as its semantic label. This semantic label is the codebook vector's ID number; one ID number corresponds to one codebook vector. The ID number is a discrete marker with semantic information, indicating that the current EEG feature belongs to the current category, similar to clustering. The overall idea is that the codebook acts like a codebook; all EEG signals use the same encoding method and are interpretable, with corresponding vectors.

[0065] Step S306: Flatten vectorize the EEG signal sequence.

[0066] This step specifically includes the following sub-steps: Step (1): Flatten the input data of the sliding window with data dimensions of time step × number of channels × number of frequency bands, and convert it into: time step × (number of channels × number of frequency bands). The data of each time step is converted into a one-dimensional vector, where its dimension is the number of channels × number of frequency bands, to obtain flattened EEG time series data.

[0067] Step 307: Generate a random mask, add positional encoding embedding, and input it into the Transformer.

[0068] This step specifically includes the following sub-steps: Step (1): Using the flattened EEG time series data obtained in Step 2, generate a time step-level mask with a 50% probability. Set the time steps of the mask to 0 to obtain the masked EEG time series data. Before inputting features from multiple times into the Transformer network, masking is required. Each time step has a 50% probability of being selected by the mask, and then all time step features in the mask are set to 0. This pre-training stage allows the model to guess which ID number in the codebook corresponds to the noisy data based on the context, thus learning a more robust representation.

[0069] Step (2): The masked EEG temporal data obtained in step (1) is mapped to the dynamic emotion latent space through a fully connected layer, with the space dimension consistent with the dimension of the codebook vector; and a set of learnable positional codes is added in chronological order to obtain the temporal input features of the Transformer. This set of learnable positional codes is a set of vectors. In order to represent the EEG features at different time steps, the model needs to add a vector to represent the temporal position of the EEG signal.

[0070] Step (3): Input the Transformer temporal input features obtained in step (2) into the Transformer, capture the dependencies of different time steps through the self-attention mechanism, and obtain the final dynamic emotional latent space representation at each time step.

[0071] Step 308: Use cross-entropy loss to predict the discrete label corresponding to the mask.

[0072] This step specifically includes the following sub-steps: Step (1): Using the dynamic emotion latent space representation obtained in step 3, calculate the cross-entropy loss based on the discrete labels with spatiotemporal semantics generated in step 1, so that the model can learn the representation of the dynamic emotion latent space. In this latent space, any EEG emotion data can be represented by a set of fixed EEG emotion states by linear weighting plus zero mean residuals.

[0073] It needs to be explained that: First, step 1 generates discrete labels for all EEG features. Then, through masking, the model guesses which category the EEG feature corresponding to the time step set to 0 belongs to. Since all values ​​are masked as 0, the result will inevitably be problematic. Therefore, the model is able to understand the context and determine the actual meaning of a time step even when inputting noisy data, thus improving the robustness of the model's learning.

[0074] Through extensive training, a long-term dynamic emotion latent space based on Transformer was finally obtained. In this latent space, the model learned the dynamics of EEG emotion data very well.

[0075] Through the training phase of the dynamic model based on Transformer Encoder, a pre-trained dynamic emotion latent space based on Transformer can be obtained.

[0076] 3. Reinforcement learning phase, such as Figure 4 As shown: This reinforcement learning phase includes the following steps: Step 1: Based on the dynamic emotion latent space trained in the second stage, reinforcement learning is performed using a fully connected layer in the Actor-Critic framework as the regression head. Continuous emotion labels are used for fine-tuning, and SAC is used for policy optimization. The reward function we designed continuously guides the agent to solve for the optimal policy.

[0077] Through the above three training or learning stages, a well-trained emotion keyframe recognition framework or network based on a neurodynamic model is obtained.

[0078] In summary, the first stage involves learning a VQ-VAE, which includes an encoder, decoder, and codebook. The second stage involves training a Transformer based on the frozen encoder and codebook. The third stage involves training a regression head based on the Transformer and optimizing sequence prediction through reinforcement learning rewards and the SAC algorithm.

[0079] 4. Dynamic emotion keyframe recognition, such as Figure 8 As shown: The keyframe recognition stage includes the following steps: Step 1: Smooth the continuous sentiment values ​​predicted by the regression head and extract key time points as keyframes. Specifically, detect a series of predicted values ​​and store them in an array. The array index represents the time point. Determine whether a keyframe is a continuous predicted value by judging the change in the predicted value.

[0080] Step 2: Extract extreme points (first derivative is 0, second derivative distinguishes between maximum / minimum) and abrupt change points (the difference between adjacent frame labels exceeds the threshold, or the second derivative is 0).

[0081] The following section will elaborate on the details of the proposed method, introducing a deep reinforcement learning method for keyframe recognition of continuous EEG signals based on a neurodynamic model. Throughout the model training process, the differential entropy features of the EEG signal are used as the input to the model.

[0082] The overall model framework is as follows Figure 3 As shown, it is mainly divided into four stages: (1) Modeling of spatiotemporal codebook of EEG signals based on causal convolution; (2) Modeling of continuous representation of codebook of dynamic emotion latent space based on Transformer; (3) Optimization of dynamic emotion prediction reinforcement learning strategy; (4) Recognition of dynamic emotion keyframes.

[0083] In the first stage, to better characterize the causal relationship of the dynamic evolution of EEG emotions over time, for EEG signal features within a sliding time window, we sequentially extract spatial and temporal features from the EEG features. We use a convolutional neural network (CNN) to independently extract the spatial features, and then use a temporal convolutional network (TCN) based on the extracted spatial features to learn the dynamic evolution relationship of the spatial features to obtain spatiotemporal features. Based on the spatiotemporal features of each EEG signal in the sliding window, we perform vector quantization and compression into a codebook vector with a fixed representation in the codebook. We define the codebook vector as a typical and representative EEG emotional state. In this way, we not only remove redundant information and learn the semantic representation of potential emotional states that do not change over time, but also use the trained codebook to achieve discrete labeling of emotional semantics in an unsupervised manner without manual annotation of EEG signal features. In addition, the spatiotemporal causal feature representation can better characterize the trend of dynamic emotional changes in people.

[0084] In the second stage, continuous representation modeling of the codebook based on the Transformer's dynamic emotion latent space is performed. Here, the discrete emotion dictionary of EEG signals, learned in the first stage, represents typical neural activity patterns related to emotion. However, traditional codebook representation uses hard-coded discrete mapping, which has significant limitations. Matching each EEG temporal block to a single codebook vector fails to depict the continuous, gradual, and mixed states of emotion, and also struggles to capture the global correlation patterns of long-term EEG signals. This embodiment proposes a core modeling hypothesis and feature space definition: constructing a dedicated dynamic emotion latent space, which is the core feature space carrying all emotion-related features of the EEG signal. Its core hypothesis is that any segment of EEG signal features in the dynamic emotion latent space can be represented as a linearly weighted combination of pre-trained codebook vectors, superimposed with zero-mean detail fluctuation residuals. For a long-sequence EEG signal feature, a Transformer encoder captures the relationships between EEG signal features at each time step using a self-attention mechanism and maps them to the latent space of dynamic emotions. In this space, we assume that continuous emotional EEG at each time step can always be constructed as a linear combination of codebook vectors. Therefore, cross-entropy loss is used to align the EEG signal features at each time step with the discrete labels of emotional semantics generated by the global codebook. During the training phase, the EEG signal features are masked at the time step level. The dynamic latent space uses the discrete index of the global codebook dictionary corresponding to the masked EEG signal features as its semantic label. By reducing the cross-entropy loss, the model can effectively distinguish the emotional state corresponding to the EEG features at each time step.

[0085] In the third stage, based on the dynamic emotion latent space of the Transformer encoder obtained from the second stage training, the parameters of its multi-layer Transformer are frozen. We need to optimize the dynamic emotion prediction head, using only reinforcement learning to fine-tune some parameters. Here, we use the SAC reinforcement learning algorithm to optimize the strategy, enabling the model to accurately predict changes in dynamic emotions. This embodiment carefully designs a reward function to guide the agent to accurately predict emotion scores based on changes in EEG emotion states. In the model optimization stage, the actor explores the action space through trial and error, learning the interaction between the feature distribution of video-level EEG samples and the global emotion state. At the end of each interaction round, the critic scores the actor's decision for that round, looking for areas for improvement in the strategy. Through continuous iteration, the actor eventually learns the optimal strategy to accurately predict the subject's emotion state.

[0086] Markov Decision Processes (MDPs) are used in the aforementioned third stage—reinforcement learning—to define the reinforcement learning process within the theoretical framework of a Markov Decision Process, thus formalizing the theory of reinforcement learning. The process of an agent predicting a subject's emotional state score from continuous EEG signals is defined as a Markov Decision Process (MDP). A Markov Decision Process is defined as a tuple... ,in Representing the state space, Represents the action space. This corresponds to the possible actions the agent might take. Transition probability function. Describe the possibility of transitioning from one state to another given a specific action. It is the decay factor, which determines the agent's trade-off between short-term and long-term interests during the decision-making process. Reward function. Numerical rewards are assigned based on state-action-state transitions to provide feedback for the agent's decision-making.

[0087] (III) Description of the methods and structures related to this embodiment: 1. Causal-based spatiotemporal codebook modeling is used in the VQ-VAE-based causal spatiotemporal codebook training phase, for all steps of the aforementioned causal spatiotemporal codebook modeling.

[0088] To enable VQ-VAE to extract time-stable semantic representations of latent emotional states from the EEG data of all subjects, this embodiment compresses the entire EEG data distribution into a codebook using a vector-quantized method. Its unsupervised pre-training process is as follows: Figure 3As shown, from a static perspective, analyzing the global data distribution of the signal reveals that each cluster of EEG signal features aggregated within the data distribution can be considered a potential emotional state. Specifically, VQ-VAE is used to perform vector quantization processing on the features of the EEG signals: first, the encoder maps the EEG signals to the latent space, and then clusters and quantizes them into codebook vectors; for each encoded vector, nearest neighbor search is performed on the codebook to quantize it into the corresponding codeword, and then the decoder restores the codewords in the latent space to the original EEG data as much as possible to ensure the integrity of emotion-related features during the quantization process.

[0089] To achieve unsupervised extraction of typical emotional states from EEG signals and learning of discrete emotion codebook vector representations, this embodiment constructs a causal convolutional vector quantization autoencoder (Causal Conv VQ-VAE). This model follows the classic autoencoder paradigm of "encoding-quantization-decoding" and is specifically designed to address the three core characteristics of EEG signals: temporal causality, spatial topology, and non-stationarity. Its core objective is to learn a physiologically meaningful dictionary of typical EEG emotion vectors (Codebook) from massive amounts of unlabeled EEG data, providing a reliable codebook vector prior for subsequent continuous representation modeling. Furthermore, the model's structural design and training process are both based on unsupervised learning, eliminating the need for any manually labeled emotion or regression labels, effectively avoiding subjective biases introduced by manual annotation.

[0090] The causal convolutional vector quantization autoencoder designed in this embodiment consists of four core cascaded parts: a spatial encoding module, a temporal encoding module, a vector quantization module, and a spatiotemporal decoder. The model's input is temporal data of EEG differential entropy features, with a dimension of [missing information]. ,in For batch size, For time step, These represent the number of channels and frequency bands per single time step (corresponding to the five frequency bands of the EEG differential entropy feature). The model output is the reconstructed EEG signal. It must maintain the same dimension as the input signal. The specific design and function of each module are as follows: Spatial Encoding Module: The core function of this module is to remove spatial redundancy from EEG signals while preserving the core neural activity patterns of the electrode channels. The spatial characteristics of EEG signals reflect the differences in neural activity distribution across different electrode channels and are one of the core dimensions of EEG representation. This module employs a two-dimensional convolutional network structure with a sliding window size of... The spatial features of each single time step EEG are extracted and dimensionality reduced. Adaptive average pooling is used to compress the spatial features into a fixed-dimensional feature vector. Finally, the vector is mapped to the preset spatial feature dimension through a fully connected layer. Output After dimensional reshaping, it is obtained The temporal spatial feature tensor, where D s It is the spatial feature vector dimension. This process preserves the core neural activity correlation patterns between electrode channels, providing clean spatial feature input for subsequent temporal modeling.

[0091] Temporal Encoding Module: This module focuses on the dynamic temporal evolution characteristics of EEG emotional signals and strictly adheres to the causal constraints of neural activity. This paper abandons traditional recurrent neural networks (RNNs) and employs Causal TCN Blocks to construct the temporal encoder. Through stacked multi-layer dilated convolutions (with the dilation rate doubling layer by layer), the temporal receptive field is expanded without future information leakage, effectively capturing long-term temporal dependencies. This module takes spatial encoded features as input, and after nonlinear transformation, outputs a temporal encoded feature tensor. The final output dimension is... The temporal feature tensor is used to extract joint spatial-temporal features of EEG signals, where D... t It is the dimension of the codebook vector.

[0092] Vector quantization: This module is the core of the entire VQ-VAE model and the key to learning the codebook vectors of typical emotions from brainwaves, providing essential prior knowledge of the codebook vectors for this research. The core design of this module is a learnable discrete codebook. ,in The number of codebook vectors, each codebook vector Typical emotional patterns corresponding to a type of EEG signal (such as emotion-related) oscillation, BORDEN neural activity pattern).

[0093] The core logic of vector quantization is to process the continuous feature vectors output by the temporal encoder. Calculate the feature vector at each time step. With all codebook vectors in the codebook The distance is used to map the vector to the nearest codebook vector to complete the quantization. Ultimately, quantized features are generated. To address the gradient discontinuity problem in the quantization process, a straight-through estimator (STE) is employed to achieve effective gradient backpropagation. To address the common dead code problem in vector quantization (where some codebook vectors are not selected for extended periods, resulting in low codebook utilization), this embodiment incorporates a dynamic codebook reset mechanism: it tracks the usage frequency and utilization rate of each codebook vector in real time. When the utilization rate of a codebook vector falls below a preset threshold, it is reinitialized using the distribution characteristics of active codebook vectors, ensuring effective codebook utilization and representational capability. The quantized feature vector is... This tensor is the discrete codebook vector representation of the EEG signal.

[0094] The decoding module includes a timing decoder and a spatial decoder.

[0095] The decoding part has a symmetrical structure to the encoding part. First, the quantized discrete codebook vector is restored to temporal features through a causal convolutional temporal decoder. Then, the temporal features are mapped back to the spatial dimension of the original EEG signal through the transposed convolution and fully connected layers of the spatial decoder. Finally, the reconstructed (restored) EEG signal is output. The core function of the decoding module is to force the codebook vector representation learned by the encoding and quantization modules to fully preserve the core information of the original EEG signal by reconstructing constraints, thus ensuring the effectiveness of the codebook vector.

[0096] Specifically, VQ-VAE is used to analyze the differential entropy characteristics of EEG signals. The input encoder maps the features of the input EEG signal into the latent space to obtain a feature vector. In the latent space, we discretize a set of continuous eigenvectors into a vector in the codebook. That is, code words.

[0097] (1) Each time, the index of the nearest codebook vector is obtained through nearest neighbor search in the Codebook. It obtains the nearest codebook vector through indexing and uses it as the semantic feature of the current EEG feature.

[0098] (2) (3) (4) In the process of vector quantization of EEG signals, we use reconstruction loss. and quantification loss Optimize. Reconstructing the loss involves inputting EEG signals. Restored output with decoder Calculate the mean square error; quantization error in the codebook optimization process. It consists of two parts: the first is the vector quantization loss, which measures the difference between the encoded feature and the codebook vector, aiming to minimize the distance between each encoded feature and the codebook vector; the second is the commitment loss, which aims to encourage the encoder's output to be as close as possible to its nearest codebook vector. 'a' represents the balancing weight.

[0099] 2. Modeling continuous representations of codebook linear combination based on the dynamic emotion latent space of Transformer, used to establish the dynamic emotion latent space, used in all stages of the training phase of the dynamic model based on Transformer Encoder, and used to complete unsupervised random mask prediction based on Transformer Encoder to complete the modeling of continuous dynamic emotions.

[0100] The core characteristics of EEG emotion are manifested in a global pattern of continuous dynamic evolution and multi-brain region collaboration. By using a causal convolutional vector quantization autoencoder, a discrete emotion lexicon of EEG signals was successfully learned. The codebook vectors in this dictionary correspond to typical neural activity patterns related to EEG emotions. However, the traditional VQ-VAE representation method is a hard-coded discrete mapping, which has significant limitations: it matches each EEG temporal block to only a single codebook vector, which cannot characterize the continuous gradual changes and mixed states of emotions, nor can it capture the global correlation patterns of long-term EEG signals, resulting in insufficient flexibility and dynamic adaptability of the representation.

[0101] To address the aforementioned issues, this embodiment proposes a core modeling hypothesis and feature space definition: Constructing a dedicated dynamic emotion latent space, which is the core feature space carrying all emotion-related features of EEG signals. Its core hypothesis is that any segment of EEG signal features in the dynamic emotion latent space can be represented as a linearly weighted combination of codebook vectors obtained through pre-training, superimposed with zero-mean detail fluctuation residuals.

[0102] The mathematical expression for this core assumption is: (5) in, These are codebook vectors in the Codebook (corresponding to typical EEG emotion-related neural patterns). These are sparse combination coefficients (representing the contribution weight of the codebook vector to the signal), representing the contribution weight of the corresponding codebook vector in the current EEG signal. Let n be the residual of the signal’s detail fluctuations near the typical pattern, and n be the total number of vectors in the codebook.

[0103] The core advantage of this representation method lies in the fact that it retains the interpretability and typical pattern capture capability of the VQ-VAE codebook vector, while also achieving a higher level of performance through continuous combination coefficients. It achieves flexible soft representation, which can accurately depict the continuous dynamic changes of brainwave emotions, and overcomes the limitations of hard-coded representation.

[0104] To realize this hypothesis, this embodiment designs a representation framework: First, a causal convolutional vector quantization autoencoder learns typical neural patterns of EEG, constructing a codebook vector dictionary as a representation prior; second, an independently learnable fully connected layer is introduced to map the high-dimensional original EEG temporal features (dimension is temporal length × 310, 310 = 62 electrodes × 5 frequency bands) to a pre-defined dynamic emotion latent space without information loss through linear transformation and nonlinear activation—this fully connected layer is independent of the VQ-VAE encoder, completing both the dimensional adaptation from the original signal to the latent space and avoiding the feature bias introduced by the pre-trained encoder, thus providing... Subsequent temporal modeling provides a clean set of latent space features as input. Finally, a Transformer encoder is introduced as the core network within the dynamic emotion latent space. By leveraging a multi-head self-attention mechanism, it captures the long-term temporal dependencies and global associations of EEG features within the latent space, accurately depicting the dynamic ratio of codebook vector combination coefficients at different time steps (such as the temporal consistency of theta wave codebook vector coefficients during periods of sustained anxiety). Simultaneously, during the training phase, random masking is applied to local temporal feature blocks in the latent space to simulate noise interference and local feature loss scenarios in EEG signals, further enhancing the anti-interference capability and generalization performance of the latent space representation.

[0105] The core output of the Transformer encoder is a combination coefficient matrix of continuous codebook vectors. ( For batch size, For timing length, (where the number of elements is the codebook vector) Represents the first The sample batch in the first The first time step The contribution weights of each codebook vector are used to dynamically reflect the continuous evolution of EEG emotions in the DynamicEmotion Latent space. Based on this coefficient matrix, a codebook vector dictionary pre-trained by a causal convolutional vector quantization autoencoder is constructed. By superimposing zero-mean residuals through linear weighted combination, the final continuous EEG emotional representation within the dynamic emotional latent space can be generated. (6) in, (D is the dimension of the codebook vector) is the codebook vector dictionary matrix. This representation preserves the typical emotional neural patterns corresponding to the codebook vectors to ensure physiological interpretability, and also achieves the characterization of continuous emotional states through the dynamic allocation of continuous coefficients, while the residual terms... It adapts to the non-stationarity and individual differences of EEG signals, providing a robust and flexible feature base for downstream emotion analysis tasks.

[0106] To achieve this representation, this embodiment designs a framework of "VQ-VAE codebook vector dictionary pre-training + fully connected layer latent space mapping + Transformer encoder modeling": The first stage learns reliable emotions through VQ-VAE, providing a priori foundation for the representation; the second stage maps the raw EEG to a dynamic emotion latent space through independent fully connected layers; the third stage introduces a Transformer encoder to extract long-term temporal features, combines a random masking strategy to enhance the model's robustness to local signal loss, and aligns the coefficient distribution with the codebook vector distribution through cross-entropy loss. During the training phase, a random sequence zero-setting masking strategy is used: randomly selecting some continuous temporal segments for zero-setting to simulate local EEG signal loss and noise interference, forcing the model to learn representations based on global contextual information. The formulaic expression of this alignment process is as follows: (7) By minimizing the cross-entropy loss L CE This forces the model to predict a strict correspondence between the discrete-time semantics of the mask features and the discrete-time codebook through contextual information. Among these, Indicates the time step as Belongs to the The predicted probability of the class codebook vector. Indicates the time step as The true label of the EEG feature corresponds to the discrete label generated by the codebook, which is a One-Hot label.

[0107] The Transformer encoder captures long-term temporal dependencies and global associations of EEG features in the latent space through a multi-head self-attention mechanism, accurately characterizing the dynamic ratio of codebook vector combination coefficients at different time steps. Its core output is a continuous codebook vector combination coefficient matrix. .

[0108] (8) in Indicates the first The sample, the first At this time step, the first The predicted contribution probability of the codebook vector.

[0109] 3. Optimization process: This process is used in the reinforcement learning stage of the aforementioned steps. The process consists of two parts: reward function design and SAC reinforcement learning algorithm; based on the dynamic emotion latent space pre-trained in the previous step, reinforcement learning is introduced to further optimize the model by using real human emotions as feedback, so that the model can accurately predict human emotions.

[0110] Regression prediction reward function: This reward function effectively ensures the accuracy of prediction by calculating the squared error between the continuous predicted value of the agent's continuous emotion at the current time step and the continuous emotion label.

[0111] (9) in It is the current time step's real label. It is the agent's predicted value at the current time step, and k1 is the coefficient of the exponential function, used to adjust the sensitivity of the reward value.

[0112] Temporal smoothing reward function: Based on the short-term continuity of human emotions, this is the behavioral constraint for the agent's emotion prediction, which punishes abrupt changes in adjacent prediction values ​​to ensure the continuity of emotion prediction.

[0113] (10) in This is the prediction result at the current time step. It is the prediction result of the agent at the previous time step, and k2 is the coefficient of the exponential function, which is used to adjust the sensitivity of the reward value.

[0114] Mean-aligned reward function: Based on the long-term similarity of human emotions, we believe that under the same emotional stimulus, the overall mean of human emotions will not change significantly; this reward function penalizes the global mean deviation between the emotion prediction result and the true prediction result, reducing the overall deviation of the prediction result.

[0115] (11) (12) (13) in This is the prediction result at the current time step. It is the prediction result of the agent in the previous time step, and k3 is the coefficient of the exponential function, which is used to adjust the sensitivity of the reward value.

[0116] (14) Through reward function Guided by this, intelligent agents can capture the transition relationships of brainwave emotional states during the exploration and utilization process, thereby accurately predicting dynamic and continuous emotions.

[0117] The SAC reinforcement learning algorithm was used to train the model. Throughout the process, the model predicted the real-time continuous emotional states of the subjects at the trial level, learning the optimal policy through trial and error in a continuous action space. Solving for the optimal policy in a continuous action space faces challenges such as a large exploration space and low sample efficiency. To make the optimization process more efficient, the SAC reinforcement learning algorithm was used to optimize the model. By introducing the constraint of maximizing entropy, the agent maintains its exploratory nature in the face of random dynamic environments, preventing the model from getting trapped in local optima, thereby improving the model's robustness and generalization ability. The SAC algorithm is an Actor-Critic-based off-policy method, and its cumulative expectation under the maximum entropy mechanism is expressed as: (15) In the formula: and These are the agent's current policy and optimal policy, respectively. For the agent in state The action to be performed The reward obtained by the intelligent agent; For strategy The resulting state-action trajectory distribution; from the current state space Distribution strategy mapped onto the action space; In strategy The entropy of the actions taken is such that a larger entropy value means more exploration of the environment, thus avoiding the strategy from converging to a local optimum. The temperature coefficient of the action entropy is used to determine the weight ratio of entropy to reward.

[0118] Based on entropy improvement The value function is defined as follows: (16) in As a reward discount factor; The state value function is expressed as follows: (17) Transforming the above equation, we obtain the optimization objective of the Actor, expressed as: (18) The Actor will output the policy entropy, where For temperature coefficient, Representation Strategy From parameters Parameterized policy network, referring to Actor network; by minimizing The method involves adaptive updates during training: (19) in The number of dimensions of the actions output by the policy network.

[0119] During training, the model is divided into two parts: Actor and Critic. The Actor continuously interacts with the latent space of emotional states during the decision-making process, iteratively searching for the optimal strategy based on the reward mechanism. The Critic guides this exploration process by predicting the cumulative expected reward for each state and action, thereby assisting the Actor in finding the optimal strategy.

[0120] (iv) Advantages: In this embodiment, a cross-subject experiment was conducted on the SEED database, and the method proposed in this embodiment was tested on this database. The training method of this embodiment uses a leave-one-out cross-validation protocol to preserve the original temporal sequence of the subjects' EEG signal features. The video-level complete EEG signal of each subject is used as training data, and supervised reward guidance is used to guide the agent to accurately predict the subject's emotional state score. Sequential reasoning is performed on the test set to obtain the emotional state score. MSE (Mean Sequential Emotional State) is used here to evaluate the relationship between the predicted emotional state score and the true label, and the range of the true label is determined. The formula for calculating MSE is as follows: (20) in, The total number of video-level EEG signal samples. It is the first The true value of each sample It is the first The model prediction value for each sample.

[0121] Table 1. Results of the SEED database participant leave-one-out cross-validation EEG regression prediction experiment for emotional state. Table 2. Results of the SEED-IV database participant leave-one-out cross-validation EEG emotional state regression prediction experiment. Table 1 shows the performance of different models on the SEED dataset for continuous emotion label regression based on leave-one-out cross-validation (LOSO). The experimental results demonstrate that the proposed method achieves the best performance across all evaluation metrics. When performing continuous emotion label regression on the SEED dataset, the proposed model achieves a mean squared error (MSE) of 0.0876, a mean absolute error (MAE) of 0.2156, and a Pearson correlation coefficient of 0.6858, exhibiting good regression ability. Compared to traditional machine learning methods, the best-performing traditional method, MLP, achieves MSE, MAE, and Correlation values ​​of 0.1273, 0.2742, and 0.6113, respectively. In contrast, our time-saving and labor-saving method reduces MSE by 0.0397, MAE by 0.0586, and improves the correlation coefficient by 0.0745, indicating that the proposed model can more effectively learn emotion-related features in EEG signals. Compared to deep learning methods, EEGNet, a baseline method, performed better with MSE of 0.1134, MAE of 0.2499, and Correlation of 0.6628. In contrast, the method in this embodiment reduced MSE by 0.0258, MAE by 0.0343, and improved the correlation coefficient by approximately 0.0230, further validating the effectiveness and advantages of the proposed model in the cross-subject sentiment regression task.

[0122] Table 2 shows the performance of different models in predicting EEG emotion states based on leave-one-out cross-validation (LOSO) on the SEED-IV dataset. The experimental results demonstrate that the method proposed in this embodiment achieves the best performance across all evaluation metrics. In the continuous label regression task on the SEED-IV dataset, the proposed model achieves a mean squared error (MSE) of 0.0932, a mean absolute error (MAE) of 0.2133, and a Pearson correlation coefficient of 0.4717, indicating that the method can effectively characterize the relationship between EEG signals and emotion states. Compared with traditional machine learning methods, the best-performing traditional method, Lasso, achieves MSE, MAE, and Correlation values ​​of 0.1081, 0.2580, and 0.4697, respectively. In contrast, the method in this embodiment reduces MSE by 0.0149, MAE by 0.0447, and the correlation coefficient by 0.0020, indicating that the proposed model can more effectively extract emotion-related features from EEG signals and improve regression prediction performance. Compared to deep learning methods, DCORAL, a baseline method, performed better with MSE of 0.1046, MAE of 0.2516, and Correlation of 0.4410. In contrast, the method in this embodiment reduced MSE by 0.0114, MAE by 0.0383, and improved the correlation coefficient by 0.0307, ​​further validating the effectiveness and advantages of the proposed method in the cross-subject EEG emotion regression task.

[0123] like Figure 5 As shown, the t-SNE visualization distribution changes of the proposed causal spatiotemporal EEG vector quantization autoencoder during training are illustrated. By observing the data distribution characteristics at different training stages, the model's feature representation ability during vector quantization learning can be intuitively analyzed. In the initial training stage, such as... Figure 5 As shown, the latent space vectors (light blue dots) and codebook vectors (dark blue dots) are relatively scattered, exhibiting strong randomness and lacking a clear clustering structure. Furthermore, the large distance between the latent space vectors and codebook vectors indicates that the model has not yet learned stable and effective feature representations, and the quantization process is still in its initial stage. As training progresses, in the intermediate stage... Figure 5As shown, the distance between the latent space vectors and the codebook vectors gradually decreases, and local clustering begins to appear in some regions, gradually forming several relatively dense cluster structures. Simultaneously, the latent space vectors gradually converge towards their corresponding codebook vectors, indicating that the model is gradually optimizing the vector quantization process, and the feature representation capability is improved to some extent, but the overall clustering structure is still not obvious. During the training completion phase, as... Figure 5 As shown, the data distribution exhibits a clearer and more stable clustering structure. The matching relationship between latent space vectors and codebook vectors is closer, the boundaries between different clusters gradually become clearer, and the data points within each cluster are more concentrated. This phenomenon indicates that the model can effectively learn the latent feature structure of EEG signals and effectively quantify the latent space representation through codebook vectors. The resulting multiple cluster structures can be regarded as typical feature representations of different emotional states in EEG signals, thus verifying the effectiveness of the proposed model in learning the distribution of EEG emotional features.

[0124] like Figure 6 As shown, this illustrates the temporal trajectory changes of grief EEG signals in the latent space of the Transformer. In the initial training phase, the trajectories of different samples are scattered and structurally chaotic, with no stable correspondence between temporal order and latent space location. As training progresses, in the middle training phase, sample trajectories gradually cluster towards similar regions, and some trajectories begin to exhibit consistent evolutionary trends, indicating that the model is gradually capturing the temporal patterns in grief EEG signals. In the final training phase, the trajectories of different samples exhibit a more obvious clustering structure and evolve along similar directions. Starting from the blue starting point at the top, they gradually descend along the vertical axis and eventually converge to the red ending point at the bottom, forming a relatively stable one-to-one correspondence between temporal order and latent space location. This demonstrates that the model has effectively learned the temporal dependency features in grief EEG signals and formed a stable dynamic representation structure in the latent space. Overall, as training progresses, the temporal trajectories of different emotional EEG signals in the latent space gradually transform from a disordered distribution to a trajectory pattern with a clear structure. This indicates that the Transformer model can effectively capture the dynamic temporal features of EEG signals and gradually establish a latent representation space for emotional states. This phenomenon further validates the effectiveness of the proposed method in the task of EEG-based dynamic emotion modeling.

[0125] like Figure 7As shown, this figure illustrates the changes in the correlation structure of sadness EEG data within the dynamic emotion latent space during model training. The horizontal and vertical axes represent time steps, and colors indicate the correlation strength between different time steps, with blue representing higher correlation and red representing lower correlation. From left to right, these represent the initial, middle, and final training stages, reflecting the model's learning process of temporal structure. In the initial training stage (left figure), the correlation matrix is ​​relatively random. Although there is some high correlation near the diagonal, the overall structure is chaotic, lacking stable pattern relationships between different time steps, indicating that the model has not yet fully learned the temporal dependency structure in the EEG signals. In the middle training stage (middle figure), the correlation matrix begins to show some structured features. The high-correlation region near the diagonal gradually expands, and clear correlation patterns form between some time periods, indicating that the model is beginning to capture local temporal dependencies in the EEG signals. In the final training stage (right figure), the correlation matrix exhibits a clearer and more stable structure, with a distinct band-like high-correlation region forming near the diagonal. This structure indicates strong correlation between adjacent time steps, while the correlation gradually weakens as the time interval increases, reflecting typical temporal dependency characteristics. Overall, as training progresses, the correlation structure in the latent space gradually evolves from a disordered distribution to a structural pattern with obvious time dependence, indicating that the model can effectively capture the dynamic temporal features in EEG signals and form a stable emotional latent space representation.

[0126] Figure 8This paper showcases the trained model's inference and prediction results for sadness and negative emotions on the SEED dataset test set. The core content includes two main parts: continuous emotion temporal prediction performance and EEG activation visualization of keyframes / non-keyframes of emotions. The specific analysis is as follows: In the figure, the horizontal axis represents the time step, and the vertical axis represents the emotion label value. The blue curve represents the true label of emotion, and the red curve represents the model's prediction value. The results show that the true label of sadness and negative emotions generally shows a continuous downward trend, accompanied by local fluctuations. The model's prediction curve accurately matches the overall trend of the true label change and effectively captures key time points where significant changes in emotion occur, verifying that the model has good fitting ability and prediction accuracy for the dynamic changes of continuous emotions. Based on the local extreme points of emotion values, keyframes of emotion changes were identified and extracted. Non-keyframes were selected as controls, and brain topography visualization of the EEG signals at the corresponding time points was performed. The comparison results show that in the brain topography corresponding to the emotion keyframes, the brain regions related to the processing of sadness and negative emotions show a significantly high activation state; while the corresponding brain regions of non-keyframes show no obvious activation response, and the activation level is much lower than that of the keyframes. This result not only confirms that the emotion keyframes extracted by the model can accurately correspond to the core neural response moments of the brain's emotion processing, but also provides an interpretable neurophysiological basis for the model's emotion prediction results.

[0127] In summary, this embodiment proposes a VQ-VAE based on causal spatiotemporal convolution to learn a spatiotemporal codebook. This codebook uses a fixed set of feature vectors to represent typical emotional states. In order to better learn the causality of dynamic emotions and the dynamic semantics of features from EEG signal data, a causal temporal convolutional network is constructed to obtain the temporal causal features, i.e., dynamic semantics, within a sliding window. The causal dynamic emotional features at each time step are quantized to learn a typical emotional semantic codebook. This codebook captures the typical global causal spatiotemporal patterns and can generate discrete labels for EEG features in an unsupervised manner, laying a solid foundation for the unsupervised masking task in subsequent dynamic long-term emotional modeling.

[0128] Traditional methods using neural networks such as LSTM and GRU cannot effectively capture long-term temporal dependencies. This embodiment uses Transformer for long-term temporal modeling to better model dynamic and continuous EEG emotions, and performs unsupervised masking to model a continuous dynamic emotion latent space. By using time-step-level masking and allowing the model to continuously reason within the context, the model learns temporal dependencies and contextual relationships better. To enable the model to learn a more generalizable and dynamic emotion latent space that better captures dynamic trends, an unsupervised masking task is performed based on the learned causal spatiotemporal codebook.

[0129] This embodiment designs three optimized reward functions to enable the model to perform well in optimizing the model; through the reward mechanism and reinforcement learning, the model's ability to predict and regress in time series is further optimized.

[0130] This embodiment also proposes an EEG keyframe recognition method based on dynamic emotion prediction, and completes continuous emotion regression prediction through the proposed model. The recognition of emotion keyframes is completed through the local extreme points of the predicted values, and the effectiveness of keyframe extraction is demonstrated through further EEG visualization.

[0131] This application also provides an electronic device, including: a processor, and a memory coupled to the processor, the memory being used to store a computer program; the processor being used to execute the computer program stored in the memory, so that the electronic device performs the method as described in any of the above embodiments.

[0132] Electronic devices can be computing devices such as desktop computers, laptops, handheld computers, and cloud servers. These electronic devices may include, but are not limited to, processors and memory.

[0133] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the electronic device, connecting various parts of the device via various interfaces and lines.

[0134] The memory can be used to store the computer program, and the processor implements various functions of the electronic device by running or executing the computer program stored in the memory and calling the data stored in the memory.

[0135] The memory may primarily include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function, etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0136] This application also provides a storage medium, which is a computer-readable storage medium. The computer program is stored in the computer-readable storage medium, and when executed by a processor, the computer program can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0137] This application also provides a computer program product, including: a computer program or instructions that, when the computer program or instructions are run on a computer, cause the computer to perform any of the above possible implementation methods.

[0138] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications are also considered to be within the scope of protection of this application.

Claims

1. A training method for a continuous dynamic emotion prediction model, characterized in that, The continuous dynamic sentiment prediction model includes: a vector quantization variational autoencoder module, a Transformer encoding module, and a regression head based on an actor-critic framework. The vector quantization variational autoencoder module includes: a vector quantization variational autoencoder, a quantization module, and a decoder. The Transformer encoding module includes: a preprocessing module and a Transformer encoder. The training method for the continuous dynamic sentiment prediction model includes: In the unsupervised learning phase, the vector quantization variational autoencoder is trained to obtain a latent space codebook containing EEG emotional states. In the unsupervised learning phase, the vector quantization variational autoencoder extracts spatiotemporal differential entropy features with causal relationships from the EEG signals. The quantization module quantizes the spatiotemporal differential entropy features into quantization vectors and finds the codebook vector with the shortest distance to the quantization vector based on the latent space codebook. The decoder decodes the codebook vector to obtain reconstructed EEG emotional data. The codebook vector corresponds to discrete labels with spatiotemporal semantics. The first loss function of the unsupervised learning phase is constructed using the EEG signals and the reconstructed EEG emotional data. During the dynamic model training phase, the parameters of the vector quantization variational auto-decoder module are frozen, and the Transformer encoder is trained to predict discrete labels. During the dynamic model training phase, the preprocessing module preprocesses the EEG signals to obtain EEG temporal data, and maps the EEG temporal data to a dynamic emotion latent space with the same dimension as the codebook vector to obtain temporal input features. Learnable positional codes are embedded in the temporal input features. The Transformer encoder captures the dependencies between the temporal input features at different time steps through a self-attention mechanism and obtains the dynamic emotion latent space representation at each time step to finally obtain a continuous dynamic emotion representation. The second loss function in the dynamic model training phase is cross-entropy loss. Furthermore, during the reinforcement learning phase, the Transformer encoder is fine-tuned using reinforcement learning rewards and a soft-actor critic algorithm.

2. The training method for the continuous dynamic emotion prediction model as described in claim 1, characterized in that, The vector quantization variational autoencoder comprises, in sequence: a spatial encoder for obtaining spatial features at each time step using a sliding window, and a temporal encoder for capturing the causal dynamic evolution of the spatial features to obtain the spatiotemporal differential entropy features. The decoder includes a causal temporal convolutional decoder and a spatial decoder connected in sequence.

3. The training method for the continuous dynamic emotion prediction model as described in claim 1, characterized in that, The first loss function L is: in, To reconstruct the loss, To quantify the loss, x represents the EEG signal. To reconstruct the EEG emotional data, For quantization vectors, is the codebook vector, and sg is the stopping gradient calculation.

4. The training method for the continuous dynamic emotion prediction model as described in claim 1, characterized in that, During the training phase of the dynamic model, the preprocessing module is further used to: flatten the EEG signal to obtain the EEG time series data, and use a mask at the time step level with a probability of 50% to set the corresponding time step of the masked EEG time series data to 0, thereby obtaining the masked EEG time series data.

5. The training method for the continuous dynamic emotion prediction model as described in claim 1, characterized in that, The continuous dynamic emotion representation for: Where A is the coefficient matrix of the continuous codebook vector combination. , For batch size, For timing length, Let E be the number of codebook vectors, and E be the latent space codebook. D is the dimension of the codebook vector. This is the residual term.

6. The training method for the continuous dynamic emotion prediction model as described in claim 1, characterized in that, The reward functions for the reinforcement learning phase include: a regression prediction reward function, a temporal smoothing reward function, and a mean-aligned reward function, wherein the regression prediction reward function... for: The time-series smoothing reward function for: The mean-aligned reward function for: in, It is the current time step's real label. This is the predicted value at the current time step. This is the predicted value from the previous time step. This is the true mean. To predict the mean, k1, k2, and k3 are coefficients.

7. The training method for the continuous dynamic emotion prediction model as described in claim 6, characterized in that, The actor-critic framework includes actors and critics. During the decision-making process, actors continuously interact with the dynamic emotional latent space and iteratively search for the optimal strategy based on a reward mechanism. The critic guides the exploration process by estimating the cumulative expected return of each state and action, thereby assisting the actor in finding the optimal strategy.

8. A continuous dynamic emotion prediction method, characterized in that, include: Obtaining EEG signals; Furthermore, the EEG signal is processed using a trained emotion prediction model as described in any one of claims 1-7 to obtain a prediction result.

9. The continuous dynamic emotion prediction method as described in claim 8, characterized in that, The continuous dynamic emotion prediction method also includes: The prediction results of the regression head are smoothed, and key time points are extracted as dynamic emotion keyframes.

10. An electronic device, characterized in that, The electronic device includes: a processor, and a memory coupled to the processor. The memory is used to store computer programs; The processor is configured to execute the computer program stored in the memory, such that the electronic device performs the training method of the continuous dynamic emotion prediction model as described in any one of claims 1-7, or performs the continuous dynamic emotion prediction method as described in claim 8 or 9.

Images