A closed-loop emotion recognition method and system

By combining individual cue vectors and heart rate variability features into the emotion decoding model, the problem of sensitivity to blinking, eye movement and electromyography interference in existing emotion recognition methods is solved, achieving stable emotion recognition and feedback, and improving the stability and accuracy of cross-subject applications.

CN122701335BActive Publication Date: 2026-11-10DONGGUAN ZKTECO ELECTRONICS TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202611207380.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-08-11
Publication Date
2026-11-10
Estimated Expiration
2046-08-11

Smart Images

  • Figure CN122701335B_ABST
    Figure CN122701335B_ABST
Patent Text Reader

Abstract

The application discloses a closed-loop emotion recognition method and system, comprising: acquiring a forehead few-channel electroencephalogram signal of a user, inputting the signal into a pre-trained emotion decoding model, and outputting an emotion continuous vector representing a current emotional state of the user in a PAD continuous space; wherein the emotion decoding model is pre-trained using sample electroencephalogram signals of multiple subjects and a PAD scale, and is determined to be successfully trained when a loss function calculated based on an individual prompt vector and a penalty mechanism meets a preset value; and it is judged whether the emotion continuous vector meets a closed-loop trigger condition, and if yes, a closed-loop feedback instruction is generated and executed for feedback adjustment. Thus, based on the closed-loop triggering and feedback adjustment in the PAD continuous space, discrete classification is converted into dynamic intervention driven by continuous coordinates, the limitation of open-loop monitoring is broken through, and stable and reliable closed-loop emotion regulation is effectively realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method and system for identifying closed-loop emotions. Background Technology

[0002] With the deep integration of neural engineering and artificial intelligence, non-invasive brain-computer interfaces have expanded from decoding motor intentions to quantitative analysis of emotions. Emotions are essentially the synergistic result of central nervous system arousal, cognitive evaluation, and autonomic nervous system responses. Therefore, using continuous coordinates rather than discrete categories to describe emotions is more conducive to building interpretable and feedback-enabled human-computer interaction systems. In recent years, a continuous emotional space based on pleasure, activation, and dominance has laid the methodological foundation for fine-grained emotion regression modeling using few-channel EEG.

[0003] Existing emotion recognition methods typically begin by acquiring prefrontal cortex multichannel EEG signals, which are then filtered and notched to remove power line interference before time-frequency domain and brain network features are extracted. A regression model is then used to map these features to a continuous coordinate space comprising pleasure, activation, and dominance, enabling point-by-point output of emotion intensity. For cross-subject scenarios, some systems introduce inter-subject feature standardization or transfer learning components to adapt to the cortical activity patterns of different individuals.

[0004] However, existing models are extremely sensitive to blinking, eye movement and electromyography interference that occur under natural wearing conditions, and do not have an adaptive compensation mechanism built in for differences in individual baseline heart rate and electrode contact quality. This results in frequent drift of emotional coordinate output when applied across subjects, which cannot meet the stability requirements of closed-loop feedback systems. Summary of the Invention

[0005] In view of the shortcomings of the prior art, this application provides a closed-loop emotion recognition method and system to solve the problem that the stability requirements of the closed-loop feedback system cannot be met.

[0006] To achieve the above objectives, this application provides the following technical solution:

[0007] The first aspect of this application provides a method for identifying closed-loop emotions, including:

[0008] Acquire the user's frontal sparse-channel EEG signal;

[0009] The prefrontal scan signal is input into a pre-trained emotion decoding model, which outputs a continuous emotion vector in the PAD continuous space representing the user's current emotional state. The emotion decoding model is pre-trained using sample prefrontal scan signals from multiple subjects and their corresponding PAD scales. The model is considered successfully trained if the loss function meets a preset value. The loss function is calculated based on individual cue vectors and a penalty mechanism.

[0010] Determine whether the continuous emotion vector satisfies the closed-loop triggering condition;

[0011] If the continuous emotion vector satisfies the closed-loop triggering condition, a closed-loop feedback instruction is generated based on the continuous emotion vector, and the closed-loop feedback instruction is executed to provide feedback adjustment to the user.

[0012] Optionally, in the above-described closed-loop emotion recognition method, the training method of the emotion decoding model includes:

[0013] Acquire individual cue vectors, sample prefrontal low-channel EEG signals from multiple subjects, and their corresponding PAD scales;

[0014] For each of the subjects, an EEG feature matrix was extracted from the sample frontal few-channel EEG signal of the subject;

[0015] When the subject's circadian rhythm signal is acquired, heart rate variability features are extracted from the circadian rhythm signal, and the heart rate variability features are concatenated with the EEG feature matrix to obtain a multi-domain feature matrix.

[0016] When no physiological rhythm signals of the subject are collected, the EEG feature matrix is ​​used as a multi-domain feature matrix;

[0017] The multi-domain feature matrix is ​​dimensionality reduced to obtain the dimensionality-reduced principal component features, and the individual cue vector is concatenated with the principal component features as the training input features.

[0018] The training input features are input into the emotion decoding model, and the output is a continuous vector of sample emotion representing the emotional state of the subject in the PAD continuous space.

[0019] The loss function is calculated based on the error between the sample emotion continuous vector and its corresponding PAD scale, the difference in latent space mean among multiple subjects under the same emotion label, and the weight norm of the individual cue vector.

[0020] Determine whether the loss function meets the preset value;

[0021] If the loss function satisfies the preset value, then the sentiment decoding model is determined to be the trained sentiment decoding model;

[0022] If the loss function does not meet the preset value, the parameters of the emotion decoding model are adjusted, and the process of inputting the training input features into the emotion decoding model and outputting a continuous vector of sample emotion in the PAD continuous space representing the sample emotional state of the subject is returned.

[0023] Optionally, the above-mentioned closed-loop emotion recognition method further includes:

[0024] The linear consistency between the sample emotion continuous vector and its corresponding PAD scale was calculated using the Pearson correlation coefficient formula to obtain the correlation coefficient value.

[0025] The mean absolute error (MAE) formula is used to calculate the average deviation between the sample emotion continuous vector and its corresponding PAD scale to obtain the MAE value. The correlation coefficient value and the MAE value are used as evaluation indicators of the predictive efficacy of the emotion decoding model.

[0026] Optionally, in the above-described closed-loop emotion recognition method, acquiring the user's frontal few-channel EEG signal includes:

[0027] Acquire the user's brainwave signals;

[0028] Based on the resting segment, an electrooculogram (EOG) and environmental noise reference template were established, and interference signals in the EEG signal were suppressed to obtain the user's frontal few-channel EEG signal.

[0029] Optionally, in the above-described closed-loop emotion recognition method, determining whether the continuous emotion vector satisfies the closed-loop triggering condition includes:

[0030] The confidence scores of multiple windows are obtained; wherein the confidence scores are predicted by the sentiment decoding model based on the feature quality, electrode contact state, prediction residual distribution and coordinate fluctuation data of adjacent windows within the current window;

[0031] Extract pleasure or dominance from the continuous emotion vector;

[0032] Determine whether the pleasure level or the dominance level is less than a preset baseline threshold within a continuous time window, and whether the confidence level of each window is not less than a preset confidence threshold;

[0033] If the level of pleasure or the level of advantage is less than the preset baseline threshold within a continuous time window, and the confidence level of each window is not less than the preset confidence threshold, then the continuous emotion vector is determined to satisfy the closed-loop triggering condition.

[0034] Optionally, in the above-described closed-loop emotion recognition method, after executing the closed-loop feedback instruction to provide feedback adjustment to the user, the method further includes:

[0035] Record the first timestamp at the end of the current analysis window and the second timestamp when the closed-loop feedback command is issued, and use the difference between the second timestamp and the first timestamp as the end-to-end response delay;

[0036] Using the time of the closed-loop feedback instruction generation as the time reference point, the first average value of the comprehensive negative emotion index within a first preset time window before the time reference point, and the second average value of the comprehensive negative emotion index within a second preset time window after the time reference point are obtained; wherein, the comprehensive negative emotion index is obtained by weighted summation of the deviation of the pleasure level in the continuous emotion vector from a preset pleasure level threshold, the deviation of the dominance level in the continuous emotion vector from a preset dominance level threshold, and the deviation of the activation level in the continuous emotion vector from a preset activation level threshold;

[0037] Based on the first average value and the second average value, the overall negative emotion attenuation rate is calculated, and the overall negative emotion attenuation rate and the end-to-end response delay are used as evaluation indicators of feedback intervention effectiveness.

[0038] A second aspect of this application provides a closed-loop emotion recognition system, comprising:

[0039] The signal acquisition unit is used to acquire the user's frontal low-channel EEG signals;

[0040] The signal input unit is used to input the prefrontal scanpoint EEG signal into a pre-trained emotion decoding model and output a continuous emotion vector in the PAD continuous space representing the user's current emotional state. The emotion decoding model is pre-trained using sample prefrontal scanpoint EEG signals from multiple subjects and their corresponding PAD scales. The emotion decoding model is considered successfully trained if the loss function meets a preset value. The loss function is calculated based on individual cue vectors and a penalty mechanism.

[0041] The condition judgment unit is used to determine whether the continuous emotion vector satisfies the closed-loop triggering condition;

[0042] An execution unit is configured to generate a closed-loop feedback instruction based on the continuous emotion vector if the continuous emotion vector satisfies the closed-loop triggering condition, and execute the closed-loop feedback instruction to provide feedback adjustment to the user.

[0043] Optionally, the aforementioned closed-loop emotion recognition system also includes:

[0044] The data acquisition unit is used to acquire individual cue vectors, sample frontal few-channel EEG signals from multiple subjects, and their corresponding PAD scales;

[0045] The extraction unit is used to extract the EEG feature matrix from the sample frontal few-channel EEG signal of each subject, respectively.

[0046] The splicing unit is used to extract heart rate variability features from the physiological rhythm signals of the subject when the signals are acquired, and to splice the heart rate variability features with the EEG feature matrix to obtain a multi-domain feature matrix.

[0047] As a unit, it is used to use the EEG feature matrix as a multi-domain feature matrix when no physiological rhythm signal of the subject is collected;

[0048] The dimensionality reduction unit is used to perform dimensionality reduction processing on the multi-domain feature matrix to obtain the dimensionality-reduced principal component features, and to concatenate the individual cue vector with the principal component features as training input features.

[0049] The output unit is used to input the training input features into the emotion decoding model and output a continuous vector of sample emotion in the PAD continuous space representing the sample emotional state of the subject.

[0050] The loss calculation unit is used to calculate the loss function based on the error between the sample emotion continuous vector and its corresponding PAD scale, the difference in the latent space mean of multiple subjects under the same emotion label, and the weight norm of the individual cue vector.

[0051] The judgment unit is used to determine whether the loss function satisfies a preset value;

[0052] The first determining unit is used to determine the sentiment decoding model as a trained sentiment decoding model if the loss function satisfies a preset value.

[0053] The adjustment unit is used to adjust the parameters of the emotion decoding model if the loss function does not meet the preset value, and return to the step of inputting the training input features into the emotion decoding model and outputting a sample emotion continuous vector in the PAD continuous space that represents the sample emotion state of the subject.

[0054] Optionally, the aforementioned closed-loop emotion recognition system also includes:

[0055] The first calculation unit is used to calculate the linear consistency between the sample emotion continuous vector and its corresponding PAD scale using the Pearson correlation coefficient formula, and obtain the correlation coefficient value.

[0056] The second calculation unit is used to calculate the average deviation between the sample emotion continuous vector and its corresponding PAD scale using the mean absolute error formula, to obtain the mean absolute error value, and to use the correlation coefficient value and the mean absolute error value as the predictive efficacy evaluation index of the emotion decoding model.

[0057] Optionally, in the above-described closed-loop emotion recognition system, the signal acquisition unit includes:

[0058] Acquisition unit, used to acquire the user's electroencephalogram (EEG) signals;

[0059] The processing unit is used to establish reference templates for electrooculography and environmental noise according to the resting segment, suppress interference signals in the EEG signal, and obtain the user's frontal few-channel EEG signal.

[0060] Optionally, in the above-described closed-loop emotion recognition system, the condition judgment unit includes:

[0061] A confidence acquisition unit is used to acquire the confidence of multiple windows; wherein the confidence is predicted by the sentiment decoding model based on the feature quality, electrode contact state, prediction residual distribution and coordinate fluctuation data of adjacent windows within the current window;

[0062] A data extraction unit is used to extract pleasure or dominance from the continuous emotion vector.

[0063] A threshold determination unit is used to determine whether the pleasure level or the dominance level is less than a preset baseline threshold within a continuous time window, and whether the confidence level of each window is not less than a preset confidence threshold.

[0064] The second determining unit is configured to determine that the continuous emotion vector satisfies the closed-loop triggering condition if the pleasure level or the dominance level is less than the preset baseline threshold within a continuous time window, and the confidence level of each window is not less than the preset confidence threshold.

[0065] Optionally, the aforementioned closed-loop emotion recognition system also includes:

[0066] The recording unit is used to record the first timestamp at the end of the current analysis window and the second timestamp when the closed-loop feedback command is issued, and to use the difference between the second timestamp and the first timestamp as the end-to-end response delay;

[0067] The average value acquisition unit is used to acquire, with the closed-loop feedback instruction generation time as the time reference point, a first average value of the comprehensive negative emotion index within a first preset time window before the time reference point, and a second average value of the comprehensive negative emotion index within a second preset time window after the time reference point; wherein, the comprehensive negative emotion index is obtained by weighted summation of the deviation of the pleasure level in the continuous emotion vector from a preset pleasure level threshold, the deviation of the dominance level in the continuous emotion vector from a preset dominance level threshold, and the deviation of the activation level in the continuous emotion vector from a preset activation level threshold;

[0068] The third calculation unit is used to calculate the overall negative emotion attenuation rate based on the first average value and the second average value, and to use the overall negative emotion attenuation rate and the end-to-end response delay as evaluation indicators of feedback intervention effectiveness.

[0069] This application provides a closed-loop emotion recognition method. It acquires a user's frontal partial-channel EEG signal and inputs it into a pre-trained emotion decoding model. The model outputs a continuous emotion vector in the PAD continuous space representing the user's current emotional state. The emotion decoding model is pre-trained using frontal partial-channel EEG signals from multiple subjects and their corresponding PAD scales. Training is considered successful if the loss function meets a preset value. The loss function is calculated based on individual cue vectors and a penalty mechanism. Finally, it determines whether the continuous emotion vector meets the closed-loop triggering condition. If so, a closed-loop feedback instruction is generated based on the emotional continuous vector and executed to provide feedback adjustment to the user. This method, using a lightweight and convenient frontal partial-channel EEG signal acquisition approach, is suitable for continuous daily wear and has good engineering versatility. Multi-subject training combined with individual cue vectors and a penalty mechanism effectively compensates for differences in physiological baselines across subjects, improving the model's generalization stability and prediction accuracy across different users. Based on the closed-loop triggering and feedback regulation of PAD continuous space, discrete classification is transformed into dynamic intervention driven by continuous coordinates, which breaks through the limitations of traditional open-loop monitoring and thus enables stable and reliable closed-loop regulation of emotions. Attached Figure Description

[0070] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0071] Figure 1 A flowchart illustrating a closed-loop emotion recognition method provided in an embodiment of this application;

[0072] Figure 2 A flowchart illustrating a method for acquiring prefrontal multichannel EEG signals according to another embodiment of this application;

[0073] Figure 3 A schematic diagram of a lightweight BCI brain-computer interface (BCI) wearable headband provided for another embodiment of this application;

[0074] Figure 4 A PAD three-dimensional emotional continuous space mapping diagram is provided as another embodiment of this application;

[0075] Figure 5 A flowchart illustrating a training method for an emotion decoding model provided in another embodiment of this application;

[0076] Figure 6 A principal component analysis feature dimension-reduced variance distribution map is provided as another embodiment of this application;

[0077] Figure 7 A flowchart illustrating a method for evaluating the predictive performance of an emotion decoding model, provided as another embodiment of this application;

[0078] Figure 8 A flowchart illustrating a method for determining closed-loop triggering conditions, provided for another embodiment of this application;

[0079] Figure 9 This is a schematic diagram of the structure of a closed-loop emotion recognition system provided in another embodiment of this application. Detailed Implementation

[0080] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0081] In this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0082] This application provides a method for identifying closed-loop emotions, such as... Figure 1 As shown, the specific steps include:

[0083] S101. Obtain the user's frontal small-channel EEG signal.

[0084] Specifically, "frontal fractional-channel EEG signal" refers to the neurophysiological signal obtained through a wearable EEG acquisition device worn on the user's forehead. It has three leads, corresponding to Fp1, Fp2, and a reference position between them in the prefrontal cortex. The prefrontal region is closely related to emotion regulation, attention maintenance, and cognitive control. The purpose of using a fractional-channel configuration is to reduce device complexity and wearing burden, meeting the lightweight requirements of continuous daily use.

[0085] It should be noted that the three-lead configuration has fewer signal channels, making it easier to integrate into portable headband devices. However, compared to multi-lead EEG acquisition systems, it places higher demands on the anti-interference capabilities of subsequent signal processing algorithms.

[0086] It should also be noted that, limited by the size and power consumption constraints of wearable front-end devices, this embodiment divides the computational task into three levels: edge acquisition, edge inference, and cloud collaboration. The edge headband is responsible for raw EEG acquisition and timestamp marking. Proximal edge nodes are responsible for filtering, artifact detection, fundamental frequency domain energy conversion, HRV feature access, PCA projection, and SVR emotion coordinate inference. The cloud server is used to store model versions, summarize anonymized statistical features, perform federated or batch updates, and generate offline analysis reports. Considering that emotion coordinates involve sensitive physiological information, the ideas of federated learning and distributed training are more suitable for model updates than for real-time inference over the public network.

[0087] Optionally, in another embodiment of this application, one specific implementation of step S101 is as follows: Figure 2 As shown, the specific steps include:

[0088] S201. Obtain the user's brainwave signals.

[0089] Specifically, in this embodiment, a three-lead wearable forehead EEG headband is used to primarily collect neural electrical activity near the prefrontal cortex as EEG signals. The prefrontal region is closely related to emotion regulation, attention maintenance, and cognitive control. The wearable forehead EEG headband can be found in [reference needed]. Figure 3 As shown.

[0090] S202. Establish reference templates for electrooculography and environmental noise according to the resting segment, suppress interference signals in the EEG signal, and obtain the user's frontal few-channel EEG signal.

[0091] Understandably, under three-lead conditions, blind source separation often fails to meet the basic requirement that the number of observation channels exceeds the number of independent sources. Therefore, the algorithm employs a more conservative short-window preprocessing strategy: establishing a reference template for EEG and environmental noise using the resting segment, and then using bandpass filtering, power frequency notch filtering, amplitude thresholding, recursive least squares regression, and abnormal window removal to suppress interference signals such as blinking, slow drift, and power frequency contamination in the EEG signal. Specifically, the "resting segment" refers to the EEG recording segment in which the user is relaxed with eyes closed and without a specific cognitive task; the signal within this segment contains typical features of EEG, EMG, and environmental noise. The "reference template" refers to the feature vector of interference signals extracted from the resting segment signal, used as a reference for adaptive filtering or regression correction in signal processing.

[0092] S102. Input the frontal few-channel EEG signal into the pre-trained emotion decoding model and output the continuous emotion vector in the PAD continuous space representing the user's current emotional state.

[0093] The emotion decoding model is pre-trained using sample prefrontal few-channel EEG signals from multiple subjects and their corresponding PAD scales. The emotion decoding model is considered successfully trained when the loss function meets the preset value. The loss function is calculated based on individual cue vectors and a penalty mechanism.

[0094] Specifically, the "PAD continuous space" refers to a three-dimensional emotional measurement space composed of three orthogonal dimensions: Pleasure (P), Arousal (A), and Dominance (D). Pleasure represents the positive or negative polarity of emotional experience; a higher value indicates a more positive subjective experience. Arousal represents the level of neural and autonomic physiological arousal; a higher value indicates a stronger degree of bodily activation. Dominance represents an individual's sense of control over the current emotion and situation; a higher value indicates a stronger sense of subjective control. In this continuous space, the emotional state at any given moment is represented as a three-dimensional vector E=[P, A, D]. The "continuous emotional vector" refers to the value of this three-dimensional vector at a specific moment. For more details, please refer to... Figure 4 The diagram shown is a three-dimensional continuous emotional space mapping of PAD.

[0095] Understandably, the PAD scale is a standardized questionnaire tool used to obtain subjective emotion ratings. Participants rate each dimension on a nine-point scale or a numbered scale from 0 to 8 through an interactive interface. The raw scores are then converted into standard coordinates after being centered and normalized.

[0096] Specifically, to avoid mixing the original scale intervals with the model input intervals, this paper divides the PAD scale conversion into two steps. The interactive interface can display a nine-level rating or a centralized rating, but the model calculation uniformly uses a centralized variable x_i, where i belongs to {P, A, D}, and the value range of x_i is [-4, 4]. When the original scale uses scores from 1 to 9, first let x_i = s_i - 5. When the software uses level numbers from 0 to 8, first let x_i = r_i - 4. If the interface uses 0 to 9 as the display scale, it needs to be converted to [-4, 4] according to the actual level mapping before proceeding to the normalization step. The normalized PAD coordinates are defined as follows:

[0097]

[0098] in, The PAD coordinates are normalized, where s_i represents the original PAD scale score (1-9 points) and r_i represents the gear number (0-8) in the software interface.

[0099] Therefore, the software-quantized standard coordinates are essentially equivalent to the centered score divided by 4. The previous version used the same expression for interval translation and scaling, which easily led to confusion regarding the two intervals: 0 to 9 and -4 to 4. This version simplifies it to the form described above. Within the PAD emotional space, the difference between the two emotional states is measured using Euclidean distance:

[0100]

[0101] Wherein, P1, A1, and D1 are the coordinate values ​​of the first emotional state in the dimensions of pleasure, activation, and dominance, respectively, and P2, A2, and D2 are the coordinate values ​​of the second emotional state in the dimensions of pleasure, activation, and dominance, respectively. It should be noted that the first and second emotional states are any two emotional states in the PAD continuous space to be measured for difference.

[0102] Optionally, another embodiment of this application provides a training method for an emotion decoding model, such as... Figure 5 As shown, it includes the following steps:

[0103] S501. Obtain individual cue vectors, sample frontal few-channel EEG signals from multiple subjects, and their corresponding PAD scales.

[0104] Understandably, the "individual cue vector" is not a cue in the form of natural language text, but rather a low-dimensional conditional vector composed of individual resting baseline statistics. Its function is to provide limited correction for individual baseline differences in the cue model. Its construction method is as follows: Obtain the resting EEG signal of each subject, and extract the baseline relative power and alpha / beta ratio of each channel. When physiological rhythm signals can be acquired synchronously, extract the baseline mean and fluctuation amplitude from the resting physiological rhythm signals. Simultaneously, obtain the mean electrode contact quality and age group grouping variable for each subject, and combine these quantities into a low-dimensional conditional vector as the individual cue vector for that subject. This vector is concatenated with principal component features and input into the regressor to compensate for physiological baseline differences between individuals.

[0105] The PAD scale is a standardized questionnaire tool used to obtain subjective emotion ratings. Participants rate each dimension on a nine-point scale or a numbered scale from 0 to 8 through an interactive interface. The raw scores are then converted into standard coordinates after centering and normalization.

[0106] Specifically, the testing environment for subjects was designed to minimize light, noise, and strong electromagnetic interference. A standard emotional image library and customized immersive videos were used as audiovisual evoked sources. The entire experimental period was divided into three phases: baseline resting-state recording, stimulus presentation, and subjective scale calibration. After each trial, subjects completed the PAD Continuous Affect Scale score through an interactive interface. This score was used only as a label for offline training, model calibration, and posterior evaluation, and not as a real-time monitoring signal during the stimulus process. During the experiment, subjects wore a wearable headband to collect EEG data, and heart rate or pulse signals were recorded simultaneously when available to support HRV feature extraction.

[0107] S502. For each subject, extract the EEG feature matrix from the sample frontal few-channel EEG signal.

[0108] Specifically, after short-window preprocessing of the sample EEG signals, multi-domain features are extracted from each time window. These multi-domain features include time-domain statistical features, frequency-domain energy features, and nonlinear dynamic features. Time-domain features include mean, standard deviation, peak value, skewness, and kurtosis. Frequency-domain features include relative power in the delta, theta, alpha, beta, and gamma bands, as well as the alpha / beta ratio. Nonlinear features include approximate entropy, sample entropy, and Hjorth complexity. These features are calculated separately at the window level and then combined into an EEG feature matrix.

[0109] S503. When the subject's physiological rhythm signal is collected, the heart rate variability feature is extracted from the physiological rhythm signal, and the heart rate variability feature is spliced ​​with the EEG feature matrix to obtain a multi-domain feature matrix.

[0110] Understandably, "physiological rhythm signals" refer to synchronously acquired heart rate or pulse signals, encompassing heart rate data from electrocardiogram (ECG) sensors and pulse wave data from photoplethysmography (PPG). "Heart rate variability characteristics" refer to variability statistics calculated from RR intervals or pulse intervals, including at least one of the following: mean NN interval, SDNN, RMSSD, pNN50, LF power, HF power, LF / HF ratio, and Poincare plot SD1 / SD2 index. Concatenating heart rate variability characteristics with the EEG feature matrix involves appending the heart rate variability characteristics to the EEG feature matrix along its feature dimensions, forming a multi-domain feature matrix that integrates central nervous system signals and peripheral physiological signals. This multi-domain feature matrix, through z-score normalization at the window level and preservation of missing masks, constitutes a matrix representing an individual's instantaneous psychophysiological state.

[0111] S504. When no physiological rhythm signals of the subject are collected, the EEG feature matrix is ​​used as the multi-domain feature matrix.

[0112] It is understood that the training method in this application embodiment is compatible with scenarios where only EEG signals are collected, and the training process can still be completed normally even when the conditions for synchronous physiological rhythm collection are not available.

[0113] S505. Perform dimensionality reduction on the multi-domain feature matrix to obtain the dimensionality-reduced principal component features, and concatenate the individual cue vector with the principal component features as the training input features.

[0114] Specifically, due to the strong collinearity in the multi-domain feature matrix, directly inputting it into the regression model can easily increase the risk of overfitting. Therefore, in this embodiment, it is necessary to first perform dimensionality reduction on the multi-domain feature matrix. Specifically, the "dimensionality reduction" adopts the principal component analysis method, which projects the original high-dimensional features to a low-dimensional principal component space by performing an orthogonal transformation on the multi-domain feature matrix. For example, the original feature dimension is 50 dimensions (42 dimensions for EEG features and 8 dimensions for rhythm variability features). A cumulative variance contribution rate threshold of 95% is set, and principal components are selected in descending order of variance contribution rate until the cumulative variance contribution rate reaches this threshold. In the complete multimodal configuration, the number of retained principal components is 8. See, for example, [link to example]. Figure 6 The content shown, Figure 6 The display shows the individual variance contribution rate and cumulative variance contribution rate of the top 10 principal components. The number of principal components retained is based on the minimum dimension that reaches the threshold. This setting references the approach of using PCA to reduce channel and frequency band feature redundancy in real-time VAD estimation.

[0115] Principal component analysis aims to reduce collinearity among features, lower the risk of overfitting, and reduce the computational load of marginal inference. In order for the model to make limited corrections to individual baseline differences, it is also necessary to concatenate the individual cue vector with the principal component features before inputting it into the sentiment decoding model.

[0116] S506. Input the training input features into the emotion decoding model and output the sample emotion continuous vector in the PAD continuous space that represents the sample emotion state of the subject.

[0117] It should be noted that, in this embodiment, the concept of a psychophysiological computational field is also introduced, treating time-varying EEG and peripheral physiological indicators together as an observed projection of emotional state. Considering that heart rate variability can reflect changes in autonomic nervous activity, in trials with synchronized heart rate or pulse data, the real-time neurophysiological state can be expressed as a function of multimodal features:

[0118]

[0119] Where G represents the nonlinear mapping operator, x EEG(t) The time-domain, frequency-domain, and nonlinear characteristics of EEG are represented by x_HRV(t), which represents the heart rate variability calculated from the RR interval or pulse interval, and q(t) represents auxiliary quality control variables such as electrode contact quality, motion state, or missing mask. If only EEG data is collected in a certain experimental batch, the HRV term will not participate in the training and inference of that batch, and the model input dimensions will be adjusted accordingly. The predicted coordinates of the PAD emotional space can be represented as:

[0120]

[0121] in, These represent the pleasure, activation, and dominance coordinates of the subject at time t, as predicted by the emotion decoding model.

[0122] This modeling approach allows the system to transform short-window physiological feature distributions into continuous emotion indicators. It is important to emphasize that the real-time output is based on model predictions that have been calibrated offline, rather than subjective rating labels obtained immediately during the stimulus process.

[0123] It should also be noted that, in the embodiments of this application, the emotion decoding model continuously decodes the three-dimensional emotion changes of the subjects. Table 1 shows the internal statistical results of the model in this paper on the current dataset.

[0124] Table 1: Statistics on the Prediction Accuracy of PAD's Three-Dimensional Emotion Dimension

[0125]

[0126] As shown in Table 1, the pleasure and activation dimensions obtained high correlation coefficients in the current sample, while the dominance dimension had a relatively low correlation coefficient, suggesting that the subjective rating consistency and physiological observability of dominance may be weaker than the first two dimensions. These results indicate that the proposed method has a certain continuous predictive ability under the current experimental settings, but it is insufficient to prove its statistical advantage over traditional models. Future research should supplement this with baseline model reproduction experiments using a unified training set, a unified test set, and the same evaluation metrics, and report the confidence intervals or significance test results.

[0127] S507. Calculate the loss function based on the error between the sample emotion continuous vector and its corresponding PAD scale, the difference in the latent space mean of multiple subjects under the same emotion label, and the regularization term of the individual cue vector.

[0128] Specifically, the formula for calculating the loss function is as follows:

[0129] in, This is an interval penalty term, calculated based on the error between the sample sentiment continuous vector and the PAD scale. It is used to control the number of samples whose prediction error exceeds the epsilon insensitivity interval. The source domain consistency penalty term is calculated based on the difference in latent space mean among multiple subjects under the same emotion label, and is used to constrain the latent space distribution shift of different subjects near the same emotion label. The regularization term for the individual cue vector is calculated based on the weight norm of the individual cue vector and is used to limit overfitting caused by excessively large cue vector weights. λ and η are preset weight coefficients used to balance the contribution ratio of each term to the total loss. The calculation of the margin penalty term and the source domain consistency penalty term can be implemented using mathematical forms commonly used in the fields of support vector regression and domain generalization, and the specific derivation process will not be elaborated in this application.

[0130] S508. Determine whether the loss function meets the preset value.

[0131] Specifically, the "loss function" is an evaluation metric used to quantify the degree of difference between the predicted output of the sentiment decoding model and the true label. Its value reflects the fitting quality of the model in the current training state. Therefore, the value of the currently calculated loss function is compared with the preset value, and the subsequent execution path is determined based on the comparison result. If the preset value requirement is met, the process proceeds to the model solidification step, i.e., step S509 is executed. If the requirement is not met, iterative training continues, i.e., step S510 is executed.

[0132] S509. Determine the sentiment decoding model as the trained sentiment decoding model.

[0133] Specifically, when the loss function meets the preset value, it indicates that the prediction accuracy and generalization performance of the current emotion decoding model have reached the preset requirements. At this time, the model parameters in the current state are fixed, iterative training is stopped, and the model is determined as a trained emotion decoding model that can be deployed and applied in actual closed-loop emotion recognition scenarios.

[0134] S510, Adjust the parameters of the emotion decoding model.

[0135] It should be noted that when the loss function does not meet the preset value, the gradient is simultaneously backpropagated to the feature mapping layer and parameter weight layer of the support vector regression model, and the model parameters are updated through backpropagation. Then, the forward calculation and loss evaluation of S505 to S507 are re-executed, and this process is repeated until the loss function converges to the preset value.

[0136] Optionally, as described in the above training method embodiments, after the emotion decoding model is trained, its predictive performance needs to be objectively and quantitatively evaluated to verify whether the model has sufficient predictive accuracy to support subsequent closed-loop feedback decisions. To comprehensively measure the predictive consistency and bias of the model in the PAD continuous emotion space, another embodiment of this application provides a method for evaluating the predictive performance of an emotion decoding model, such as... Figure 7 As shown, the specific steps include:

[0137] S701. Calculate the linear consistency between the sample sentiment continuous vector and its corresponding PAD scale using the Pearson correlation coefficient formula to obtain the correlation coefficient value.

[0138] Specifically, the Pearson correlation coefficient is used to characterize the linear correlation between the predicted coordinate sequence and the scale label sequence. Its value ranges from -1 to 1. The closer the value is to 1, the higher the linear consistency between the predicted coordinates and the true labels, meaning the better the model's prediction accuracy in the continuous sentiment space. Its calculation formula is:

[0139]

[0140] in, The coordinate sequence representing the continuous vector of sample sentiment. This represents the PAD scale label sequence. and Let represent the mean of the two values, and N be the total number of trials involved in the calculation.

[0141] S702. Calculate the average deviation between the sample emotion continuous vector and its corresponding PAD scale using the mean absolute error formula to obtain the mean absolute error value, and use the correlation coefficient value and the mean absolute error value as evaluation indicators of the predictive efficacy of the emotion decoding model.

[0142] Specifically, the mean absolute error (MAE) reflects the average deviation between the predicted coordinates and the label coordinates. A smaller value indicates that the predicted coordinates are closer to the true label, meaning the model has higher absolute positioning accuracy. Its calculation formula is:

[0143]

[0144] in, The predicted coordinates represent the continuous vector of sample sentiment. This represents the PAD scale label corresponding to the continuous vector of sample emotions.

[0145] It is understandable that the correlation coefficient focuses on evaluating the trend consistency between the prediction and the label, while the mean absolute error focuses on evaluating the magnitude of the absolute deviation between the prediction and the label. The two complement each other and together constitute the predictive performance evaluation index of the model.

[0146] S103. Determine whether the continuous emotion vector satisfies the closed-loop triggering condition.

[0147] It should be noted that the “closed-loop triggering condition” refers to the preset decision criteria used to determine whether feedback intervention needs to be initiated. When the user’s current emotional state deviates from the preset normal range and the model prediction results meet the reliability requirements, the condition is met, thereby triggering subsequent feedback adjustment actions. That is, when the continuous emotional vector meets the closed-loop triggering condition, step S104 is executed.

[0148] Optionally, if the continuous emotion vector does not meet the closed-loop triggering condition, no operation needs to be performed.

[0149] It should also be noted that the closed-loop trigger condition is not a binary judgment based on a single threshold, but a comprehensive judgment based on multiple prediction results within a continuous time window. Its design aims to avoid false triggers caused by fluctuations in a single prediction, while ensuring timely intervention when the user's emotional state continues to deviate.

[0150] Optionally, in another embodiment of this application, one specific implementation of step S103 is as follows: Figure 8 As shown, the specific steps include:

[0151] S801, Obtain the confidence scores of multiple windows.

[0152] It should be noted that, based on continuously analyzed three-dimensional emotion coordinates, this embodiment incorporates a dynamic threshold warning module, which provides warnings based on the confidence levels of multiple windows. The confidence level is predicted by the emotion decoding model based on the feature quality within the current window, electrode contact state, prediction residual distribution, and coordinate fluctuation data of adjacent windows.

[0153] Specifically, the system divides the EEG signal into sliding windows with a 4-second window size and a 1-second step size. Features are extracted for each valid window and input into the emotion decoding model. The model outputs the PAD coordinates and a confidence estimate for that window. The confidence level is determined by four factors: the quality of the signal features within the window (quantified by amplitude threshold discrimination results), the electrode contact state (quantified by impedance detection values), the distribution of the model's prediction residuals in the current window (quantified by historical prediction error statistics on the validation set), and the amplitude of coordinate fluctuations between adjacent windows (quantified by the Euclidean distance between the PAD coordinates of the current window and the previous window). These quantified values ​​are weighted and summed, then mapped to the [0, 1] interval to obtain the confidence estimate.

[0154] S802, Extract pleasure or dominance from the continuous emotion vector.

[0155] It should be noted that the reason for extracting pleasure level in this step is that the closed-loop triggering condition in this application embodiment uses pleasure level or advantage level as the main judgment basis.

[0156] S803. Determine whether the pleasure level or advantage level is less than the preset baseline threshold within a continuous time window, and whether the confidence level of each window is not less than the preset confidence threshold.

[0157] It is understood that a "continuous time window" refers to a time interval containing the most recent valid windows, with the current time as the endpoint. For example, the number of valid windows is 3, and the window length is 4 seconds. The "preset confidence threshold" is 0.70, but other thresholds can also be used, and the specific setting can be adjusted according to requirements. The purpose of setting the confidence threshold is to ensure that intervention is triggered only when the model's prediction reliability is high, avoiding false triggers due to low-quality predictions. Therefore, if the pleasure level or dominance level is less than the preset baseline threshold within the continuous time window, and the confidence level of each window is not less than the preset confidence threshold, it indicates that the user's emotion meets the early warning mechanism. Therefore, it is determined that the continuous emotion vector meets the closed-loop triggering condition, i.e., step S804 is executed.

[0158] Optionally, if the pleasure level or dominance level is not less than a preset baseline threshold within a continuous time window, and the confidence level of each window is less than a preset confidence threshold, it indicates that the user's emotions do not meet the early warning mechanism, and therefore it is determined that the continuous emotion vector does not meet the closed-loop triggering condition.

[0159] S804. Determine if the continuous emotion vector satisfies the closed-loop triggering condition.

[0160] It's important to note that during the online testing phase of the closed-loop intervention, the model, calibrated offline, is used to predict physiological characteristics within a sliding time window. This means the continuous emotional coordinate triggering logic is limited to model output-driven, rather than driven by the subject's subjective rating after a one-minute trial. Specifically, the system extracts features using a 4-second window and a 1-second step, outputting the PAD predicted coordinates and confidence level estimates. Confidence level is determined by the quality of features within the window, electrode contact status, prediction residual distribution, and coordinate fluctuations in adjacent windows. Feedback is triggered only if the pleasure level is below the individual's resting baseline preset threshold for three consecutive valid windows, and the confidence level is not lower than 0.70. The scale scores after the trial are used to update the offline dataset and verify prediction bias, but are not added back to the real-time triggering decision for that trial. This approach ensures consistency in the time logic of offline calibration, online inference, and closed-loop intervention. The experiment synchronously recorded timestamps, raw microvolt signals, preprocessing flags, dimensionality reduction features, predicted coordinates, confidence levels, and intervention execution flags throughout the entire cycle, providing a basis for subsequent verification. See Table 2 for the standardized emotion induction and closed-loop regulation experimental stage parameter table.

[0161] Table 2

[0162]

[0163] S104. Generate closed-loop feedback instructions based on the continuous emotion vector, and execute the closed-loop feedback instructions to provide feedback adjustment to the user.

[0164] Specifically, a "closed-loop feedback instruction" refers to a control signal that triggers the terminal to execute a feedback action. For example, the feedback action provides gentle audio-visual guidance and breathing rhythm regulation suggestions through an interactive interface, guiding the user to regulate their own state. This process forms a complete closed-loop path from physiological perception and state estimation to feedback regulation. Existing reviews of emotion-based brain-computer interfaces indicate that closed-loop aBCI systems typically include two stages: emotion recognition and feedback regulation. However, real-world applications still require careful consideration of cross-subject differences, feedback safety, and the design of controlled experiments.

[0165] Optionally, as described above regarding the generation and execution of closed-loop feedback instructions, after generating and executing the closed-loop feedback instructions in step S104, the system completes the feedback adjustment intervention for the user. To objectively and quantitatively evaluate the actual effectiveness of this feedback intervention, a comprehensive evaluation of the intervention process is needed from two dimensions: response speed and emotion regulation effect. Response speed determines whether the system can intervene promptly after the emotional state deviates from the threshold, while emotion regulation effect determines whether the intervention can effectively improve the user's negative emotional state. Therefore, this application provides a method for evaluating the effectiveness of feedback intervention, which achieves a quantitative evaluation of the effectiveness of closed-loop feedback intervention by calculating the end-to-end response delay and the overall negative emotion attenuation rate, specifically including the following processes Z1-Z3:

[0166] Z1: Record the first timestamp at the end of the current analysis window and the second timestamp when the closed-loop feedback command is issued, and use the difference between the second timestamp and the first timestamp as the end-to-end response delay.

[0167] As can be understood, "end-to-end response latency" refers to the time interval between the end of the current analysis window (i.e., the moment when the system outputs a continuous emotion vector after completing feature extraction and emotion decoding model inference) and the moment the terminal feedback command is issued. Its latency includes edge buffering, preprocessing, feature extraction, PCA projection, SVR inference, and local feedback command generation time, but excludes public network cloud transmission and cloud queuing time. This latency metric is used to verify whether the system's real-time response capability meets the engineering requirements of closed-loop control.

[0168] Specifically, the formula for calculating the end-to-end response latency is: Tlink = tfeedback - tacq.

[0169] Where t_acq represents the timestamp at the end of the current analysis window (first timestamp), and t_feedback represents the time when the terminal feedback command was issued (second timestamp). This metric includes the time for edge buffering, preprocessing, feature extraction, PCA projection, SVR inference, and local feedback command generation, but excludes public network cloud transmission and cloud queuing time. Under local edge inference and stable local area network conditions, the system's average link record is 176 ms. This result only demonstrates the engineering feasibility of the prototype system in a controlled environment and cannot be extrapolated to the stable latency of cloud public network inference.

[0170] Z2: Using the time of the closed-loop feedback instruction generation as the time reference point, obtain the first average value of the comprehensive negative emotion index within the first preset time window before the time reference point, and the second average value of the comprehensive negative emotion index within the second preset time window after the time reference point.

[0171] The negative emotion comprehensive index is obtained by weighted summation of the deviations of the pleasure level in the continuous emotion vector from a preset pleasure level threshold, the deviations of the dominance level in the continuous emotion vector from a preset dominance level threshold, and the deviations of the activation level in the continuous emotion vector from a preset activation level threshold.

[0172] Understandably, the "first preset time window" refers to a preset time interval before the feedback instruction is generated, used to characterize the user's baseline emotional level before the feedback intervention. For example, this time window is set to 10 seconds before the feedback instruction is generated. The "second preset time window" refers to a preset time interval after the feedback instruction is generated, used to characterize the user's emotional state after the feedback intervention. For example, this time window is set to 30 seconds after the feedback instruction is generated.

[0173] Specifically, the formula for calculating the comprehensive negative emotion index is as follows:

[0174]

[0175] in, (Pleasure level) (Dominance) and (Activation) represents the pleasure, dominance, and activation coordinates obtained by the emotion decoding model based on the user's prefrontal few-channel EEG signal at time t during closed-loop application. w_P, w_D, and w_A are empirical weights. , , These are the pleasure threshold, dominance threshold, and activation threshold, respectively.

[0176] Z3: Based on the first and second average values, calculate the overall negative emotion attenuation rate, and use the overall negative emotion attenuation rate and the end-to-end response delay as evaluation indicators of feedback intervention effectiveness.

[0177] Specifically, the formula for calculating the overall negative emotion attenuation rate is as follows:

[0178]

[0179] in, The first average value represents the baseline level of the comprehensive negative emotion index before feedback intervention. The second average value represents the tracking level of the comprehensive negative emotion index after feedback intervention. A higher attenuation rate indicates a more significant improvement in negative emotions through closed-loop feedback intervention. This attenuation rate, together with the end-to-end response latency, constitutes the evaluation system for the effectiveness of feedback intervention—latency reflects the system's real-time response capability, and the attenuation rate reflects the actual improvement effect of the intervention on the user's emotions. Both verify the closed-loop regulatory efficacy of the technical solution in this application from the dimensions of engineering feasibility and physiological effectiveness, respectively. According to this definition, this paper observed a DR of 37.2% and a state recovery achievement rate of 88.5%.

[0180] This application provides a closed-loop emotion recognition method. It acquires a user's frontal partial-channel EEG signal and inputs it into a pre-trained emotion decoding model. The model outputs a continuous emotion vector in the PAD continuous space representing the user's current emotional state. The emotion decoding model is pre-trained using frontal partial-channel EEG signals from multiple subjects and their corresponding PAD scales. Successful training is determined when the loss function meets a preset value. The loss function is calculated based on individual cue vectors and a penalty mechanism. Finally, it determines whether the continuous emotion vector meets the closed-loop triggering condition. If the condition is met, a closed-loop feedback instruction is generated and executed to provide feedback adjustment to the user. This method, using a lightweight and convenient frontal partial-channel EEG signal acquisition approach, is suitable for continuous daily wear and has good engineering versatility. Multi-subject training combined with individual cue vectors and a penalty mechanism effectively compensates for differences in physiological baselines across subjects, improving the model's generalization stability and prediction accuracy across different users. Based on the closed-loop triggering and feedback regulation of PAD continuous space, discrete classification is transformed into dynamic intervention driven by continuous coordinates, which breaks through the limitations of traditional open-loop monitoring and thus enables stable and reliable closed-loop regulation of emotions.

[0181] Another embodiment of this application provides a closed-loop emotion recognition system, such as... Figure 9 As shown, it includes the following units:

[0182] The signal acquisition unit 901 is used to acquire the user's frontal few-channel EEG signal.

[0183] The signal input unit 902 is used to input prefrontal scanpoint EEG signals into a pre-trained emotion decoding model and output a continuous emotion vector representing the user's current emotional state in the PAD continuous space. The emotion decoding model is pre-trained using sample prefrontal scanpoint EEG signals from multiple subjects and their corresponding PAD scales. The emotion decoding model is considered successfully trained if the loss function meets a preset value. The loss function is calculated based on individual cue vectors and a penalty mechanism.

[0184] Condition judgment unit 903 is used to determine whether the continuous emotion vector satisfies the closed-loop triggering condition.

[0185] The execution unit 904 is used to generate a closed-loop feedback instruction based on the continuous emotion vector if the continuous emotion vector satisfies the closed-loop triggering condition, and execute the closed-loop feedback instruction to provide feedback adjustment to the user.

[0186] It should be noted that the specific working process of the above-mentioned units in the embodiments of this application can be referred to steps S101 to S104 in the above method embodiments, and will not be repeated here.

[0187] Optionally, another embodiment of this application provides a closed-loop emotion recognition system, which further includes:

[0188] The data acquisition unit is used to acquire individual cue vectors, sample frontal few-channel EEG signals from multiple subjects, and their corresponding PAD scales.

[0189] The extraction unit is used to extract the EEG feature matrix from the sample frontal few-channel EEG signal of each subject.

[0190] The splicing unit is used to extract heart rate variability features from the physiological rhythm signals of the subject when they are acquired, and to splice the heart rate variability features with the EEG feature matrix to obtain a multi-domain feature matrix.

[0191] As a unit, it is used to treat the EEG feature matrix as a multi-domain feature matrix when no physiological rhythm signals of the subject are collected.

[0192] The dimensionality reduction unit is used to reduce the dimensionality of the multi-domain feature matrix to obtain the dimensionality-reduced principal component features. The individual cue vector is then concatenated with the principal component features and used as the training input features.

[0193] The output unit is used to input the training input features into the emotion decoding model and output a continuous vector of sample emotion in the PAD continuous space, representing the sample emotional state of the subject.

[0194] The loss calculation unit is used to calculate the loss function based on the error between the sample emotion continuous vector and its corresponding PAD scale, the difference in the latent space mean of multiple subjects under the same emotion label, and the weight norm of the individual cue vector.

[0195] The judgment unit is used to determine whether the loss function meets the preset value.

[0196] The first determining unit is used to determine the sentiment decoding model as the trained sentiment decoding model if the loss function meets the preset value.

[0197] The adjustment unit is used to adjust the parameters of the emotion decoding model if the loss function does not meet the preset value, and then return to the process of inputting the training input features into the emotion decoding model and outputting a continuous vector of sample emotion in the PAD continuous space that represents the sample emotional state of the subject.

[0198] Optionally, another embodiment of this application provides a closed-loop emotion recognition system, which further includes:

[0199] The first calculation unit is used to calculate the linear consistency between the sample emotion continuous vector and its corresponding PAD scale using the Pearson correlation coefficient formula, and obtain the correlation coefficient value.

[0200] The second calculation unit is used to calculate the average deviation between the sample emotion continuous vector and its corresponding PAD scale using the mean absolute error formula, obtain the mean absolute error value, and use the correlation coefficient value and the mean absolute error value as evaluation indicators of the predictive efficacy of the emotion decoding model.

[0201] Optionally, in another embodiment of this application, a closed-loop emotion recognition system includes a signal acquisition unit 901, comprising:

[0202] The acquisition unit is used to acquire the user's electroencephalogram (EEG) signals.

[0203] The processing unit is used to establish reference templates for electrooculography and environmental noise according to the resting segment, suppress interference signals in the EEG signal, and obtain the user's frontal few-channel EEG signal.

[0204] Optionally, in another embodiment of this application, a closed-loop emotion recognition system includes a condition judgment unit 903, comprising:

[0205] The confidence acquisition unit is used to acquire the confidence scores of multiple windows. The confidence scores are predicted by the sentiment decoding model based on the feature quality within the current window, electrode contact state, prediction residual distribution, and coordinate fluctuation data of adjacent windows.

[0206] The data extraction unit is used to extract pleasure or dominance from the continuous vector of emotions.

[0207] The threshold judgment unit is used to determine whether the pleasure or advantage score is less than a preset baseline threshold within a continuous time window, and whether the confidence score of each window is not less than a preset confidence threshold.

[0208] The second determining unit is used to determine that the continuous emotion vector satisfies the closed-loop triggering condition if the pleasure level or dominance level is less than a preset baseline threshold within a continuous time window, and the confidence level of each window is not less than a preset confidence threshold.

[0209] Optionally, another embodiment of this application provides a closed-loop emotion recognition system, which further includes:

[0210] The recording unit is used to record the first timestamp at the end of the current analysis window and the second timestamp when the closed-loop feedback command is issued, and uses the difference between the second timestamp and the first timestamp as the end-to-end response delay.

[0211] The average value acquisition unit is used to acquire, with the closed-loop feedback instruction generation time as the time reference point, a first average value of the comprehensive negative emotion index within a first preset time window before the time reference point, and a second average value of the comprehensive negative emotion index within a second preset time window after the time reference point. The comprehensive negative emotion index is obtained by weighted summation of the deviations in the continuous emotion vector: the deviation of pleasure level below a preset pleasure level threshold, the deviation of dominance level below a preset dominance level threshold, and the deviation of activation level above a preset activation level threshold.

[0212] The third calculation unit is used to calculate the overall negative emotion attenuation rate based on the first average and the second average, and to use the overall negative emotion attenuation rate and the end-to-end response delay as evaluation indicators of feedback intervention effectiveness.

[0213] It should be noted that the specific working process of each unit provided in the above embodiments of this application can be referred to the corresponding steps in the above method embodiments, and will not be repeated here.

[0214] It should also be noted that the closed-loop emotion recognition system provided in this application has the technical effects of any of the above embodiments, and will not be described in detail here.

[0215] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0216] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A closed-loop emotion recognition method, characterized in that, include: Acquire the user's three-channel frontal EEG signals; The three-channel frontal EEG signal is input into a pre-trained emotion decoding model, which outputs a continuous emotion vector in the PAD continuous space representing the user's current emotional state. The emotion decoding model is pre-trained using sample three-channel frontal EEG signals from multiple subjects and their corresponding PAD scales. The model is considered successfully trained if the loss function meets a preset value. The loss function is calculated based on individual cue vectors and a penalty mechanism. The individual cue vector is constructed as follows: Resting-state EEG signals of each subject are acquired, and the baseline relative power and alpha / beta ratio of each channel are extracted. When physiological rhythm signals can be acquired synchronously, the baseline mean and fluctuation amplitude are extracted from the resting-state physiological rhythm signals. Simultaneously, the mean electrode contact quality and age group grouping variable of each subject are acquired. These quantities are combined into a low-dimensional conditional vector as the subject's individual cue vector. Determine whether the continuous emotion vector satisfies the closed-loop triggering condition; If the continuous emotion vector satisfies the closed-loop triggering condition, a closed-loop feedback instruction is generated based on the continuous emotion vector, and the closed-loop feedback instruction is executed to provide feedback adjustment to the user. The step of determining whether the continuous emotion vector satisfies the closed-loop triggering condition includes: The confidence scores of multiple windows are obtained; wherein the confidence scores are predicted by the sentiment decoding model based on the feature quality, electrode contact state, prediction residual distribution and coordinate fluctuation data of adjacent windows within the current window; Extract pleasure or dominance from the continuous emotion vector; Determine whether the pleasure level or the dominance level is less than a preset baseline threshold within a continuous time window, and whether the confidence level of each window is not less than a preset confidence threshold; If the level of pleasure or the level of advantage is less than the preset baseline threshold within a continuous time window, and the confidence level of each window is not less than the preset confidence threshold, then the continuous emotion vector is determined to satisfy the closed-loop triggering condition.

2. The method according to claim 1, characterized in that, The training method for the emotion decoding model includes: Acquire individual cue vectors, sample three-channel frontal EEG signals from multiple subjects, and their corresponding PAD scales; For each of the subjects, an EEG feature matrix was extracted from the three-channel EEG signal of the frontal region of the subject. When the subject's circadian rhythm signal is acquired, heart rate variability features are extracted from the circadian rhythm signal, and the heart rate variability features are concatenated with the EEG feature matrix to obtain a multi-domain feature matrix. When no physiological rhythm signals of the subject are collected, the EEG feature matrix is ​​used as a multi-domain feature matrix; The multi-domain feature matrix is ​​dimensionality reduced to obtain the dimensionality-reduced principal component features, and the individual cue vector is concatenated with the principal component features as the training input features. The training input features are input into the emotion decoding model, and the output is a continuous vector of sample emotion representing the emotional state of the subject in the PAD continuous space. The loss function is calculated based on the error between the sample emotion continuous vector and its corresponding PAD scale, the difference in latent space mean among multiple subjects under the same emotion label, and the weight norm of the individual cue vector. Determine whether the loss function meets the preset value; If the loss function satisfies the preset value, then the sentiment decoding model is determined to be the trained sentiment decoding model; If the loss function does not meet the preset value, the parameters of the emotion decoding model are adjusted, and the process of inputting the training input features into the emotion decoding model and outputting a continuous vector of sample emotion in the PAD continuous space representing the sample emotional state of the subject is returned.

3. The method according to claim 2, characterized in that, Also includes: The linear consistency between the sample emotion continuous vector and its corresponding PAD scale was calculated using the Pearson correlation coefficient formula to obtain the correlation coefficient value. The mean absolute error (MAE) formula is used to calculate the average deviation between the sample emotion continuous vector and its corresponding PAD scale to obtain the MAE value. The correlation coefficient value and the MAE value are used as evaluation indicators of the predictive efficacy of the emotion decoding model.

4. The method according to claim 1, characterized in that, The acquisition of the user's three-channel frontal EEG signal includes: Acquire the user's brainwave signals; Based on the resting segment, an electrooculogram (EOG) and environmental noise reference template were established, and interference signals in the EEG signal were suppressed to obtain the user's frontal three-channel EEG signal.

5. The method according to claim 1, characterized in that, After executing the closed-loop feedback instruction to adjust the feedback for the user, the method further includes: Record the first timestamp at the end of the current analysis window and the second timestamp when the closed-loop feedback command is issued, and use the difference between the second timestamp and the first timestamp as the end-to-end response delay; Using the time of the closed-loop feedback instruction generation as the time reference point, the first average value of the comprehensive negative emotion index within a first preset time window before the time reference point, and the second average value of the comprehensive negative emotion index within a second preset time window after the time reference point are obtained; wherein, the comprehensive negative emotion index is obtained by weighted summation of the deviation of the pleasure level in the continuous emotion vector from a preset pleasure level threshold, the deviation of the dominance level in the continuous emotion vector from a preset dominance level threshold, and the deviation of the activation level in the continuous emotion vector from a preset activation level threshold; Based on the first average value and the second average value, the overall negative emotion attenuation rate is calculated, and the overall negative emotion attenuation rate and the end-to-end response delay are used as evaluation indicators of feedback intervention effectiveness.

6. A closed-loop emotion recognition system, characterized in that, include: The signal acquisition unit is used to acquire the user's three-channel frontal EEG signals; The signal input unit is used to input the three-channel frontal EEG signal into a pre-trained emotion decoding model and output a continuous emotion vector in the PAD continuous space representing the user's current emotional state. The emotion decoding model is pre-trained using sample three-channel frontal EEG signals from multiple subjects and their corresponding PAD scales. The emotion decoding model is considered successfully trained if the loss function meets a preset value. The loss function is calculated based on individual cue vectors and a penalty mechanism. The individual cue vector is constructed as follows: The resting-state EEG signal of each subject is acquired, and the baseline relative power and alpha / beta ratio of each channel are extracted. When physiological rhythm signals can be acquired synchronously, the baseline mean and fluctuation amplitude are extracted from the resting-state physiological rhythm signals. Simultaneously, the mean electrode contact quality and age group grouping variable of each subject are acquired, and these quantities are combined into a low-dimensional conditional vector as the subject's individual cue vector. The condition judgment unit is used to determine whether the continuous emotion vector satisfies the closed-loop triggering condition; An execution unit is configured to generate a closed-loop feedback instruction based on the continuous emotion vector if the continuous emotion vector satisfies the closed-loop triggering condition, and execute the closed-loop feedback instruction to provide feedback adjustment to the user. The condition judgment unit includes: A confidence acquisition unit is used to acquire the confidence of multiple windows; wherein the confidence is predicted by the sentiment decoding model based on the feature quality, electrode contact state, prediction residual distribution and coordinate fluctuation data of adjacent windows within the current window; A data extraction unit is used to extract pleasure or dominance from the continuous emotion vector. A threshold determination unit is used to determine whether the pleasure level or the dominance level is less than a preset baseline threshold within a continuous time window, and whether the confidence level of each window is not less than a preset confidence threshold. The second determining unit is configured to determine that the continuous emotion vector satisfies the closed-loop triggering condition if the pleasure level or the dominance level is less than the preset baseline threshold within a continuous time window, and the confidence level of each window is not less than the preset confidence threshold.

7. The system according to claim 6, characterized in that, Also includes: The data acquisition unit is used to acquire individual cue vectors, sample frontal three-channel EEG signals from multiple subjects, and their corresponding PAD scales; The extraction unit is used to extract the electroencephalogram feature matrix from the three-channel electroencephalogram signal of the frontal region of each subject. The splicing unit is used to extract heart rate variability features from the physiological rhythm signals of the subject when the signals are acquired, and to splice the heart rate variability features with the EEG feature matrix to obtain a multi-domain feature matrix. As a unit, it is used to use the EEG feature matrix as a multi-domain feature matrix when no physiological rhythm signal of the subject is collected; The dimensionality reduction unit is used to perform dimensionality reduction processing on the multi-domain feature matrix to obtain the dimensionality-reduced principal component features, and to concatenate the individual cue vector with the principal component features as training input features. The output unit is used to input the training input features into the emotion decoding model and output a continuous vector of sample emotion in the PAD continuous space representing the sample emotional state of the subject. The loss calculation unit is used to calculate the loss function based on the error between the sample emotion continuous vector and its corresponding PAD scale, the difference in the latent space mean of multiple subjects under the same emotion label, and the weight norm of the individual cue vector. The judgment unit is used to determine whether the loss function satisfies a preset value; The first determining unit is used to determine the sentiment decoding model as a trained sentiment decoding model if the loss function satisfies a preset value. The adjustment unit is used to adjust the parameters of the emotion decoding model if the loss function does not meet the preset value, and return to the step of inputting the training input features into the emotion decoding model and outputting a sample emotion continuous vector in the PAD continuous space that represents the sample emotion state of the subject.

8. The system according to claim 7, characterized in that, Also includes: The first calculation unit is used to calculate the linear consistency between the sample emotion continuous vector and its corresponding PAD scale using the Pearson correlation coefficient formula, and obtain the correlation coefficient value. The second calculation unit is used to calculate the average deviation between the sample emotion continuous vector and its corresponding PAD scale using the mean absolute error formula, to obtain the mean absolute error value, and to use the correlation coefficient value and the mean absolute error value as the predictive efficacy evaluation index of the emotion decoding model.

9. The system according to claim 6, characterized in that, The signal acquisition unit includes: Acquisition unit, used to acquire the user's electroencephalogram (EEG) signals; The processing unit is used to establish reference templates for electrooculography and environmental noise according to the resting segment, suppress interference signals in the EEG signal, and obtain the user's frontal three-channel EEG signal.

Citation Information

Patent Citations

  • Closed-loop electroencephalogram neural feedback system based on wearable glasses and closed-loop feedback method thereof

    CN121817913A

  • Emotion closed-loop regulation system and method based on electroencephalogram feedback

    CN122515784A