Emotion recognition dual-flow hidden space calibration method based on wearable sensor
Patent Information
- Application Number
- CN202610364643.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-24
- Publication Date
- 2026-08-07
AI Technical Summary
[0003]然而，现有基于深度学习的可穿戴传感器情感识别方法仍面临关键技术瓶颈：
[0067]（一）本发明通过构建生理-运动双流特征编码架构，实现生理信号与运动加速度信号的独立表征学习，能够精准区分情感相关信息与运动干扰信息，有效解耦生理信号中的情感表征与运动伪影，避免传统多模态融合方法简单拼接特征导致的运动噪声掩盖弱生理情感特征的问题。
Smart Images
Figure CN122527469A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing and emotion computing recognition technology, and in particular to a two-stream latent space calibration method for emotion recognition based on wearable sensors. Background Technology
[0002] With the development of wearable sensors and physiological signal processing technologies, multimodal physiological signal emotion recognition based on wearable sensors has been widely applied in fields such as intelligent health monitoring, human-machine collaborative integration, industrial human factors engineering, and sports health management. Deep learning technology, with its end-to-end temporal feature extraction and modeling capabilities, has made significant progress in the field of physiological signal emotion recognition, enabling the direct extraction of fine-grained emotion-related features from raw multimodal time-series data such as electrocardiograms, skin conductance, and motion acceleration. Compared to traditional manual feature engineering methods, this technology significantly improves the automation level and scene adaptability of emotion recognition.
[0003] However, existing deep learning-based wearable sensor emotion recognition methods still face key technical bottlenecks:
[0004] (i) In practical applications, human limb movements can introduce significant motion artifacts into core physiological signals such as electrocardiogram and skin conductance. Motion noise is deeply coupled with emotion-related signals, reducing the signal-to-noise ratio and seriously interfering with the effective extraction and learning of emotion-related features.
[0005] (ii) There are strong individual differences in physiological signals. The physiological baseline levels and emotional response patterns of different subjects are significantly different. There is a clear data distribution bias in cross-subject assessment, which leads to insufficient cross-subject generalization ability of the model.
[0006] (iii) Traditional multimodal fusion methods often use simple feature splicing or joint training modes, which do not clearly distinguish between emotion-related information and motion interference information. They may even cause problems such as motion branch gradient backpropagation interfering with the extraction of physiological features, making it difficult to achieve effective decoupling of motion artifacts and emotion representation.
[0007] In addition, emotion recognition datasets generally suffer from class imbalance. Traditional loss functions tend to favor the emotion class with the largest sample size, resulting in poor recognition performance of minority emotion states. Summary of the Invention
[0008] In view of the shortcomings of the prior art, the purpose of this invention is to provide a dual-stream latent space calibration method for emotion recognition based on wearable sensors, so as to solve one or more problems in the prior art.
[0009] To achieve the above objectives, the technical solution of the present invention is as follows:
[0010] A dual-stream latent space calibration method for emotion recognition based on wearable sensors includes the following steps:
[0011] S1. Model construction, including a physiological-motor dual-stream feature encoding architecture, a latent space residual calibration module with gradient isolation, and a fully connected classification layer;
[0012] S2, Collect multimodal time-series data through wearable sensors and based on Normalization preprocessing was performed to eliminate amplitude differences in the physiological signals of the subjects.
[0013] S3. Normalized multimodal temporal data are input into a physiological-motor dual-stream feature encoding architecture to extract physiological initial and motor-related representations respectively;
[0014] S4. Input the two types of representations into the latent space residual calibration module with gradient isolation to obtain the calibrated clean physiological representations.
[0015] S5. Input the calibrated clean physiological representations into the fully connected classification layer to obtain the emotion category prediction results and form a training set.
[0016] S6. Based on the joint optimization strategy of balance loss and contrast center regularization, the model is trained and optimized using the training set.
[0017] Furthermore, the physiological-motor dual-stream feature encoding architecture includes a physiological signal encoding stream and a motion acceleration signal encoding stream. The two encoding streams are structurally isomorphic and their parameters are independent of each other. Both are composed of a one-dimensional convolutional module, a group normalization layer, and an attention pooling layer.
[0018] Furthermore, the normalized preprocessing calculation formula is shown below:
[0019]
[0020] In the formula: This is a sample of the original multimodal time series data. This represents the mean of the signal sequence corresponding to the current subject. The standard deviation of the signal sequence corresponding to the current subject. The normalized time series data sample has a distribution with a mean of 0 and a standard deviation of 1.
[0021] Furthermore, the extraction of physiological initials and movement-related representations includes the following steps:
[0022] S31. Input the preprocessed multimodal timing data into the corresponding encoded streams respectively;
[0023] S32. Local temporal features of two types of signals are extracted by combining a one-dimensional convolutional neural network with group normalization. The calculation formula for the one-dimensional convolution is shown below:
[0024]
[0025] In the formula: The size of the one-dimensional convolution kernel. , The weights and biases of the convolutional layer, For group normalization activation operation, For the convolutional layer at position Output eigenvalues For the input signal at position The original value;
[0026] S33. Calculate dynamic weights for features at each time step using an attention pooling layer. The calculation formula is shown below:
[0027]
[0028] In the formula: , These are the learnable parameters of the attention layer. For the first Attention weights at each time step This represents the time step of the feature sequence after convolution;
[0029] S34. The feature sequences are weighted and summed along the time dimension, as shown in the following formula:
[0030]
[0031]
[0032] In the formula: This represents the initial physiological characteristics with motion interference. The encoded stream of motion acceleration signals outputs motion-related representations. Encoding the physiological signal stream in the first Eigenvalues of the time step Encoding the motion acceleration signal stream in the first... Eigenvalues of the time step.
[0033] Furthermore, the latent space residual calibration module includes:
[0034] The bounded gain calculation unit is based on a multilayer perceptron, a sigmoid activation function, and hyperparameters for a preset gain range.
[0035] Gradient isolation units are constructed based on the StopGradient operator;
[0036] The calibration direction vector estimation unit is composed of feature concatenation operation, multilayer perceptron, Tanh activation function and scaling factor hyperparameter;
[0037] The small-step residual correction unit is based on fixed-step hyperparameters, additive correction operations, and layer normalization.
[0038] Furthermore, obtaining clean physiological characterization includes the following steps:
[0039] S41. Calculate the bounded gain using a multilayer perceptron and based on motion-related representations, and convert the scalar gain... Constrained within a preset range Inside, there are multilayer perceptrons The basic mapping calculation formula is shown below:
[0040]
[0041] In the formula: Motion representation via multilayer perceptron The intermediate feature vector after the first layer of nonlinear mapping , For multilayer perceptron Learnable parameters for each layer It is a linear rectification activation function;
[0042] scalar gain The calculation formula is shown below:
[0043]
[0044] In the formula: For multilayer perceptron Learnable parameters for each layer Use the Sigmoid activation function;
[0045] S42. Perform gradient isolation on the initial physiological representation to avoid motor branches interfering with the learning of physiological features, where the expression for stopping gradient isolation is shown in the following formula:
[0046]
[0047] In the formula: To stop the gradient operation function, Learnable parameters for encoding physiological signal streams;
[0048] S43. The calibration direction vector is obtained by concatenating the physiological and motion representations after gradient isolation and mapping them together, as shown in the following equation:
[0049]
[0050] In the formula: Bit calibration direction vector This is the scaling factor;
[0051] S44. Targeted residual correction of initial physiological characteristics is performed using small-step additive correction, as shown in the following formula:
[0052]
[0053] In the formula: For calibrated clean physiological characterization, For fixed hyperparameters.
[0054] Furthermore, the calculation formula for the fully connected classification layer is as follows:
[0055]
[0056] In the formula: , Here are the weight and bias matrices for the classification layer, where K=3 represents the number of emotion categories, corresponding to baseline, stress, and pleasure, respectively. Predict the logit for the sentiment category.
[0057] Furthermore, the balancing loss is addressed by introducing a balancing soft maximization loss. To alleviate the class imbalance problem in the dataset, the calculation formula is as follows:
[0058]
[0059] In the formula: For the true label of the sample, For the model to perform on the current sample The logit output on the top For the model to evaluate the current sample in the th... Logit output on the class, The true labels in the training set are The number of samples, For the training set The number of samples in a class.
[0060] Furthermore, contrast center regularization The calculation formula is shown below:
[0061]
[0062] In the formula: For batch size, The first in the batch Clean physiological characterization of each sample, For the first The feature center of the class For the first The feature center of the class It is the L2 norm. This is the hyperparameter for the distance boundary between features of different classes.
[0063] Furthermore, the joint optimization strategy constructs a total optimization loss function that integrates balanced soft maximization loss and contrast center regularization. The calculation formula is shown below:
[0064]
[0065] In the formula: To balance the hyperparameters.
[0066] Compared with the prior art, the beneficial technical effects of the present invention are as follows:
[0067] (i) This invention constructs a physiological-motor dual-stream feature coding architecture to achieve independent representation learning of physiological signals and motion acceleration signals. It can accurately distinguish between emotion-related information and motion interference information, effectively decouple emotional representations and motion artifacts in physiological signals, and avoid the problem of motion noise masking weak physiological and emotional features caused by simple feature splicing in traditional multimodal fusion methods.
[0068] (ii) This invention achieves adaptive orientation correction of physiological representation by using a latent space residual calibration module with gradient isolation, combined with a bounded gain mechanism and a small step size update strategy. At the same time, it uses a stopping gradient strategy to cut off the gradient interference of motion branches on the physiological feature extractor, ensuring the purity of physiological emotion representation, greatly improving the model's ability to suppress motion artifacts, and enhancing the stability of emotion recognition in dynamic motion environments.
[0069] (III) This invention adopts a joint optimization strategy of balanced loss and contrast center regularization. By balancing soft maximization loss, it alleviates the class imbalance problem of emotion recognition dataset. Combined with contrast center regularization to constrain feature space distribution, it strengthens the intra-class compactness and inter-class separability of features, effectively alleviates the data distribution bias problem in cross-subject evaluation, and significantly improves the cross-subject generalization ability of the model. Attached Figure Description
[0070] Figure 1 The diagram illustrates a flowchart of a dual-stream latent space calibration method for emotion recognition based on wearable sensors, according to an embodiment of the present invention. Detailed Implementation
[0071] To make the objectives, technical solutions, and advantages of this invention clearer, the following detailed description of a dual-stream latent space calibration method for emotion recognition based on wearable sensors, in conjunction with the accompanying drawings and specific embodiments, provides further clarity. The advantages and features of this invention will become clearer from the following description. It should be noted that the accompanying drawings are in a very simplified form and use non-precise proportions, used only to facilitate and clearly illustrate the purpose of the embodiments of this invention. Please refer to the accompanying drawings to make the objectives, features, and advantages of this invention more apparent and understandable. It should be understood that the structures, proportions, sizes, etc., depicted in the accompanying drawings are only for illustrative purposes to aid those skilled in the art and are not intended to limit the implementation conditions of this invention. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in proportions, or adjustments to the size, without affecting the effects and objectives achieved by this invention, should still fall within the scope of the technical content disclosed in this invention.
[0072] A dual-stream latent space calibration method for emotion recognition based on wearable sensors includes the following steps:
[0073] S1. Model construction, including a physiological-motor dual-stream feature encoding architecture, a latent space residual calibration module with gradient isolation, and a fully connected classification layer.
[0074] Furthermore, the physiological-motor dual-stream feature encoding architecture includes a physiological signal encoding stream and a motion acceleration signal encoding stream. The two encoding streams are structurally isomorphic and their parameters are independent, each consisting of a one-dimensional convolutional module, a group normalization layer, and an attention pooling layer. Specifically, the one-dimensional convolutional module is used to extract local temporal patterns of the signal. The group normalization layer is used to maintain the diversity of feature representations while stabilizing the training process. The attention pooling layer generates attention weights for each time step through a learnable fully connected mapping, achieving adaptive aggregation of temporal information.
[0075] Furthermore, the latent space residual calibration module includes:
[0076] The bounded gain calculation unit, based on a multilayer perceptron, a sigmoid activation function, and preset gain range hyperparameters, is used to adaptively adjust the calibration amplitude according to the dynamic intensity of motion-related characteristics, avoiding over-calibration or under-calibration.
[0077] The gradient isolation unit, based on the StopGradient operator, is used to cut off the gradient propagation path from the motion branch to the physiological coding flow, thereby fundamentally eliminating the interference of motion artifacts on physiological feature learning.
[0078] The calibration direction vector estimation unit, based on feature splicing operations, multilayer perceptron, Tanh activation function and scaling factor hyperparameter, is used to learn the optimal transformation direction from motion disturbance state to clean physiological state in joint feature space.
[0079] The small-step residual correction unit, based on fixed-step hyperparameters, additive correction operations, and layer normalization, is used to gradually correct physiological characterization in a stable and controllable manner, ensuring the numerical stability and convergence of the calibration process.
[0080] Furthermore, the fully connected classification layer uses a single-layer linear mapping to project the calibrated clean physiological representations onto the emotion category space, and the output dimension is consistent with the preset number of emotion categories, thereby achieving end-to-end emotion state discrimination.
[0081] S2, Collect multimodal time-series data through wearable sensors and based on Normalization preprocessing was performed to eliminate amplitude differences in the physiological signals of the subjects.
[0082] Normalization preprocessing is performed individually for each subject, using the mean and standard deviation of the subject's current signal sequence for standardization transformation. This effectively suppresses baseline drift and amplitude fluctuations caused by individual differences among subjects, while preserving the relative dynamic characteristics of signal changes within each subject. Specifically, the normalization preprocessing calculation formula is shown in Equation 1 below:
[0083] (1)
[0084] In the formula: This is a sample of the original multimodal time series data. This represents the mean of the signal sequence corresponding to the current subject. The standard deviation of the signal sequence corresponding to the current subject. The normalized time series data sample has a distribution with a mean of 0 and a standard deviation of 1.
[0085] By employing a subject-independent normalization strategy, the distribution shift caused by individual physiological baseline differences in cross-subject emotion recognition was effectively resolved, providing a stable and comparable input distribution for subsequent two-stream feature encoding.
[0086] S3. Normalized multimodal time-series data is input into a physiological-motor dual-stream feature encoding architecture to extract physiological initial and motor-related representations, respectively. Specifically, the extraction of physiological initial and motor-related representations includes the following steps:
[0087] S31. Input the preprocessed multimodal time-series data into the corresponding encoding streams. That is, input the normalized physiological signal and motion acceleration signal into the physiological signal encoding stream and the motion acceleration signal encoding stream, respectively. The two encoding streams adopt an isomorphic one-dimensional convolutional feature extraction structure.
[0088] S32. Local temporal features of two types of signals are extracted by combining a one-dimensional convolutional neural network with group normalization. The calculation formula of the one-dimensional convolution is shown in Equation 2 below:
[0089] (2)
[0090] In the formula: The size of the one-dimensional convolution kernel. , The weights and biases of the convolutional layer, For group normalization activation operation, For the convolutional layer at position Output eigenvalues For the input signal at position The original value.
[0091] By stacking multiple layers of one-dimensional convolutional and group normalization layers, feature abstraction is performed layer by layer on the two types of signals, resulting in a dimension of [missing information]. Feature sequences ,in For feature dimension, This represents the time step after convolution.
[0092] S33. Dynamic weights are calculated for features at each time step using an attention pooling layer. To capture temporal segments in the feature sequence that are discriminative for emotion recognition, an attention pooling layer is introduced to calculate dynamic weights for each time step of the feature sequence. First, the features at each time step are mapped using a learnable linear transformation, and then the attention weights are obtained by normalization using the Softmax function. The calculation formula is shown in Equation 3 below:
[0093] (3)
[0094] In the formula: , These are the learnable parameters of the attention layer. For the first Attention weights for each time step. The time step of the convolutional feature sequence (output length of the convolutional layer) is determined by the time step of the original multimodal temporal data. (Input layer length) convolution is used to obtain the result.
[0095] S34. Weighted summation of the feature sequence along the time dimension is performed to compress the temporal features into a fixed-dimensional global representation, and the physiological signal encoded stream outputs an initial physiological representation containing motion interference. The output of the motion acceleration signal encoded stream is a motion-related representation. The calculation formulas are shown in equations 4 and 5 below:
[0096] (4)
[0097] (5)
[0098] In the formula: Encoding the physiological signal stream in the first Eigenvalues of the time step Encoding the motion acceleration signal stream in the first... The eigenvalues of the time step are finally obtained. , Fixed-dimensional representation.
[0099] By employing a parallel processing mechanism within a dual-stream coding architecture, complete decoupling of physiological signals and motion acceleration signals during the feature extraction stage is achieved, avoiding the motion artifact penetration problem caused by early feature confounding in traditional single-stream fusion methods. Furthermore, the introduction of attention pooling layers allows the model to adaptively focus on temporal segments crucial for emotional state discrimination, rather than simply averaging global temporal information, thereby enhancing the discriminativeness and robustness of the representation.
[0100] S4. Input the two types of representations into the latent space residual calibration module with gradient isolation to obtain the calibrated clean physiological representations. The specific clean physiological representation acquisition includes the following steps:
[0101] S41, via multilayer sensor And based on motion-related representations Calculate the bounded gain for motion representation Perform a nonlinear mapping and apply the scalar gain to the Sigmoid activation function. Constrained within a preset range Within this context, to avoid excessive calibration amplitude leading to a shift in the characterization distribution, the multilayer perceptron... The basic mapping calculation formula is shown in Equation 6 below:
[0102] (6)
[0103] In the formula: Motion representation via multilayer perceptron The intermediate feature vector after the first layer of nonlinear mapping , For multilayer perceptron Learnable parameters for each layer It is a linear rectification activation function.
[0104] Furthermore, scalar gain The calculation formula is shown in Equation 7 below:
[0105] (7)
[0106] In the formula: For multilayer perceptron Learnable parameters for each layer This is the Sigmoid activation function.
[0107] S42. Perform gradient isolation on the initial physiological representations to avoid motor branching interfering with physiological feature learning. First, perform gradient isolation on the initial physiological representations. Perform a gradient-stopping operation to cut off the backpropagation interference of the motion branch gradients on the physiological feature extractor. The expression for gradient isolation stopping is shown in Equation 8 below:
[0108] (8)
[0109] In the formula: To stop the gradient operation function, These are learnable parameters for encoding physiological signal streams.
[0110] S43. After concatenating the gradient-isolated physiological and motion representations, a calibration direction vector is obtained by mapping. This vector is then input into the multilayer perceptron. Perform nonlinear mapping, via Multiply by scaling factor after activation To improve numerical stability in the early stages of training, a calibration direction vector is obtained. The definition is shown in Equation 9 below:
[0111] (9)
[0112] In the formula: To calibrate the direction vector, This is the scaling factor.
[0113] S44. Oriented residual correction is performed on the initial physiological representation using small-step additive correction. This embodiment introduces fixed hyperparameters by performing small-step feature updates. By limiting the update step size of single-step calibration, directional residual correction is performed on the initial physiological representation containing motion disturbances using an additive correction method, resulting in a clean physiological representation after calibration. To ensure the smoothness of the calibration process and the stability of training, the calculation formula is shown in Equation 10 below:
[0114] (10)
[0115] In the formula: For calibrated clean physiological characterization, To fix the hyperparameters, this formula achieves bounded control over the physiological representation of the latent space, making... To keep the feature distribution shift within a limited range and reduce motion disturbances.
[0116] By combining gradient isolation and bounded gain strategies, the gradient propagation of motion artifacts into the physiological feature learning process is fundamentally blocked. Simultaneously, adaptive gain adjustment enables dynamic control of calibration intensity, avoiding the limitations of fixed calibration amplitudes that cannot adapt to diverse motion disturbance intensities. Furthermore, the small-step residual correction strategy ensures the gradual and controllable nature of characterization updates, preventing distortion of clean physiological features due to aggressive calibration.
[0117] S5. Input the calibrated clean physiological representation into the fully connected classification layer to realize the mapping from the fixed-dimensional representation to the logit of the sentiment category prediction, and obtain the sentiment category prediction results to form a training set. The calculation formula of the fully connected classification layer is shown in Equation 11 below:
[0118] (11)
[0119] In the formula: , Here are the weight and bias matrices for the classification layer, where K=3 represents the number of emotion categories, corresponding to baseline, stress, and pleasure, respectively. Predict the logit for the sentiment category.
[0120] Through a further classification process, a complete mapping from raw multimodal time-series data input to emotional state probability output is achieved. This classification layer directly projects clean physiological representations after residual calibration onto the emotional category space without additional feature engineering or post-processing steps, ensuring the efficiency of model inference and ease of deployment.
[0121] S6. An optimization model is trained using a joint optimization strategy based on balanced loss and contrast center regularization, combined with the training set. This is achieved by introducing a balanced soft maximization loss. By combining the prior frequency of samples from each category in the training set, the classification decision boundary is adaptively adjusted to alleviate the class imbalance problem in the dataset and improve the model's ability to identify minority class emotion samples. Then, contrast center regularization is added to constrain similar features to cluster towards the class center and maintain the minimum distance boundary between dissimilar features, thereby enhancing the intra-class compactness and inter-class separability of features, alleviating the domain shift problem in cross-subject evaluation, and improving the model's generalization ability.
[0122] Specifically, balancing soft maximization loss The calculation formula is shown in Equation 12 below:
[0123] (12)
[0124] In the formula: For the true label of the sample, For the model to perform on the current sample On Output, For the model to evaluate the current sample in the th... On class Output, The true labels in the training set are The number of samples, For the training set The number of samples in a class.
[0125] Furthermore, contrast center regularization The calculation formula is shown in Equation 13 below:
[0126] (13)
[0127] In the formula: For batch size, The first in the batch Clean physiological characterization of each sample, For the first The feature center of the class For the first The feature center of the class It is the L2 norm. This is the hyperparameter for the distance boundary between features of different classes.
[0128] Furthermore, the joint optimization strategy constructs a total optimization loss function that integrates balanced soft maximization loss and contrast center regularization. Through hyperparameters By balancing the weights of the two losses and jointly optimizing all learnable parameters of the model, the model can learn feature representations with stronger cross-subject generalization ability while alleviating class imbalance. The calculation formula is shown in Equation 14 below:
[0129] (14)
[0130] During training, a stochastic gradient descent optimizer is used to minimize the total loss, iteratively updating the learnable parameters of the model until the loss converges to a stable value, thus obtaining the optimized sentiment recognition model. This joint optimization strategy, through the synergistic effect of balancing soft maximization loss and contrast center regularization, effectively improves the discriminative structure of the feature space while addressing the class imbalance problem.
[0131] In particular, the performance comparison results of the method of this invention with existing traditional machine learning and deep learning models on the WESAD public dataset based on the leave-one-out-of-participation (LOSO) cross-validation protocol are shown in Table 1 below (evaluation metric: overall recognition accuracy Acc (%), values are mean):
[0132] Table 1
[0133] RF 74.85 62.86 AB 80.34 72.51 LDA 79.35 74.43 MLP 78.11 72.19 FCN 79.04 74.23 ResNet 79.48 72.54 RF+Sensor Translation 81.40 70.50 Method of the present invention 82.27 71.04
[0134] The results show that the method of this invention significantly outperforms existing traditional machine learning and deep learning methods in terms of recognition accuracy and Macro-F1, as demonstrated in Table 2 of the ablation experiment. The results prove that the physiological-motor dual-stream feature encoding architecture, the latent space residual calibration module with gradient isolation, and the joint optimization strategy of balance loss and contrast center regularization are indispensable. Their synergistic effect greatly improves the model's cross-subject generalization ability and the robustness of emotion recognition in dynamic motion environments, resulting in an overall recognition accuracy (Acc) that is nearly 10 percentage points higher than the traditional machine learning baseline.
[0135] Table 2
[0136] Model variants Acc(%) Macro-F1(%) Baseline of physiological signals only 72.31 65.12 Remove gradient isolation mechanism 78.90 68.10 Remove the soft maximization loss 80.90 70.70 Complete DSLC Model 82.27 71.04
[0137] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0138] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.
Claims
1. A dual-stream latent space calibration method for emotion recognition based on wearable sensors, characterized in that, Includes the following steps: S1. Model construction, including a physiological-motor dual-stream feature encoding architecture, a latent space residual calibration module with gradient isolation, and a fully connected classification layer; S2, Collect multimodal time-series data through wearable sensors and based on Normalization preprocessing was performed to eliminate amplitude differences in the physiological signals of the subjects. S3. Normalized multimodal temporal data are input into a physiological-motor dual-stream feature encoding architecture to extract physiological initial and motor-related representations respectively; S4. Input the two types of representations into the latent space residual calibration module with gradient isolation to obtain the calibrated clean physiological representations. S5. Input the calibrated clean physiological representations into the fully connected classification layer to obtain the emotion category prediction results and form a training set. S6. Based on the joint optimization strategy of balance loss and contrast center regularization, the optimization model is trained using the training set.
2. The dual-stream latent space calibration method for emotion recognition based on wearable sensors as described in claim 1, characterized in that: The physiological-motor dual-stream feature encoding architecture includes a physiological signal encoding stream and a motion acceleration signal encoding stream. The two encoding streams are structurally isomorphic and their parameters are independent of each other. Both streams consist of a one-dimensional convolutional module, a group normalization layer, and an attention pooling layer.
3. The dual-stream latent space calibration method for emotion recognition based on wearable sensors as described in claim 1, characterized in that: The normalized preprocessing calculation formula is shown below: In the formula: This is a sample of the original multimodal time series data. This represents the mean of the signal sequence corresponding to the current subject. The standard deviation of the signal sequence corresponding to the current subject. The normalized time series data sample has a distribution with a mean of 0 and a standard deviation of 1.
4. The dual-stream latent space calibration method for emotion recognition based on wearable sensors as described in claim 1, characterized in that: The extraction of physiological initials and movement-related representations includes the following steps: S31. Input the preprocessed multimodal timing data into the corresponding encoded streams respectively; S32. Local temporal features of two types of signals are extracted by combining a one-dimensional convolutional neural network with group normalization. The calculation formula for the one-dimensional convolution is shown below: In the formula: The size of the one-dimensional convolution kernel. , The weights and biases of the convolutional layer, For group normalization activation operation, For the convolutional layer at position Output eigenvalues For the input signal at position The original value; S33. Calculate dynamic weights for features at each time step using an attention pooling layer. The calculation formula is shown below: In the formula: , For the learnable parameters of the attention layer, For the first Attention weights at each time step This represents the time step of the feature sequence after convolution; S34. The feature sequences are weighted and summed along the time dimension, as shown in the following formula: In the formula: This represents the initial physiological characteristics with motion interference. The encoded stream of motion acceleration signals outputs motion-related representations. Encoding the physiological signal stream in the first Eigenvalues of the time step Encoding the motion acceleration signal stream in the first... Eigenvalues of the time step.
5. The dual-stream latent space calibration method for emotion recognition based on wearable sensors as described in claim 1, characterized in that, The implicit space residual calibration module includes: The bounded gain calculation unit is based on a multilayer perceptron, a sigmoid activation function, and hyperparameters for a preset gain range. Gradient isolation units are constructed based on the StopGradient operator; The calibration direction vector estimation unit is composed of feature concatenation operation, multilayer perceptron, Tanh activation function and scaling factor hyperparameter; The small-step residual correction unit is based on fixed-step hyperparameters, additive correction operations, and layer normalization.
6. The dual-stream latent space calibration method for emotion recognition based on wearable sensors as described in claim 5, characterized in that: Obtaining clean physiological characterization involves the following steps: S41. Calculate the bounded gain using a multilayer perceptron and based on motion-related representations, and convert the scalar gain... Constrained within a preset range Inside, there are multilayer perceptrons The basic mapping calculation formula is shown below: In the formula: Motion representation via multilayer perceptron The intermediate feature vector after the first layer of nonlinear mapping , For multilayer perceptron Learnable parameters for each layer It is a linear rectification activation function; scalar gain The calculation formula is shown below: In the formula: For multilayer perceptron Learnable parameters for each layer Use the Sigmoid activation function; S42. Perform gradient isolation on the initial physiological representation to avoid motor branches interfering with the learning of physiological features, where the expression for stopping gradient isolation is shown in the following formula: In the formula: To stop the gradient operation function, Learnable parameters for encoding physiological signal streams; S43. The calibration direction vector is obtained by concatenating the physiological and motion representations after gradient isolation and mapping them together, as shown in the following equation: In the formula: Bit calibration direction vector This is the scaling factor; S44. Targeted residual correction of initial physiological characteristics is performed using small-step additive correction, as shown in the following formula: In the formula: For calibrated clean physiological characterization, For fixed hyperparameters.
7. The dual-stream latent space calibration method for emotion recognition based on wearable sensors as described in claim 1, characterized in that: The calculation formula for the fully connected classification layer is as follows: In the formula: , Here are the weight and bias matrices for the classification layer, where K=3 represents the number of emotion categories, corresponding to baseline, stress, and pleasure, respectively. Predict the logit for the sentiment category.
8. The dual-stream latent space calibration method for emotion recognition based on wearable sensors as described in claim 1, characterized in that: The balance loss is introduced by balancing soft maximization loss. To alleviate the class imbalance problem in the dataset, the calculation formula is as follows: In the formula: For the true label of the sample, For the model to perform on the current sample The logit output on the top For the model to evaluate the current sample in the th... Logit output on the class, The true labels in the training set are The number of samples, For the training set The number of samples in a class.
9. The dual-stream latent space calibration method for emotion recognition based on wearable sensors as described in claim 8, characterized in that: Contrast center regularization The calculation formula is shown below: In the formula: For batch size, The first in the batch Clean physiological characterization of each sample, For the first The feature center of the class For the first The feature center of the class It is the L2 norm. This is the hyperparameter for the distance boundary between features of different classes.
10. The dual-stream latent space calibration method for emotion recognition based on wearable sensors as described in claim 9, characterized in that: The joint optimization strategy constructs a total optimization loss function that integrates balanced soft maximization loss and contrast center regularization. The calculation formula is shown below: In the formula: To balance the hyperparameters.