Method and system for decoding emotion awakening degree

Abstract

The invention provides an emotion awakening degree decoding method and system, and relates to the field of emotion calculation, and the method comprises the steps: collecting a skin electric signal under emotion stimulation; the skin electric signals are preprocessed; the preprocessed skin electric signals are processed through a sympathetic nerve inversion network, and a network inversion value representing the sympathetic nerve activation level is obtained; the network inversion value is subjected to inverse mapping through an emotional wakeup degree inverse mapping network, a predicted value of the emotional wakeup degree is obtained, the sympathetic nerve inversion network and the emotional wakeup degree inverse mapping network form an end-to-end emotional wakeup degree decoding model, and a loss function of the emotional wakeup degree decoding model comprises the weighted sum of physiological loss and psychological loss. The physiological loss is used for calculating the difference between the network inversion value of the sympathetic nerve activation level and the theoretical inversion value, and the psychological loss is used for calculating the difference between the predicted value and the label value. According to the method, the problem of inaccurate emotion awakening degree calculation is solved.

Description

Technical Field

[0001] This application relates to the field of affective computing technology, and in particular to a method and system for decoding emotional arousal. Background Technology

[0002] Nowadays, more and more wearable devices support real-time collection of users' physiological signals, such as electrodermal signals. By analyzing these signals, users' emotional changes can be provided with emotional feedback and intervention suggestions, adding to the fun and practicality of wearable devices.

[0003] Emotional arousal and emotional valence are two core dimensions used to describe emotions. Emotional arousal describes the intensity of an emotional experience, ranging from calm to arousal, reflecting the physiological activation level of the emotion. Emotional valence describes the degree of pleasure or unpleasantness of an emotion, ranging from pleasant to unpleasant, reflecting the subjective tendency of the emotion. Compared to emotional valence, emotional arousal is more significantly correlated with physiological signals, especially during periods of intense emotional change. Changes in physiological signals such as electrodermal signals are highly sensitive and easily captured; therefore, monitoring emotional arousal on wearable devices is becoming increasingly common.

[0004] In related technologies, to obtain the correlation between emotional arousal and physiological signals, a data-driven research approach is typically used to decode emotional arousal. For example, pre-trained deep learning models are used to extract discriminative features from physiological signals, and then regression tasks are performed to predict emotional arousal based on these discriminative features. However, the deep learning models constructed in these methods are usually black-box models. The prediction process is based on a direct fitting of the statistical correlation between physiological signals and emotional arousal, making it difficult to accurately construct the mapping mechanism between physiological signals and emotional arousal, resulting in poor accuracy in emotional arousal decoding. Summary of the Invention

[0005] Some embodiments of this application provide a method and system for decoding emotional arousal to solve the problem of poor accuracy in decoding emotional arousal.

[0006] In a first aspect, some embodiments of this application provide a method for decoding emotional arousal, including: Collect skin electroretinogram signals under emotional stimulation; Preprocessing is performed on the electrodermal signals; The preprocessed electrodermal signal is processed by a sympathetic neural inversion network to obtain network inversion values ​​that characterize the level of sympathetic neural activation. The sympathetic neural inversion network calculates the derivative of the preprocessed electrodermal signal to obtain derivative information, constructs an input feature tensor based on the electrodermal signal and derivative information, and obtains the network inversion value after processing the input feature tensor through a convolution-long short-term memory hybrid architecture. The predicted value of emotional arousal is obtained by inverse mapping the network inversion value through the emotional arousal inverse mapping network. The emotional arousal inverse mapping network performs inverse mapping through an invertible function. The sympathetic nerve inversion network and the emotional arousal inverse mapping network constitute an end-to-end emotional arousal decoding model. The loss function of the emotional arousal decoding model includes a weighted sum of physiological loss and psychological loss. The physiological loss is used to calculate the difference between the network inversion value and the theoretical inversion value of the sympathetic nerve activation level. The theoretical inversion value is calculated based on the physiological dynamics equations on the preprocessed skin electroreception signal and derivative information. The psychological loss is used to calculate the difference between the predicted value and the labeled value.

[0007] In the above embodiments, an end-to-end emotional arousal decoding model, including a sympathetic inversion network and an emotional arousal inverse mapping network, is used to predict emotional arousal, achieving staged decoding of emotional arousal prediction. Theoretical inversion values ​​based on physiological prior knowledge are introduced as intermediate supervision signals, increasing the physiological interpretability of the model in the emotional arousal decoding process. At the same time, by jointly optimizing physiological and psychological losses, the model follows the physiological mechanism of sympathetic nerve-driven skin conductance response while fitting the statistical regularity of the data, which is beneficial to improving the prediction accuracy of emotional arousal.

[0008] In some embodiments, the method for obtaining derivative information using a sympathetic neural inversion network includes: Based on the set batch size, construct the temporal tensor corresponding to the preprocessed electrodermal signal; Calculate the first and second derivatives of the time series tensor; The boundaries of the first and second derivatives are processed by reflection filling, so that the processed first and second derivatives are consistent with the length of the temporal tensor. The derivative information includes the processed first and second derivatives.

[0009] In the above embodiments, by introducing a reflection filling strategy, the boundary distortion problem caused by traditional zero-filling or truncation methods is effectively avoided, ensuring the integrity and physiological rationality of derivative calculation in the temporal dimension, thereby improving the accuracy of sympathetic nerve activation level inversion.

[0010] In some embodiments, the physiological dynamics equations are based on the orifice valve model and are second-order differential equations that calculate the level of sympathetic nerve activation based on the dynamics of sweat diffusion and evaporation.

[0011] In the above embodiments, the physiological dynamic equation based on the orifice valve model can accurately describe the dynamic response process of sweat gland activity driven by the sympathetic nervous system, directly linking the generation mechanism of skin electroreaction with neurophysiological activity, thereby providing theoretical constraints that conform to biophysical laws for network inversion.

[0012] In some embodiments, the convolutional-long short-term memory hybrid architecture includes a convolutional neural subnetwork and a long short-term memory subnetwork connected in sequence. The convolutional neural subnetwork has a 3×1 one-dimensional convolutional kernel with a stride and padding of 1, and the activation function is a modified linear activation function. The first layer has 3 input channels to match the input feature tensor.

[0013] In the above embodiments, a one-dimensional convolutional kernel is used to extract local features from the input electrodermal signal and temporal derivative information, effectively capturing the subtle change patterns of the signal in the time dimension; the nonlinear expression capability is enhanced by modifying the linear activation function, while avoiding the gradient vanishing problem.

[0014] In some embodiments, the physiological dynamics equations include a rise time parameter characterizing the skin conductance response and a decay time parameter characterizing the skin conductance response, which are determined by the emotional arousal decoding model after pre-training.

[0015] In the above embodiments, the rise time and fall time parameters are optimized through a pre-training strategy to improve the accuracy of parameter estimation.

[0016] In some embodiments, the emotional arousal inverse mapping network includes an inverse operation unit, which inversely maps the network inversion value to the predicted value of emotional arousal through an exponential function with variable parameters. The variable parameters are determined by the emotional arousal decoding model after pre-training.

[0017] In the above embodiments, the variable parameters of the exponential function are optimized through a pre-training strategy to improve the accuracy of variable parameter estimation.

[0018] In some embodiments, during the pre-training of the emotional arousal decoding model, the input sample data is the sample data of the test object under emotional stimulation.

[0019] In the above embodiments, by collecting sample data of the test object for pre-training, the model's ability to adapt to the individualized test object is improved, which is conducive to improving the accuracy of model prediction.

[0020] In some embodiments, preprocessing of the electrodermal signal includes filtering, artifact removal, and segmentation of the electrodermal signal.

[0021] In the above embodiments, power frequency interference and motion artifacts are removed by filtering, and effective frequency band signals are retained; signal tailing effect is eliminated by artifact processing, and the clarity of temporal features is improved; segmentation processing divides continuous signals into equal-length windows, which facilitates subsequent batch input into the network model for training and inference, while ensuring that the dynamic continuity of physiological response is not destroyed.

[0022] Secondly, some embodiments of this application provide a decoding system for emotional arousal, the system comprising: The skin conductance signal acquisition module is used to acquire skin conductance signals; The signal preprocessing module is used to perform preprocessing of the electrodermal signals; The emotional arousal decoding module includes: The sympathetic neural inversion network is used to calculate the derivative of the preprocessed electrodermal signal to obtain derivative information. Based on the electrodermal signal and derivative information, an input feature tensor is constructed. After processing the input feature tensor through a convolution-long short-term memory hybrid architecture, the network inversion value representing the level of sympathetic neural activation is obtained. The emotional arousal inverse mapping network is used to perform inverse mapping on the network inversion values ​​through an invertible function to obtain the predicted value of emotional arousal. The sympathetic inversion network and the emotional arousal inverse mapping network constitute an end-to-end emotional arousal decoding model. The loss function of the emotional arousal decoding model includes a weighted sum of physiological loss and psychological loss. The physiological loss is used to calculate the difference between the network inversion value and the theoretical inversion value of the sympathetic activation level. The theoretical inversion value is calculated based on the physiological dynamics equations on the preprocessed skin electroretinopathy signal and derivative information. The psychological loss is used to calculate the difference between the predicted value and the labeled value.

[0023] In the above embodiments, the end-to-end emotional arousal decoding model is constructed as a two-stage interpretable architecture including a sympathetic neural inversion network and an emotional arousal inverse mapping network. Theoretical inversion values ​​based on physiological prior knowledge are introduced as intermediate supervision signals, which increases the physiological interpretability of the model in the emotional arousal decoding process. At the same time, by jointly optimizing physiological loss and psychological loss, the model can follow the physiological mechanism of sympathetic nerve-driven skin conductance response while fitting the statistical regularity of data, which is conducive to improving the prediction accuracy of emotional arousal.

[0024] In some embodiments, the sympathetic inversion network includes: The input feature construction subnetwork includes a temporal tensor construction unit, a derivative calculation unit, and a tensor splicing unit. The temporal tensor construction unit is used to construct the temporal tensor corresponding to the preprocessed ESC signal according to the set batch size. The derivative calculation unit is used to calculate the first and second derivatives of the temporal tensor, and to process the boundaries of the first and second derivatives by reflection filling, so that the processed first and second derivatives are consistent with the length of the temporal tensor, thus obtaining derivative information including the processed first and second derivatives. The tensor splicing unit is used to splice the temporal tensor and derivative information along the feature dimension to form a three-dimensional input feature tensor. The convolutional neural subnetwork is used to extract features from the input feature tensor to obtain the first feature vector; A long short-term memory subnetwork is used to capture the temporal dependencies between the first feature vectors to obtain the second feature vector; A fully connected layer is used to map the second feature vector into a network inversion value representing the level of sympathetic neural activation.

[0025] In the above embodiments, the input feature construction subnetwork concatenates the electrodermal signal and its derivative information, expanding the input dimension. This facilitates the optimization of the convolutional neural subnetwork towards extracting key features related to sympathetic nerve activity, enhancing the model's sensitivity to dynamic changes in physiological signals. Simultaneously, combining the long short-term memory subnetwork's ability to model temporal dependencies further improves the accuracy and stability of the network inversion values, thereby achieving more physiologically consistent and interpretable decoding within the two-stage architecture. By mapping the second feature vector to the network inversion value of the sympathetic nerve activation level through a fully connected layer, a nonlinear approximation from multi-scale physiological features to the state of autonomic nerve activity is achieved, ensuring the physiological interpretability of the network inversion values. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in some embodiments of this application or in the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A schematic diagram illustrating changes in physiological signals caused by emotional stimulation, provided in some embodiments of this application; Figure 2 A schematic diagram illustrating the reverse modeling process provided for some embodiments of this application; Figure 3 A partial modeling diagram of the emotional arousal decoding model provided in some embodiments of this application; Figure 4 This is another partial modeling schematic diagram of the emotional arousal decoding model provided in some embodiments of this application; Figure 5 A flowchart illustrating a method for decoding emotional arousal provided in some embodiments of this application; Figure 6 A flowchart illustrating a method for decoding emotional arousal provided in other embodiments of this application; Figure 7 This is a schematic diagram of the structure of an emotion arousal decoding system provided in some embodiments of this application; Figure 8 This is a schematic diagram of the structure of the sympathetic inversion network provided in some embodiments of this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of some embodiments of this application clearer, the technical solutions of some embodiments of this application will be clearly and completely described below with reference to specific embodiments and corresponding drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0029] It should be noted that the brief descriptions of terms in some embodiments of this application are only for the convenience of understanding the implementation methods described below, and are not intended to limit the implementation methods of some embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0030] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0031] In the field of affective computing, emotional arousal is a core indicator of emotion, and accurate decoding of it is crucial for achieving intelligent human-computer interaction, psychological state monitoring, and personalized health services. For example, real-time monitoring of emotional arousal in wearable devices can provide data support for stress management, fatigue warnings, and mental health interventions, enhancing the user experience and practicality of wearable devices.

[0032] It should be noted that this application uses wearable devices as the application carrier for decoding emotional arousal, but the proposed method and system can also be applied to other scenarios. For example, in medical and mental health monitoring, by continuously sensing changes in a patient's emotional arousal, doctors can be assisted in assessing stress, anxiety, and psychological reactions during treatment; in the field of automotive driving safety, the system can identify risk states such as fatigue, distraction, or emotional agitation based on the driver's emotional arousal, thereby triggering safety alerts; in virtual reality and interactive systems, emotional arousal can be used to dynamically adjust stimulus intensity or interaction strategies, improving user immersion and experience stability; furthermore, in smart home or office scenarios, the system can automatically adjust environmental parameters or perform stress monitoring based on an individual's emotional arousal, achieving more intelligent human-machine collaboration and health management. The above scenarios all demonstrate the usability and necessity of emotional arousal decoding in multiple fields.

[0033] To improve the accuracy of emotional arousal decoding, this disclosure proposes a method and system for decoding emotional arousal based on the correlation analysis between emotional arousal and physiological signals in relevant studies. By modeling the intrinsic mechanism by which physiological signals cause changes in emotional arousal, it provides a physiological basis for the prediction of emotional arousal and solves the problem of poor prediction accuracy of traditional black box models.

[0034] In some embodiments of this application, the physiological signal collected for decoding emotional arousal is the electrodermal signal. Other types of physiological signals, such as near-infrared signals, can also be collected to decode emotional arousal, referring to the technical ideas of the embodiments of this application.

[0035] Figure 1 This is a schematic diagram illustrating changes in physiological signals caused by emotional stimuli, as provided in some embodiments of this application. Figure 1 As shown, when a user wearing a wearable device that measures electroskin activity is subjected to emotional stimulation, such as playing a selected movie clip, the user experiences changes in emotional arousal. According to relevant research, there is a positive correlation between these changes in emotional arousal and changes in the sympathetic nervous system; higher emotional arousal corresponds to stronger sympathetic nerve activity, and vice versa. Electroskin activity is one of the external physiological manifestations of sympathetic nerve activity, and analyzing sympathetic nerve activity through electroskin activity has a physiological basis: postganglionic sympathetic fibers extend to the subepithelial layer of the skin along blood vessels, hair follicles, and other structures, where they further branch and extend into the more superficial dermis; changes in sympathetic nerve activity under emotional stimulation trigger the eccrine sweat glands in the dermis to secrete sweat, which accumulates within the ducts of these glands; when the accumulated hydraulic pressure within the ducts exceeds a threshold, the sweat diffuses and evaporates in the epithelial layer, leading to changes in skin conductivity. The above physiological mechanism reveals a positive process of conversion from psychological variables to physiological signals: changes in emotional arousal - changes in sympathetic nerve activity - changes in electrodermal signals, where emotional arousal is a psychological variable and electrodermal signals are a physiological signal.

[0036] Based on the above analysis, this application embodiment performs reverse modeling of the aforementioned forward process, that is, starting from the collected electrodermal signals, it reverse-engineers the change process of emotional arousal. By establishing a mathematical model driven by physiological mechanisms, the dynamic characteristics of electrodermal signals are mapped to the level of sympathetic nerve activity, and the corresponding emotional arousal state is further deduced. This reverse modeling method enhances the interpretability of the prediction results, improves the accuracy of emotional arousal decoding, and enables the system to more realistically reflect the fluctuation patterns of psychological states.

[0037] Figure 2 This is a schematic diagram illustrating the reverse modeling process provided for some embodiments of this application. For example... Figure 2 As shown, the method may include the following steps: Step S110: Collect sample data.

[0038] In some embodiments, this step may include the following steps: Step S1101: Screening subjects.

[0039] Because individuals of different ages, genders, and health conditions exhibit significant differences in physiological responses, the generalization ability of emotional arousal decoding models faces a severe challenge. Therefore, representative subject groups can be selected based on the target application scenario. For example, subjects can be divided into youth and elderly groups to enhance the model's applicability to specific populations. Simultaneously, gender differences should be considered, with male and female subjects included in a certain proportion. All subjects are healthy individuals without neurological or skin diseases to avoid excluding abnormal physiological signals caused by pathological factors, ensuring data quality and modeling reliability.

[0040] In some embodiments, considering that even among subjects of the same age and gender, there may still be significant differences in the positive process from emotional arousal to electrodermal signal transmission, different individuals may exhibit varying degrees of variation. To avoid data leakage and distribution bias among different subjects, independent modeling can be performed on the data of each subject. This personalized model can effectively verify the accuracy of emotional arousal decoding. Subsequently, when applying the model, for the test subject, the emotional arousal decoding model can be pre-trained specifically for that subject, allowing for fine-tuning of the model. After personalized pre-training is complete, the emotional arousal decoding model corresponding to that subject is obtained, enabling accurate monitoring of emotional arousal in that subject.

[0041] In some embodiments, to facilitate user use, an emotion arousal decoding model with a certain generalization ability can be trained for subjects of the same age and gender. The test subjects can quickly monitor emotion arousal by selecting the emotion arousal decoding model that matches their age and gender without personalized pre-training.

[0042] Step S1102: Collect skin electrical signals.

[0043] Electrodermal (ED) signals can be acquired using specific devices, such as a dedicated EED activity acquisition system. This system includes an EED sensor, which can be worn on areas with dense sweat gland distribution, such as the fingers or palms, to ensure the sensitivity and stability of signal acquisition. During the acquisition process, it is necessary to maintain a constant ambient temperature and humidity to avoid external interference affecting the EED response.

[0044] Simultaneously, to obtain labeled data, namely the emotional arousal rating, participants can be provided with interactive devices such as tablets or button-operated devices to synchronously record their subjective ratings of emotional arousal under specific emotional stimuli. These interactive devices need to be able to record the rating data input by the participants and its corresponding timestamps, ensuring precise alignment between physiological signals and subjective emotional labels in the time dimension. Ratings can be conducted using internationally recognized emotion scales, such as the Self-Assessment Manikin (SAM), allowing participants to rate their current emotional arousal level on a scale of 1 to 9.

[0045] In some embodiments, emotional stimuli can be triggered by standardized emotional evoked materials, such as pictures, emotional video clips, or sound stimuli from the International Affective Picture System (IAPS), to ensure the reliability and repeatability of emotional evokedness.

[0046] In some embodiments, emotional stimuli can also be induced through custom videos, real-time interactive tasks, and virtual reality scenes to enhance the diversity of test scenarios and make the triggering of emotional stimuli more closely resemble real-life situations.

[0047] The sampling frequency of the electrodermal signal can be set to 1000 Hz to ensure that subtle changes in the electrodermal response are captured. A total of 10 minutes of electrodermal signals were collected from the participants under emotional stimulation, along with simultaneously recorded subjective emotional arousal scores. The reciprocal of the sampling frequency was the time step t.

[0048] Electrodermal signals are , . This represents the skin conductance value obtained at the t-th sampling time, i.e., the t-th time step; The dimension is T×1, representing It is a one-dimensional time series composed of skin conductance values ​​obtained at T sampling times.

[0049] For each subject, training, validation and test sets are divided according to their own electrodermal signals and labeled data to ensure that there is no information leakage between subjects during model training and evaluation.

[0050] Step S120: Data preprocessing.

[0051] In some embodiments, data preprocessing of electrodermal signals may include the following steps: Step S1201: Filtering.

[0052] When performing preprocessing on the electrodermal signal, the range of human electrodermal signals can first be obtained from relevant medical data. Based on this range, a filtering threshold is determined, and then a filtering interval is obtained according to the filtering threshold. After that, the electrodermal signal is filtered according to the determined filtering interval to obtain the electrodermal signal within the filtering interval.

[0053] Step S1202: Artifact removal.

[0054] Artifact removal filters are used to remove artifact signals from the target frequency band of the electrodermal signal to obtain an effective electrodermal signal.

[0055] Step S1203: Segmentation processing.

[0056] The effective EEG signal is divided into data segments using a pre-defined, non-overlapping time window to generate a preprocessed EEG signal. This preprocessed EEG signal comprises multiple data segments, each corresponding to EEG signals across multiple time steps. During model training, a preprocessed EEG signal from one data segment serves as a sample in the EEG signal dimension; during model application, a preprocessed EEG signal from one data segment serves as input data in the EEG signal dimension.

[0057] For example, based on the range of the electrodermal signal (EDS) being 0.05~3Hz, the upper limit of the filtering threshold can be set to 3Hz, and the lower limit to 0.05Hz, thus defining the filtering range as 0.05~3Hz. During preprocessing, the EDS signal can first be filtered using a low-pass filter with a cutoff frequency of 3Hz, followed by smoothing using a 4th-order Butterworth filter. This removes high-frequency noise from the EDS signal, leaving only the main EDS frequency band, i.e., the EDS data within 3Hz. Then, a high-pass filter with a cutoff frequency of 0.05Hz is used to remove low-frequency drift and other artifacts, thereby achieving phase and tone separation of the signal. Finally, the EDS signal collected from the subject is segmented into segments with a fixed length of 15s, and normalization parameters are calculated based on the training set after segmentation to prevent data leakage. Through these steps, the preprocessed EDS signal is obtained.

[0058] Step S130: Construct an emotional arousal decoding model.

[0059] Based on the above analysis, the emotional arousal decoding model can include a network for predicting the level of sympathetic nerve activation and a network for predicting emotional arousal. The former can be called the sympathetic nerve inversion network, which inverts sympathetic nerve activity through electrodermatology signals to obtain network inversion values ​​representing the level of sympathetic nerve activation. The latter can be called the emotional arousal inverse mapping network, which performs inverse mapping on the network inversion values ​​through an invertible function to obtain the predicted values ​​representing emotional arousal.

[0060] Based on the above analysis, step S130 may include the following steps: Step S1301: Construct a sympathetic inversion network.

[0061] The sympathetic neural inversion network is used to invert sympathetic neural activity from electrodermal signals. Since the quantitative index of sympathetic neural activity, namely the sympathetic neural activation level, is a continuously changing one-dimensional physiological index, the sympathetic neural inversion network is designed as a regression model. The main network architecture is a Convolutional Neural Network–Long Short-Term Memory (CNN-LSTM) hybrid architecture, in which CNN is used to extract local temporal features in electrodermal signals, and LSTM is used to capture the dynamic temporal dependencies of the signal.

[0062] In some embodiments, to improve the consistency between the network inversion value output by the sympathetic inversion network and the actual sympathetic activation level, i.e. the physiological sympathetic activation level, the prediction accuracy and interpretability of the sympathetic inversion network are improved by addressing both the network input and network loss of the sympathetic inversion network.

[0063] Regarding network input, the level of sympathetic nerve activation, based on physiological principles, can be estimated using physiological dynamics equations. The estimated value is the theoretical inversion value of the sympathetic nerve activation level. The parameters of the skin electrodermal signal required to calculate the sympathetic nerve activation level using the physiological dynamics equations are all used as the data source for the input feature vector of the CNN network, to ensure that the network input contains data with the same feature dimensions as the physiological mechanism.

[0064] For example, the physiological dynamic equation for calculating sympathetic nerve activation levels includes a second-order differential equation based on the orifice valve model and the dynamic process of sweat diffusion and evaporation, as shown in the following formula: (1) In the above formula, Indicates skin electrical signals, for The first derivative, for The second derivative of .

[0065] This is the theoretical inversion value of the sympathetic nerve activation level; It is a rise time parameter that characterizes the skin's electrical response and is used to describe the rise time of electrical activity (EDA). It is a decay time parameter characterizing the skin's electrical response, used to describe the decline time of skin electrical activity.

[0066] Both the rise time parameter and the fall time parameter are time constants that vary from person to person. The initial value can be set to 0.7, in seconds. The initial value can be set to 2.0, in seconds. and The value of is continuously adjusted during the pre-training process of the emotional arousal decoding model and is determined after the pre-training is completed.

[0067] According to equation (1), the relevant parameters of the skin electrical signal required for calculating the sympathetic nerve activation level using this physiological dynamics equation include the skin conductance value, as well as the first and second derivatives of the skin conductance value. Therefore, it is possible to base the calculation on... , and An input feature tensor is constructed to enhance the correspondence between network input and physiological mechanisms. By jointly inputting skin conductance values ​​and their dynamic changes, the CNN-LSTM hybrid architecture can more accurately capture sweat gland response patterns driven by the sympathetic nervous system, thereby improving the continuity and physiological plausibility of the network's retrieved values ​​in the temporal dimension.

[0068] To facilitate feature extraction from input data using the CNN-LSTM hybrid architecture, a sub-network can be constructed to build upon the input features for computation. and And use this to construct the input feature tensor.

[0069] The input feature construction subnetwork may include temporal tensor construction units, derivative calculation units, and tensor splicing units.

[0070] The temporal tensor construction unit is used to construct a temporal tensor corresponding to the preprocessed EEG signals based on the set batch size. One temporal tensor corresponds to one batch of preprocessed EEG signals, including data from multiple data segments.

[0071] The derivative calculation unit is used to calculate the first and second derivatives of the time series tensor, and to process the calculated first and second derivatives.

[0072] In practice, for the preprocessed electrodermal signals, with The data format is input into the derivative calculation unit for processing, that is... Let be the temporal tensor corresponding to the preprocessed electrodermal signal, where This corresponds to the set batch size, which is the number of samples in a batch. This corresponds to each data segment, that is, the number of time steps in each sample, representing the time series length. The data in a single batch is y. This indicates that there are n preprocessed electrodermal signals within a single batch, where n = .

[0073] The first and second derivatives are calculated through one-dimensional convolution operations, that is, convolutional differentiation is performed using a specified convolution kernel.

[0074] First derivative Using convolution kernels : (2) Second derivative Using convolution kernels : (3) Furthermore, the derivative calculation unit also employs reflection filling to handle the boundaries between the first and second derivatives, ensuring that the convolution output length is consistent with the input signal. The first and second derivatives after processing are identical to the length of the time series tensor, thus ensuring the continuity and stability of the time series data at the boundary.

[0075] The tensor splicing unit then takes the input signal The processed first and second derivatives are concatenated along the feature dimension to form a three-dimensional input feature tensor. Its time step is related to the input signal. Consistency is maintained, thus preserving the complete temporal structure. This construction method not only enhances the ability of the CNN-LSTM hybrid architecture to perceive the dynamic evolution of sympathetic neural activity, but also enables the network to learn the physiological lag and recovery process of sweat secretion from multi-order differential features, further improving the prediction accuracy and physiological interpretability of the sympathetic neural inversion network.

[0076] The CNN-LSTM hybrid architecture is used to process the input feature tensor to obtain network inversion values ​​representing the level of sympathetic neural activation. .

[0077] See Figure 3 The image shown is a partial modeling diagram of the emotional arousal decoding model provided in some embodiments of this application. Figure 3As shown, the CNN-LSTM hybrid architecture consists of sequentially connected CNN and LSTM sub-networks. The CNN sub-network includes multiple convolutional layers, such as three convolutional layers. These layers enable the CNN sub-network to apply convolutional matrices to specific parts of the input feature vector. The first convolutional layer has 3 input channels to match the three-dimensional input feature tensor and 64 output channels, balancing network efficiency and accuracy. The second convolutional layer has 64 input channels and 128 output channels, and the third convolutional layer has 128 input channels and 256 output channels.

[0078] Figure 3 The time steps involved are from the 1st to the 2nd. A time step, that is Figure 3 This describes the processing procedure for a single sample within a batch. A batch contains B samples. Figure 3 For the first The process of processing each sample The value range is 1 to B.

[0079] In CNN, for the first in a batch Sample , Its corresponding first The characteristics of each output channel are: (4) in, ; , Indicates the input channel. for The Middle Data from each input channel For CNN in the Learnable parameters for each output channel; the CNN uses a 3×1 one-dimensional convolutional kernel, with stride and padding both set to 1, and the time step index remains constant throughout the training process. Kernel index Corresponding original kernel weights , The third subscript refers to the convolutional layer, for example, This refers to the first convolutional layer at the 1st... The first input channel, the first The kernel weights of the convolution kernels for each output channel. The size can be set to an initial value according to actual needs, and can be automatically updated during the model training phase.

[0080] Figure 3 middle, Generally, this refers to the kernel weights of the convolutional kernel in the first input channel of the first convolutional layer. Generally, this refers to the kernel weights of the convolutional kernel in the second input channel of the first convolutional layer. Generally refers to the kernel weights of the convolutional kernel in the third input channel of the first convolutional layer. For CNNs, this refers to the kernel weights of the first convolutional layer in a batch. For each sample, the last convolutional layer outputs a 256-dimensional feature vector. .

[0081] Feature vector The input sequence is fed into the LSTM subnetwork, i.e., the Long Short-Term Memory subnetwork.

[0082] For feature vectors The Long Short-Term Memory (LSTM) subnetwork is processed through the core information processing unit. to The iterative computation captures the temporal dependencies between features, ultimately establishing the input sequence. With output sequence The mapping relationship is established. This core information processing unit includes three gates (forget gate, input gate, and output gate), a candidate cell state generation module, and a cell state update module. Combined with a fully connected layer, it collectively achieves the conversion of the input sequence into an output sequence. The core computations are as follows: (5) Among them, feature vector Total corresponding Taking one of the time steps as the current time step as an example, similar to formula (4), the current time step is taken as the first time step. Taking a time step as an example This represents the input feature vector at the current time step, which includes features from all output channels of the CNN network at that time step. For time step index, This indicates that the number of time step indices in a sample is [number missing]. indivual.

[0083] This indicates the hidden state at the current time step. This represents the hidden state at the previous time step. For ease of description, It can be called the first eigenvector. This can be called the second feature vector. The second feature vector is used to capture the temporal dependency between the first feature vectors and is mapped to a network inversion value representing the level of sympathetic nerve activation through a fully connected layer. hour, , It can be set as a zero vector. This indicates that the input feature vector of the current time step and the hidden state of the previous time step are concatenated to obtain a concatenated vector. This concatenated vector contains the skin electrophysiological fusion features of the current time step (the physiological dynamic information of the fused skin electrophysiological signal raw value, first derivative, and second derivative) and the temporal information of the sympathetic nerve activation level of the previous time step (composed of...). (bearer), providing the core input for LSTM to capture temporal dependencies across time steps.

[0084] The candidate cell state is a candidate value for new information at the current time step. This candidate value carries temporal information introduced by the concatenation vector at the current time step for calculating the sympathetic nerve activation level. The candidate cell state generation module is used to calculate... ; This represents the cell state at the current time step. The cell state is a comprehensive information that integrates temporal information about historical sympathetic nerve activation levels with new candidate values. The cell state update module is used to calculate the cell state at the previous time step, i.e., the historical cell state. . hour, , It can be set as a zero vector. This represents element-wise multiplication. This is the Sigmoid activation function.

[0085] This is the forget gate, used to control the state of historical cells. The retention ratio; This is an input gate used to regulate the state of candidate cells. The injection intensity; This is an output gate used to determine the hidden state. The output ratio; the LSTM is connected to the fully connected layer at the 1st... Output at each time step That is This is mapped to network inversion values ​​representing the level of sympathetic nerve activation. Weight parameters and bias vector For learnable parameters, where the subscript is... Corresponding to the forgetting gate, subscript Corresponding input gate, subscript Corresponding to the candidate cell state, subscript Corresponding output gate, subscript Corresponding to the fully connected layer.

[0086] Still using the current time step as the first step Taking a time step as an example Figure 3 In China, with For example, The first index 1 indicates that it is the first hidden layer; similarly, The first subscript 2 indicates that it is the second hidden layer. The first subscript 3 indicates that it is the third hidden layer. The above calculations of forget gate, input gate and output gate are performed in each hidden layer. The second subscript indicates the time step index; ~ The meaning of is deduced by analogy. Indicates that LSTM is in the first... The output of each time step The meaning of is deduced by analogy.

[0087] The output sequence of the fully connected layer is , , This is the predicted temporal sequence of sympathetic neural activity, i.e., the network inversion value. for The Middle The value at each time step.

[0088] Step S1302: Construct an inverse mapping network for emotional arousal.

[0089] The emotional arousal inverse mapping network is used to achieve the inverse mapping from the temporal sequence of sympathetic neural activity to emotional arousal. Since there are various mapping functions and their corresponding inverse mapping functions in mathematics, it is necessary to select a suitable inverse mapping function to ensure the accuracy and interpretability of the conversion from sympathetic neural activity sequence to emotional arousal.

[0090] See Figure 4 This is another partial modeling diagram of the emotional arousal decoding model provided in some embodiments of this application, such as... Figure 4 As shown, during the modeling phase, multiple inverse mapping subnetworks can be constructed. The inputs of these subnetworks are connected in parallel to the output of the sympathetic neural inversion network. Each subnetwork corresponds to a candidate inverse mapping function form, thus obtaining multiple candidate inverse mapping results. During the model training phase, based on the difference between the candidate inverse mapping results and the label values, only the inverse mapping subnetwork corresponding to the optimal candidate inverse mapping function form is retained, and the network parameters of that inverse mapping subnetwork are updated. Other inverse mapping subnetworks are deleted, so that during the model application phase, only one inverse mapping subnetwork is used to output the inverse mapping result.

[0091] In some embodiments, during the inverse modeling process, a forward function describing the forward process from change in emotional arousal to change in sympathetic neural activity is established, in the following form: (6) Among them, the aforementioned positive functions all describe the mapping relationship from changes in emotional arousal to changes in sympathetic nervous activity. It is a representation of the level of sympathetic nerve activation. It is a representation of emotional arousal, with a value of The Middle The value at each time step. It is a predictive value for emotional arousal.

[0092] The aforementioned positive functions encompass nonlinear functions such as linear, exponential, and logarithmic functions. These functions all describe positive correlations, consistent with the positive correlation between emotional arousal and sympathetic nervous activity. This indicates the degree to which emotional arousal influences sympathetic nervous system activity under the current mapping function. This represents the baseline offset value indicating the degree to which emotional arousal affects sympathetic nerve activity under the current mapping function. , All are variable parameters. , The value is updated during the pre-training process of the emotional arousal decoding model and is determined after pre-training is complete. Taking the current batch of data as an example, under the ln function... The mean is 0.7127. The mean value is 0.4042, which means that in the current mapping relationship, the sensitivity coefficient of emotional arousal to sympathetic nerve activity is about 0.7, and the basic offset for translating sympathetic nerve activity is about 0.4.

[0093] The definition of emotional arousal changes triggering changes in sympathetic nervous system activity, which in turn leads to... The forward process of change can be represented by a nonlinear invertible function. To describe: (7) Based on the above definition, the expression for sympathetic nerve activity in the forward process is defined as formula (7) above. Due to the properties of invertible functions, it can be obtained through the following equation. : (8) For the above multiple forward functions, calculate the inverse function respectively, as shown in the following formula: (9) In this embodiment, the inverse function is short for inverse mapping function. As can be seen from the above modeling process, the emotional arousal inverse mapping network can inversely map sympathetic nerve activity to emotional arousal through the inverse function. Therefore, for the network inversion value output by the sympathetic nerve inversion network, which represents the level of sympathetic nerve activation, By mapping the predicted value of emotional arousal through an inverse function, the joint modeling of psychological and physiological processes is achieved, enabling end-to-end interpretable quantitative modeling.

[0094] Step S1303: Joint modeling.

[0095] The loss function is used to adjust the network parameters of the emotional arousal decoding model to improve the prediction accuracy and interpretability of the trained emotional arousal decoding model.

[0096] The emotion arousal decoding model mainly includes a sympathetic neural inversion network and an emotion arousal inverse mapping network. It is an end-to-end model. The loss function can be divided into two parts: one part is used for parameter optimization of the sympathetic neural inversion network, and the other part is used for parameter learning of the emotion arousal inverse mapping network.

[0097] For the loss function in the first part, it can be used This is represented as physiological loss, used to calculate the network inversion value of sympathetic nerve activation levels. Compared with theoretical inversion values The difference, among which, theoretical inversion value It is calculated based on the preprocessed electrodermal signal and derivative information using physiological dynamics equations, for example, based on the physiological dynamics equation in formula (1). Physiological loss can be calculated using the mean squared error loss, as shown in the following formula: (10) For the loss function in the second part, it can be used This is represented as psychological loss, used to calculate the predicted value. With tag value Differences, predicted values With tag value All of them can be written in chronological order as a sequence: , . Indicates the first The psychological variable values ​​predicted at each time step This indicates that the data is concentrated in the first... The time steps provide the true labeled psychological variable values, and both sequences have a dimension of T×1, i.e., a one-dimensional time series composed of T time points. Psychological loss can be calculated using the mean squared error loss formula, as shown below: (11) Combining the two losses mentioned above, the loss function of the emotion arousal decoding model can be set as follows: (12) In the above formula, The loss function for the emotional arousal decoding model. , This is a hyperparameter used to balance the two parts of the loss. The initial values ​​can be set to 0.3 and 0.7 respectively. This hyperparameter can be dynamically adjusted according to the convergence during the training process to ensure that the physiological loss and psychological loss maintain a reasonable weight during the training process.

[0098] The aforementioned loss function constrains the joint optimization process of the sympathetic neural inversion network and the emotional arousal inverse mapping network, enabling the model to improve the accuracy of psychological variable predictions while preserving physiological interpretability. During backpropagation, gradients are applied to both sub-networks, ensuring that the physiological mechanism of skin conductance response and the perceptual changes in emotional arousal are synchronously fitted. The final predicted sequence of emotional arousal values ​​output by the model not only conforms to the dynamics of the human autonomic nervous system but also achieves high correlation with time-aligned labeled data, realizing accurate decoding from physiological signals to subjective emotions.

[0099] After constructing the emotional arousal decoding model, the input feature tensor of each subject obtained in step S130 is input into the emotional arousal decoding model for training, resulting in multiple training results. These multiple training results correspond to multiple inverse mapping subnetworks, from which the inverse mapping subnetwork with the best performance is determined. For the next subject, the same process is repeated, inputting the subject's input feature tensor into the emotional arousal decoding model for training, resulting in multiple training results, from which the inverse mapping subnetwork with the best performance is determined. This process continues until all input feature tensors of multiple subjects have been processed, finally obtaining the optimal inverse mapping subnetwork for each subject.

[0100] To ensure the continuity of the time series and the consistency of the labels, the electrodermal signal was divided into multiple fixed-length time segments based on video IDs. Subsequently, all segmented samples were globally randomized, while maintaining the one-to-one correspondence between electrodermal activity, emotional arousal, and video IDs. Modeling and evaluation were performed separately on each participant's randomized data segment to explore the performance differences of different mapping functions in the emotional arousal modeling task. During each training and testing phase, each participant's own physiological signals and labeled data were independently divided into training, validation, and test sets to ensure no information leakage between participants during model training and evaluation. During training, the Adam optimizer (Adaptive Moment estimation with Weight decay, AdamW) was used to update the parameters in the neural network model. The initial learning rate was 1e-3, and weight decay was set to enhance generalization ability. During this process, after training and testing each participant's data, the Mean Squared Error (MSE) and Mean Absolute Error (MAE) on the test set were calculated. For the results of 30 independent experiments, the within-group mean and standard deviation of MSE and MAE were further calculated and used as an overall performance evaluation of the current neural network model under a specific mapping function.

[0101] Data analysis shows that the decoding performance indicators of the technical solution proposed in this application achieved the lowest mean square error (MSE) of 0.208 and mean absolute error (MAE) of 0.339. Specifically, the logarithmic function... The corresponding inverse function, i.e., the exponential function The inverse mapping form performs optimally. The inverse function of the logarithmic function, i.e., the exponential function, is used. The emotional arousal decoding model achieved the lowest mean squared error (MSE) of 0.208 ± 0.078 and mean absolute error (MAE) of 0.339 ± 0.084, significantly outperforming other nonlinear mapping forms such as linear, quadratic, cubic, and tangent functions; while the inverse function corresponding to the exponential function... The performance deteriorated significantly due to large fluctuations in error caused by numerical instability. This result indicates that a reasonable design of the inverse mapping function is crucial for the model to fit the complex relationship between physiological signals, sympathetic neural activity, and emotional arousal. Therefore, the neural network model with optimized parameters provides a more accurate decoding result for emotional arousal, thus improving the accuracy of the emotional arousal decoding.

[0102] Based on the above training results, except for the exponential function The inverse mapping subnetworks corresponding to the inverse mapping functions other than those specified are removed, and only the exponential function is retained. The optimal model structure under the mapping is finally obtained as an emotion arousal decoding model, which includes a sympathetic neural inversion network and an emotion arousal inverse mapping network. At this time, the emotion arousal inverse mapping network includes an inverse operation unit. The inverse operation unit inversely maps the network inversion value to the predicted value of emotion arousal through an exponential function with variable parameters. The variable parameters are determined by the emotion arousal decoding model after pre-training.

[0103] As can be seen from the above reverse modeling process, the emotional arousal decoding model constructed in this application embodiment is a neural network model based on physiological-physical-psychological information. Physiological-physical-psychological information includes physiological information, physical information, and psychological information. Physiological information includes electrodermal signals, physical information includes the physiological dynamics equation of electrodermal activity, and psychological information includes emotional arousal. In the emotional arousal decoding model, by embedding the physiological dynamics equation of electrodermal activity into the CNN-LSTM hybrid architecture as a constraint at the physical information level—that is, a constraint of physiological loss—the CNN-LSTM hybrid architecture can follow the physiological mechanism of sympathetic nerve-driven electrodermal response while fitting the statistical regularity of the data. This improves the accuracy and interpretability of sympathetic nerve activation level prediction. This constraint of physiological loss, combined with the constraint of psychological loss between the emotional arousal label value and the predicted value, jointly ensures the accuracy of psychological variable prediction.

[0104] Based on the above reverse modeling process, an emotional arousal decoding model is obtained. This model can be used to decode the emotional arousal of the test subjects. The test subjects are users in the application phase of the emotional arousal decoding model, and are usually not subjects in the training phase of the model.

[0105] Figure 5 A flowchart illustrating a method for decoding emotional arousal provided in some embodiments of this application includes the following steps: Step S210: Collect skin electroretinopathy signals under emotional stimulation.

[0106] In the application phase, emotional stimuli are induced by emotional events or external stimuli actually experienced by the test subject. Depending on the application scenario, the method for collecting electrodermal signals can be the same as in the training phase, such as by using a dedicated electrodermal activity acquisition system, or by using other devices, such as wearable devices, as long as the electrodermal signals of the same modality as in the training phase can be obtained.

[0107] Step S220: Perform preprocessing on the electrodermal signal.

[0108] In the application phase, the preprocessing steps can also include filtering, artifact removal, and segmentation to remove noise interference from the signal.

[0109] Step S230: Process the preprocessed electrodermal signal using a sympathetic inversion network to obtain network inversion values ​​that characterize the level of sympathetic nerve activation.

[0110] In the application phase, the preprocessed electrodermal signal is input into the trained emotional arousal decoding model. The input feature generation unit in the emotional arousal decoding model processes the electrodermal signal to obtain an input feature tensor. This input feature tensor is then input into the sympathetic neural inversion network, and through forward propagation, a network inversion value representing the sympathetic neural activation level is calculated. This inversion value serves as the input feature for the next stage.

[0111] Step S240: Perform inverse mapping on the network inversion value through the emotional arousal inverse mapping network to obtain the predicted value of emotional arousal.

[0112] In the application phase, the sympathetic neural inversion network in the emotional arousal decoding model inputs the network inversion value into the emotional arousal inverse mapping network. Through forward propagation, a predicted value of emotional arousal is obtained. This predicted value is a continuous real number, representing an individual's emotional arousal level at a specific moment. This predicted value can be output in real time for dynamically monitoring emotional change trends without requiring additional physiological signals, demonstrating good independence and practicality. By continuously inputting electrodermal signals from different time points, the model can generate a time series of emotional arousal, thus providing a reliable basis for applications such as affective computing, psychological assessment, and human-computer interaction.

[0113] It should be noted that for the test subject, a general emotional arousal decoding model that matches their age and gender can be used to predict emotional arousal, thus eliminating the need for debugging and enabling real-time decoding of emotional states quickly.

[0114] Furthermore, to improve prediction accuracy, the model can be fine-tuned beforehand for the target individual. The adjusted model is then used to predict the emotional arousal level of the target individual, adapting to individual physiological differences and improving decoding accuracy. The fine-tuning process can be completed using a small amount of labeled data for that individual, optimizing network parameters to make the model output more closely resemble the actual emotional state. This method balances versatility and personalization, improving prediction accuracy while ensuring deployment efficiency, and is suitable for emotion monitoring applications in various scenarios.

[0115] Figure 6 A flowchart illustrating a method for decoding emotional arousal provided in other embodiments of this application includes the following steps: Step S310: Train the emotional arousal decoding model using sample data from the test subjects.

[0116] To improve prediction accuracy, audiovisual stimuli related to emotional arousal can be played for the test subjects. These stimuli can be a combination of short clips containing various emotional intensities. To shorten training time, the total playback duration is controlled within 30 seconds to 1 minute. Simultaneously, their electrodermal signals and corresponding emotion annotation data are collected. The emotional arousal decoding model is then trained in a supervised manner based on these signals, optimizing network parameters to minimize the error between predicted values ​​and labeled values. This pre-training strategy optimizes the variable parameters in the emotional arousal decoding model, improving the accuracy of variable parameter estimation and thus enhancing the adaptability of the variable parameters to the test subjects.

[0117] Step S320: Collect skin electrical signals under emotional stimulation.

[0118] After training is complete, the network parameters of the emotional arousal decoding model corresponding to the test subject are saved. This allows the model parameters to be directly loaded when the test subject uses the model in the future, enabling real-time decoding of personalized emotional arousal.

[0119] The wearable device is configured to monitor the subject's emotional arousal in real time, allowing for real-time monitoring of changes in emotional arousal. When the subject comes into contact with external stimuli, the electrodermal signal is collected in real time and input into the pre-trained emotional arousal decoding model.

[0120] Alternatively, the wearable device can be set to manually trigger the monitoring of emotional arousal based on the subject being tested. After the subject turns on the emotional arousal monitoring function, the skin conductance signal can be collected in real time.

[0121] Step S330: Perform preprocessing on the electrodermal signal.

[0122] Step S340: Process the preprocessed electrodermal signal using a sympathetic inversion network to obtain network inversion values ​​that characterize the level of sympathetic nerve activation.

[0123] Step S350: Perform inverse mapping on the network inversion value through the emotional arousal inverse mapping network to obtain the predicted value of emotional arousal.

[0124] As can be seen from the above embodiments, by using an end-to-end emotional arousal decoding model that includes a sympathetic inversion network and an emotional arousal inverse mapping network to predict emotional arousal, a staged decoding of emotional arousal prediction is achieved. The introduction of theoretical inversion values ​​based on physiological prior knowledge as intermediate supervision signals increases the physiological interpretability of the model in the emotional arousal decoding process. At the same time, by jointly optimizing physiological and psychological losses, the model can follow the physiological mechanism of sympathetic nerve-driven skin conductance response while fitting the statistical regularity of the data, which is conducive to improving the prediction accuracy of emotional arousal.

[0125] To further illustrate the emotional arousal decoding method provided in some embodiments of this application, this application also provides an emotional arousal decoding system. This decoding system runs as a software program on a terminal device, such as a wearable hardware device, to provide users with emotional arousal monitoring and feedback services.

[0126] Figure 7 This is a schematic diagram of the structure of an emotional arousal decoding system provided in some embodiments of this application. The system includes a skin conductance signal acquisition module, a signal preprocessing module, and an emotional arousal decoding module.

[0127] The electrodermal signal acquisition module is used to acquire electrodermal signals. Since the emotional arousal decoding system is a software program, the electrodermal signal acquisition module specifically refers to the input unit that acquires electrodermal signals. This input unit can acquire electrodermal signals collected by a dedicated electrodermal activity acquisition system or other devices and transmit the real-time acquired signals to the preprocessing module.

[0128] The signal preprocessing module is used to perform preprocessing of the electrodermal signals and transmit the preprocessed electrical signals to the emotional arousal decoding module.

[0129] The emotional arousal decoding module includes a sympathetic neural inversion network and an emotional arousal inverse mapping network. The sympathetic neural inversion network calculates the derivative of the preprocessed electrodermal signal to obtain derivative information. Based on the electrodermal signal and derivative information, it constructs an input feature tensor. This input feature tensor is then processed by a CNN-LSTM sub-network to obtain network inversion values ​​representing the sympathetic neural activation level. The emotional arousal inverse mapping network performs the inverse mapping to obtain the predicted emotional arousal value.

[0130] Figure 8 This is a schematic diagram of the structure of the sympathetic neural inversion network provided in some embodiments of this application, such as... Figure 8 As shown, the sympathetic neural inversion network includes an input feature construction subnetwork, a convolutional neural subnetwork, a long short-term memory subnetwork, and a fully connected layer.

[0131] The input feature construction subnetwork includes a temporal tensor construction unit, a derivative calculation unit, and a tensor concatenation unit. The temporal tensor construction unit constructs a temporal tensor corresponding to the preprocessed electrodermal signal based on the set batch size. The derivative calculation unit calculates the first and second derivatives of the temporal tensor and uses reflection padding to process the boundaries of the first and second derivatives, ensuring that the processed first and second derivatives are consistent with the length of the temporal tensor, thus obtaining derivative information including the processed first and second derivatives. The tensor concatenation unit concatenates the temporal tensor and derivative information along the feature dimension to form a three-dimensional input feature tensor.

[0132] The convolutional neural subnetwork is used to extract features from the input feature tensor to obtain the first feature vector.

[0133] The Long Short-Term Memory subnetwork is used to capture the temporal dependencies between the first feature vectors to obtain the second feature vector.

[0134] A fully connected layer is used to map the second feature vector into a network inversion value representing the level of sympathetic neural activation.

[0135] The same or similar parts between the various embodiments in this specification can be referred to each other, and will not be repeated here.

[0136] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein.

[0137] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method of decoding arousal of emotion, characterized by, The method comprises: collecting electrodermal signals under emotional stimuli; performing preprocessing on the electrodermal signals; processing the preprocessed electrodermal signals through a sympathetic nerve inversion network to obtain a network inversion value representing a sympathetic nerve activation level, wherein the sympathetic nerve inversion network is obtained by performing derivative calculation on the preprocessed electrodermal signals to obtain derivative information, constructing an input feature tensor based on the electrodermal signals and the derivative information, and processing the input feature tensor through a convolution-long short-term memory hybrid architecture to obtain the network inversion value. performing inverse mapping of the network inversion value through an emotional arousal degree inverse mapping network to obtain a predicted value of emotional arousal degree, wherein the emotional arousal degree inverse mapping network performs the inverse mapping through a reversible function, the sympathetic nerve inversion network and the emotional arousal degree inverse mapping network constitute an end-to-end emotional arousal degree decoding model, and a loss function of the emotional arousal degree decoding model comprises a weighted sum of a physiological loss and a psychological loss, the physiological loss is used to calculate a difference between the network inversion value of the sympathetic nerve activation level and a theoretical inversion value, the theoretical inversion value is calculated based on a physiological kinetics equation on the preprocessed electrodermal signals and the derivative information, and the psychological loss is used to calculate a difference between the predicted value and a label value.

2. The method of claim 1, wherein, The method for the sympathetic nerve inversion network to obtain the derivative information comprises: constructing a time series tensor corresponding to the preprocessed electrodermal signals according to a set batch size; calculating a first-order derivative and a second-order derivative of the time series tensor; processing boundaries of the first-order derivative and the second-order derivative through reflection padding, so that the processed first-order derivative and the processed second-order derivative are consistent with the length of the time series tensor, and the derivative information comprises the processed first-order derivative and the processed second-order derivative.

3. The method of claim 1, wherein, The physiological kinetics equation is a second-order differential equation based on a pore valve model and is used to calculate the sympathetic nerve activation level according to the kinetics process of sweat diffusion and evaporation.

4. The method of claim 1, wherein, The convolution-long short-term memory hybrid architecture comprises a convolutional neural subnetwork and a long short-term memory subnetwork connected in sequence, wherein the convolution kernel of the convolutional neural subnetwork is a one-dimensional convolution kernel with a size of 3×1, the stride and the padding are both 1, the activation function is a rectified linear activation function, and the input channel number of the first layer is 3 to match the input feature tensor.

5. The method of claim 1, wherein, The physiological kinetics equation comprises a rise time parameter representing a skin electrical response and a decay time parameter representing a skin electrical response, and the rise time parameter and the decay time parameter are determined by the emotional arousal degree decoding model after pre-training.

6. The method of claim 1, wherein, The emotional arousal degree inverse mapping network comprises an inverse operation unit, the inverse operation unit inversely maps the network inversion value to the predicted value of emotional arousal degree through an exponential function with a variable parameter, and the variable parameter is determined by the emotional arousal degree decoding model after pre-training.

7. The method of decoding the degree of emotional arousal according to claim 5 or 6, characterized in that, During pre-training of the emotional arousal degree decoding model, the input sample data is sample data of a subject to be tested under the emotional stimuli.

8. The method of claim 1, wherein, The preprocessing on the electrodermal signals comprises filtering, artifact processing and segmentation processing.

9. A system for decoding arousal of emotion, characterized by The method comprises: A skin electrical signal acquisition module is configured to acquire a skin electrical signal. A signal preprocessing module is configured to perform preprocessing on the skin electrical signal. An emotional arousal decoding module includes: A sympathetic nerve inversion network is configured to perform derivative calculation on the preprocessed skin electrical signal to obtain derivative information, construct an input feature tensor based on the skin electrical signal and the derivative information, and obtain network inversion values representing sympathetic nerve activation levels by processing the input feature tensor through a convolution-long short-term memory hybrid architecture. An emotional arousal inverse mapping network is configured to perform inverse mapping on the network inversion values through an invertible function to obtain predicted values of emotional arousal, wherein the sympathetic nerve inversion network and the emotional arousal inverse mapping network constitute an end-to-end emotional arousal decoding model, and a loss function of the emotional arousal decoding model includes a weighted sum of a physiological loss and a psychological loss, the physiological loss is used to calculate the difference between the network inversion values representing sympathetic nerve activation levels and theoretical inversion values, the theoretical inversion values are obtained by calculating the preprocessed skin electrical signal and the derivative information based on a physiological kinetics equation, and the psychological loss is used to calculate the difference between the predicted values and label values.

10. The decoding system of the degree of emotional arousal according to claim 9, characterized in that, The sympathetic nerve inversion network includes: An input feature construction subnetwork includes a time series tensor construction unit, a derivative calculation unit, and a tensor concatenation unit, the time series tensor construction unit is configured to construct a time series tensor corresponding to the preprocessed skin electrical signal according to a set batch size, the derivative calculation unit is configured to calculate a first-order derivative and a second-order derivative of the time series tensor, and process the boundaries of the first-order derivative and the second-order derivative through reflection padding, so that the processed first-order derivative and the processed second-order derivative are consistent with the length of the time series tensor, to obtain derivative information including the processed first-order derivative and the processed second-order derivative, and the tensor concatenation unit is configured to concatenate the time series tensor and the derivative information along a feature dimension to form a three-dimensional input feature tensor; A convolutional neural subnetwork is configured to extract features from the input feature tensor to obtain a first feature vector; A long short-term memory subnetwork is configured to capture time sequence dependency between the first feature vectors to obtain a second feature vector; A fully connected layer is configured to map the second feature vector to network inversion values representing sympathetic nerve activation levels.

Images