Mental stress identification method based on passive field self-adaption

By constructing the source domain and target domain training sets, data transformation and pseudo-label allocation are carried out, and self-supervised training and transfer learning are used to solve the problems of low multi-classification accuracy and difficulty in cross-domain identification of mental stress recognition methods in the existing technology. Efficient mental stress recognition and model migration are achieved.

CN118983057APending Publication Date: 2024-11-19UNIV OF ELECTRONICS SCI & TECH OF CHINA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411039626.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

The existing mental stress recognition methods have problems such as low multi-category recognition accuracy, difficulty in cross-domain identification migration, and privacy protection and label distortion, which lead to the inability to perform identification.

Method used

Adopting a passive domain adaptive method, by constructing the source domain and target domain training set, data transformation and pseudo-label allocation are carried out, and features are extracted using self-supervised training and transfer learning to realize adaptive optimization of feature extractors and classifiers. Construct a mental stress recognition model.

Benefits of technology

The accuracy of multi-classification identification is improved, model migration between different fields is realized, and effective identification is carried out in the case of pressure data loss or label distortion.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118983057A_ABST
    Figure CN118983057A_ABST
Patent Text Reader

Abstract

The invention relates to the field of physiological signal processing, discloses a mental stress recognition method based on passive field self-adaption, and solves the problems that a traditional mental stress recognition method is low in multi-classification recognition accuracy and difficult in cross-domain recognition migration, and recognition cannot be carried out due to privacy protection and label distortion. The method comprises the steps of firstly constructing a source domain training set and a target domain training set; secondly, preprocessing the data in the two training sets; secondly, transforming data in the source domain training set, distributing pseudo labels, and performing self-supervised pre-training on a feature extractor and a source domain classifier; performing adaptive training on the feature extractor, the source domain classifier and the target domain classifier based on the target domain training set; and finally, constructing a mental stress identification model based on the trained feature extractor and the target domain classifier. When the recognition task is executed, electrocardiosignals of the operator are collected through the wearable device and serve as input of the model, and the mental stress category of the operator is recognized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of physiological signal processing, and in particular to a mental stress recognition method based on passive field adaptation. Background Art

[0002] Mental stress is a complex physiological reaction that can cause changes in human biology and may lead to a series of psychological problems, including depression and emotional disorder. For professionals with special responsibilities, such as drivers, the nature of their work often puts them in a high-pressure working environment. Facing excessive mental stress may not only pose a threat to society and public safety, such as the possibility of serious consequences due to misjudgment at critical moments of driving, but also, this continuous stress may also cause various physical and mental illnesses, and have a long-term and far-reaching impact on the individual's physical and mental health.

[0003] Researchers in the field of mental stress recognition have found that physiological signals have higher accuracy and stability in identifying mental stress, because these physiological indicators directly reflect the body's physiological response to mental stress.

[0004] At present, the research on the use of physiological signals for the identification of mental stress is mainly divided into two categories: traditional machine learning methods and deep learning methods. The traditional machine learning methods generally use statistical methods, K-nearest neighbor algorithms, Bayesian methods, and support vector machine models, etc.; the deep learning-based identification methods generally use deep convolutional neural networks, channel fusion convolutional networks, discriminant clustering, etc.

[0005] There are three main problems with existing mental stress recognition methods: First, the accuracy of multi-classification recognition is not high. The main reason is that existing methods mostly use a single statistical feature or a single deep feature in the process of feature extraction, and lack the extraction of physiological signal features; second, cross-domain and cross-individual recognition migration is difficult. The main reason is that individuals in different fields, such as driving, medical care, rescue and other environments have different perceptions of the same stress, which makes it difficult to migrate models across different fields; third, privacy protection in specific situations and label distortion lead to the inability to carry out recognition. The main reason is that the source data provider does not provide physiological signal data in order to protect privacy, and different participants have different perceptions of the same stress in the process of data collection, which leads to label distortion and data cannot be used. Summary of the invention

[0006] The technical problem to be solved by the present invention is to propose a mental stress recognition method based on passive domain adaptation to solve the problems of low multi-classification recognition accuracy, difficulty in cross-domain recognition migration, and inability to carry out recognition due to privacy protection and label distortion in traditional mental stress recognition methods.

[0007] The technical solution adopted by the present invention to solve the above technical problems is:

[0008] A method for identifying mental stress based on passive domain adaptation comprises the following steps:

[0009] A. Training the mental stress recognition model

[0010] A1. Based on the preset mental stress level, construct an operation scenario; collect the original physiological data of the operator in various operation scenarios, wherein the original physiological data is various physiological index data including electrocardiogram signals; construct source domain physiological data using the collected original physiological data in various operation scenarios;

[0011] A portion of the source domain physiological data is divided according to a preset ratio to form source domain training samples, which are constructed as a source domain training set; the electrocardiogram signals contained in the remaining source domain physiological data are extracted as target domain physiological data, and the target domain physiological data and its corresponding source domain physiological data constitute target domain training samples, which are constructed as a target domain training set; the source domain training set and the target domain training set both include training samples corresponding to all operation scenarios;

[0012] A2. For each source domain physiological data contained in the source domain training set and the target domain training set, first, each physiological index data contained in the source domain physiological data except the electrocardiogram signal is resampled at the sampling frequency of the electrocardiogram signal; then, each physiological index data obtained by resampling is fused with the electrocardiogram signal to obtain source domain single-channel data with the same sampling rate and dimension as the electrocardiogram signal;

[0013] A3. Based on the source domain single-channel data of the source domain training samples contained in the source domain training set, various transformation models are used to perform data transformation to obtain transformed data samples; based on the source domain single-channel data of the source domain training samples and the transformed data samples, an extended data set is constructed, and based on the data sources of the samples of the extended data set, pseudo labels are assigned to them; using the samples of the extended data set and their pseudo labels, the feature extractor and the source domain classifier are self-supervised trained;

[0014] A4. Extract target domain training samples from the target domain training set as input for this round of training;

[0015] The source domain single channel data of the source domain physiological data in the input target domain training sample is used to obtain the source domain category distribution vector using the feature extractor and source domain classifier The target domain physiological data in the input target domain training sample is used to obtain the target domain category distribution vector using the feature extractor and the target domain classifier. Among them, s represents the source domain, t represents the target domain, and i represents the sequence number of the input target domain training sample in the batch;

[0016] The source domain classifier and the target domain classifier are classifiers with the same network structure;

[0017] A5, determine whether the training of the feature extractor and the source domain classifier is completed, if so, execute step A6;

[0018] Otherwise, based on the target domain category distribution vector and the source domain category distribution vector Calculate the first joint loss The first joint loss Including classification loss Adversarial Classification Loss and contrast loss Based on the first joint loss Update the parameters of the feature extractor and the source domain classifier, and return to step A4;

[0019] A6, determine whether the training of the target domain classifier is completed, if so, proceed to step A7;

[0020] Otherwise, based on the target domain category distribution vector and the source domain category distribution vector Calculate the second joint loss The second joint loss Including classification loss and target classification loss Based on the second joint loss Update the parameters of the target domain classifier and return to step A4;

[0021] The classification loss is the sum of the classification losses of the source domain classifier and the target domain classifier;

[0022] The target classification loss is the classification loss of the target domain classifier on the target domain physiological data;

[0023] The adversarial classification loss The alignment loss between the source domain and the target domain for the features generated by the feature extractor;

[0024] The contrast loss is the contrast loss between two samples of each sample pair included in the positive set; the construction process of the positive set includes: based on the target domain category distribution vector and the source domain category distribution vector Based on the similarity judgment between the two, the target domain training samples input in step A4 are classified into a source similar set and a source dissimilar set respectively; each sample in the source similar set is combined with each sample in the source similar set to form a sample pair; each sample in the source dissimilar set is combined with each sample in the source dissimilar set to form a sample pair; the category similarity of the two samples in each sample pair is calculated, and the sample pairs that meet the conditions are selected to form a positive set;

[0025] A7. Construct a mental stress recognition model based on the trained feature extractor and target domain classifier;

[0026] B. Perform mental stress identification tasks

[0027] The operator's electrocardiogram signal is input, and the mental stress type of the current operator is identified using the mental stress recognition model.

[0028] Furthermore, in step A1, five mental stress levels are preset, including no stress, slight stress, relatively tense, great stress and extremely stress;

[0029] In step A1, the original physiological data of the operator in various operation scenarios are collected, including:

[0030] The original physiological data of each operator in various operation scenarios are collected by using the chest-worn RespiBAN device and the wrist-worn Empatica E4 device. The original physiological data are various physiological index data including electrocardiogram signals, including electrocardiogram signals, electromyogram signals, respiratory signals, galvanic skin response, photoelectric volume pulse wave, triaxial acceleration signals and skin temperature.

[0031] In step A1, a portion of the source domain physiological data of the operators is divided according to a preset ratio to construct a source domain training set; the source domain physiological data of the remaining operators is used to construct a target domain training set;

[0032] In step B, the operator's electrocardiogram signal is input from a wearable device.

[0033] Furthermore, in step A2, for each source domain physiological data contained in the source domain training set and the target domain training set, first, each physiological index data contained in the source domain physiological data except the electrocardiogram signal is resampled at the sampling frequency of the electrocardiogram signal; then, each physiological index data obtained by resampling is fused with the electrocardiogram signal to obtain source domain single-channel data with the same sampling rate and dimension as the electrocardiogram signal, including:

[0034] A21, resampling the physiological index data except the electrocardiogram signal contained in the source domain physiological data by using an interpolation resampling method, and unifying the sampling rate of each physiological index data to the sampling rate of the electrocardiogram signal;

[0035] A22. For each physiological index data and ECG signal obtained by resampling, the maximum and minimum values ​​of each physiological index data are used to normalize them respectively to obtain source domain multi-channel data;

[0036] A23. For the data of each channel in the source domain multi-channel data, Fourier transform is performed respectively, and the real part of the Fourier transform result is extracted and standardized; the real part standardized results corresponding to each channel are averaged to form the source domain single-channel data.

[0037] Furthermore, in step A2, before resampling, it also includes: using a full convolution denoising autoencoder to preprocess various physiological indicator data.

[0038] Furthermore, the feature extractor is a convolutional neural network, the source domain classifier and the target domain classifier are both fully connected layers; the target domain category distribution vector and the source domain category distribution vector It is expressed as:

[0039]

[0040] Among them, F represents the feature extractor, C s represents the source domain classifier, C t represents the target domain classifier; The source domain single-channel data representing the source domain physiological data of the target domain training samples, Represents the target domain physiological data of the target domain training samples.

[0041] Furthermore, the transformation model in step A3 includes: noise transformation, scaling transformation, negative transformation, shuffle transformation and time warp transformation; the data sources of the samples include untransformed and transformed by each transformation model, and each data source is marked with a serial number, and unique hot encoding is performed based on the serial number to obtain a pseudo label of each data source;

[0042] In step A3, using the samples of the extended dataset and their pseudo labels, the loss L of the feature extractor and source domain classifier is calculated according to the following formula: total , perform self-supervised training:

[0043]

[0044] Among them, L j is the loss of the jth data source, P j is the pseudo label of the jth data source, is the probability that the input extended dataset sample predicted by the source domain classifier is the jth data source, λ jis the loss weight of the preset j-th data source, and J is the number of data sources.

[0045] Furthermore, in the process of constructing the positive set, based on the target domain category distribution vector and the source domain category distribution vector Based on the similarity judgment between the two, the target domain training samples input in step A4 are classified into a source similarity set and a source dissimilarity set, including:

[0046] First, the target domain category distribution vector is fused according to the following formula: and the source domain category distribution vector Get the source target mixed category distribution vector

[0047]

[0048] Among them, σ(·) represents the Softmax activation function, [·] represents concatenation, and T represents matrix transposition;

[0049] Then, based on Calculate the source domain voting score according to the following formula: and the target domain voting score

[0050]

[0051]

[0052] Where K represents the number of preset mental stress levels. Represents the input of the i-th training sample The characteristics corresponding to the k-th mental stress level;

[0053] Afterwards, if Greater than If the source is similar to the source set, it is classified into the source similar set; otherwise, it is classified into the source dissimilar set.

[0054] Furthermore, the classification loss is the sum of the classification losses of the source domain classifier and the target domain classifier, calculated as follows:

[0055]

[0056] The target classification loss is the classification loss of the target domain classifier on the target domain physiological data, calculated as follows;

[0057]

[0058] The adversarial classification loss The alignment loss between the source domain and the target domain of the features generated by the feature extractor is calculated as follows;

[0059]

[0060] Among them, σ(·) represents the Softmax activation function, and Represent the target domain category distribution vector and the source domain category distribution vector respectively, and denote the source domain voting score and the target domain voting score respectively, Represents the input of the i-th training sample , which corresponds to the feature of the kth mental stress level, I represents the number of training samples in the input batch of this round of training, and K represents the number of levels of the preset mental stress level.

[0061] Furthermore, the category similarity of the two samples in each sample pair is calculated, and the sample pairs that meet the conditions are selected to form a positive set, including:

[0062] First, based on the source domain category distribution vector of the two samples (m1, m2) in the sample pair and According to the following formula, the probability distribution q of its source domain category is calculated m1 and q m2 :

[0063]

[0064] Where σ(·) represents the Softmax activation function;

[0065] Then, using the probability distribution q of the source domain category m1 and q m2 , calculate the similarity s between the two according to the following formula m1m2 :

[0066]

[0067] Where T represents matrix transpose;

[0068] Afterwards, the similarity s m1m2 Compare with the upper bound μ(n) and the lower bound l(n) respectively, and classify the values ​​greater than the upper bound μ(n) into the positive set, the values ​​less than the lower bound l(n) into the negative set, and the rest into the uncertain set;

[0069] The upper bound μ(n) and the lower bound l(t) are calculated according to the following formulas:

[0070] μ(n)=μ0-λ μ ·n

[0071] l(n)=l0+λ l ·n

[0072] 0≤l(n)≤μ(n)≤1

[0073] Among them, μ(n) and l(n) are linear functions with training round n as independent variables, μ0 and l0 are the initial upper and lower bounds respectively, and λ μ and λ l are the parameters that control the slopes of μ0 and l0 respectively.

[0074] Furthermore, the contrast loss is the contrast loss between the two samples of each sample pair included in the positive set;

[0075]

[0076] Among them, ξ(m1,m2) represents the contrast loss of the sample pair (m1,m2), s m1m2 Indicates the similarity of sample pair (m1, m2); D[μ(n)>l(n)] represents the indicator function. If the upper bound of similarity μ(n) is greater than the lower bound of similarity l(n), the value is 1, otherwise it is 0; D(γ m1m2 =1) means that if the sample pair (m1, m2) is classified as a positive set, the value is 1, otherwise it is 0; μ(n) and l(n) are linear functions with the training round n as the independent variable; γ m1v Represents the eigenvalue of the subset of sample pair (m1,v). If the subset is a positive set, then γ m1v =1, if the subset is a negative set, then γ m1v =-1, if the subset is an uncertain set, then γ m1v =0;|γ m1v | means γ m1v The absolute value of , I represents the number of training samples in the input batch of this round of training.

[0077] The beneficial effects of the present invention are:

[0078] (1) The present invention uses a signal transformation method to generate pseudo labels and pseudo tasks for large-scale pre-training, and then uses a transfer learning method to introduce the training parameters into the real mental stress recognition task for feature extraction. The features extracted in this way can solve the problem of feature simplification, distinguish the subtle differences between stress levels in actual classification, and improve the accuracy of multi-classification recognition.

[0079] (2) In the absence of physiological data samples of operators in the target domain for training, the present invention can use data from other fields to train the model, and apply the trained model to the target domain, thereby realizing the migration of models between different domains.

[0080] (2) The adaptive adversarial and contrast category matching strategies proposed in the present invention utilize the extracted features for adaptive optimization. In special cases such as operator pressure data loss and pressure data label distortion, the classifier and feature extractor can be optimized to get rid of the dependence on source data. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 The flowchart of training a driver mental stress recognition model in an embodiment of the present invention. DETAILED DESCRIPTION

[0082] The present invention aims to provide a mental stress recognition method based on passive domain adaptation, which solves the problems of low multi-classification recognition accuracy, difficulty in cross-domain recognition migration, and inability to carry out recognition due to privacy protection and label distortion in traditional mental stress recognition methods. In the present invention, a source domain training set and a target domain training set are first constructed, and the samples of the two training sets are derived from the collection and division of source domain physiological data; wherein, the data samples in the source domain training set contain various types of physiological signals, and the data samples in the target domain training set contain various types of physiological signals and also contain separately extracted electrocardiogram signals; then, the data in the two training sets are subjected to denoising and unified preprocessing of different types of signal dimensions; then, the data in the source domain training set is transformed, and pseudo labels are assigned to expand the source domain training set, and the feature extractor and the source domain classifier are self-supervised pre-trained based on the expanded source domain training set and pseudo labels; then, the feature extractor, the source domain classifier and the target domain classifier are adaptively trained based on the target domain training set; finally, a mental stress recognition model is constructed based on the trained feature extractor and the target domain classifier. When performing mental stress recognition tasks, the operator's electrocardiogram signal is collected through wearable devices and used as the input of the mental stress recognition model. The mental stress recognition model can be used to identify the mental stress category of the current operator.

[0083] Example

[0084] Taking the application of this solution in the scenario of identifying the mental stress of the driver as an example, the mental stress identification method based on passive field adaptation in this embodiment includes: first training a driver mental stress identification model, and in a specific application, inputting the driver's electrocardiogram signal, and using the trained mental stress identification model to obtain the current driver's mental stress state.

[0085] The process of training the driver mental stress recognition model can be found in Figure 1 , which includes the following processes:

[0086] S1. Construct source domain training set and target domain training set

[0087] In this step, the source domain training set is used for subsequent pre-training of the feature extractor and source domain classifier, and the target domain training set is used for adaptive training of the feature extractor, source domain classifier, and target domain classifier. The samples in the two training sets are derived from the source domain physiological data.

[0088] In an exemplary solution, the process of constructing the source domain training set and the target domain training set is as follows:

[0089] (1) Five mental stress levels are preset for drivers, namely: calm and stress-free, slightly stressed, relatively tense, very stressed, and extremely stressed.

[0090] (2) Based on the above five levels of mental stress, a driving scenario experiment that can reflect the five levels of mental stress is designed, and the original physiological data of various physiological indicators of the driver are collected during the experiment.

[0091] For example: after the experiment begins, a period of rest before the actual driving is used as a calm and non-stressful situation;

[0092] Next, the driver drove the vehicle to a busy main street in the city as a stressful situation;

[0093] Afterwards, the driver drove the vehicle into a relatively spacious suburban street in the city as a mildly stressful scenario;

[0094] Next, the driver drove the vehicle onto the highway as a more stressful scenario;

[0095] Finally, a car engine failure was simulated and an extreme accident scenario occurred as a very stressful situation.

[0096] In the above experiment, the chest-worn RespiBAN device and the wrist-worn Empatica E4 device were used to simultaneously collect various types of physiological index data of the driver as source domain physiological data, including physiological data such as electrocardiogram (ECG), electromyography (EMG), and respiratory signals (RESP). The Empatica E4 device uses photoelectric and pressure sensors to collect a variety of physiological signals, including galvanic skin response (EDA), photoplethysmography (BVP), triaxial acceleration signal (ACC), and skin temperature (TEMP).

[0097] In order to obtain sufficient source domain physiological data samples, multiple subjects can be used as drivers to conduct the above driving scenario experiments, and physiological data samples of all drivers in the corresponding five mental stress driving scenarios will be obtained.

[0098] (3) Construct source domain training set and target domain training set:

[0099] For all drivers' physiological data samples, 80% of the drivers' physiological data are divided to construct the source domain training set, and the remaining 20% ​​of the drivers' physiological data are used to construct the target domain training set.

[0100] Among them, the data samples in the source domain training set are all types of physiological signals collected above under five driving scenarios.

[0101] As for the construction of the target domain training set, the ECG signals of the remaining 20% ​​of the drivers in the five driving scenarios are first extracted as the target domain physiological data, and then the target domain physiological data and the corresponding source domain physiological data are used to form the target domain training samples to construct the target domain training set. That is, the training samples in the target domain training set include all the above-mentioned types of physiological signals collected in the five driving scenarios, as well as the ECG signals extracted separately from them.

[0102] S2. Data preprocessing of source domain training set and target domain training set

[0103] In this step, in order to reduce the noise interference of data samples and unify the data dimension, the samples in the source domain training set and the target domain training set are first denoised, and then the dimension of each source domain physiological data contained in the source domain training set and the target domain training set is transformed.

[0104] In an exemplary scheme, a fully convolutional denoising autoencoder is used to denoise various physiological indicator data in the source domain training set and the target domain training set. The encoder is derived from "Noise Reduction in ECGSignals Using Fully Convolutional Denoising Autoencoders". The fully convolutional denoising autoencoder learns how to extract useful features from noise through its encoder-decoder structure, which can not only reconstruct physiological signals but also compress data size.

[0105] On this basis, the dimensionality transformation is performed on each source domain physiological data contained in the source domain training set and the target domain training set:

[0106] Firstly, the physiological index data except the ECG signal contained in the source domain physiological data are resampled at the sampling frequency of the ECG signal; then, the physiological index data obtained by resampling are fused with the ECG signal to obtain the source domain single-channel data with the same sampling rate and dimension as the ECG signal.

[0107] The specific process of dimension transformation is as follows:

[0108] a. Resample the physiological index data except the ECG signal contained in the source domain physiological data by using the interpolation resampling method, and unify the sampling rate of each physiological index data to the sampling rate of the ECG signal;

[0109] b. For each physiological index data and ECG signal obtained by resampling, the maximum and minimum values ​​of each physiological index data are used to normalize them respectively to obtain source domain multi-channel data;

[0110] c. Perform Fourier transform on the data of each channel in the source domain multi-channel data, extract the real part of the Fourier transform result and standardize it; average the real part standardization results corresponding to each channel to form the source domain single-channel data.

[0111] S3: Self-supervised training of feature extractor and source domain classifier based on source domain training set

[0112] In this step, first, based on the source domain single-channel data of the source domain training samples contained in the source domain training set, various transformation models are used to perform data transformation to obtain transformed data samples; then, based on the source domain single-channel data of the source domain training samples and the transformed data samples, an extended data set is constructed, and based on the data sources of the samples of the extended data set, pseudo labels are assigned to them; finally, the samples of the extended data set and their pseudo labels are used to perform self-supervised training on the feature extractor and source domain classifier in the mental stress recognition model.

[0113] In an exemplary solution, data transformation can adopt noise transformation, scaling transformation, negation transformation, shuffle transformation and time warp transformation; among them, noise transformation is to add random Gaussian noise to the original signal; scaling transformation is to adjust the amplitude of the original signal by assigning a scaling factor; negation transformation is to multiply the amplitude of the original signal by -1 to make the signal time series spatially inverted; shuffle transformation is to segment the original signal and randomly disrupt the order of the segments; time warp transformation is to stretch or compress the time segments of the original signal. These five transformations are all common data transformation methods.

[0114] The above transformation can enrich the sample sources of the source domain training set, construct the source domain single-channel data and transformed data samples in the source domain training set into an extended data set, and then assign pseudo labels to each sample in the extended data set.

[0115] The assignment of pseudo labels is based on the data source, where the data source is divided into untransformed data and various transformed data. Untransformed data is the single-channel data in the source domain before transformation. Based on this, each sample in the extended data set can be assigned a number from 0 to 5 as a pseudo label, for example: if the sample is untransformed data, the pseudo label is 0; if the sample is a noise transformed data sample, the pseudo label is 1; if the sample is a scaled transformed data sample, the pseudo label is 2; if the sample is a negative transformed data sample, the pseudo label is 3; if the sample is a disordered transformed data sample, the pseudo label is 4; if the sample is a time-warped transformed data sample, the pseudo label is 5.

[0116] Finally, the feature extractor and source domain classifier are self-supervised trained using the samples of the extended dataset and their pseudo labels. The purpose of self-supervised training is to train the feature extractor and source domain classifier through these pseudo labels and determine their optimal parameters so that they can learn effective feature representations without real labels and accurately classify data.

[0117] The loss function L used in self-supervised training total It is expressed as:

[0118]

[0119] Among them, L j is the loss of the jth data source, P j is the pseudo label of the jth data source, is the probability that the input extended dataset sample predicted by the source domain classifier is the jth data source, λ j is the loss weight of the preset j-th data source; J is the number of data sources, in this embodiment, J=6, λ j A value such as 1 / 6 can make each data source contribute equally to the loss function.

[0120] The optimal parameters of the feature extractor and source domain classifier can be obtained through self-supervised training.

[0121] S4, adaptively training the feature extractor, the source domain classifier, and the target domain classifier based on the target domain training set;

[0122] In this step, the source domain is aligned to the target domain through adaptive confrontation between the source domain classification and the target domain classification, so that the model can effectively apply the learned source domain feature distribution knowledge to the target domain and adapt to the feature distribution of the target domain. For parameter optimization, an alternating iterative optimization method is adopted. First, the target domain classifier is fixed, the feature extractor and the source domain classifier are optimized, and then the parameters of the feature extractor and the source domain classifier are fixed, and the target domain classifier is optimized.

[0123] In an exemplary solution, the process of adaptively training a feature extractor, a source domain classifier, and a target domain classifier based on a target domain training set includes:

[0124] a. Input the target domain training sample, extract the sample features, and calculate the source domain category distribution vector and the target domain category distribution vector;

[0125] Specifically, the source domain single-channel data of the source domain physiological data in the input target domain training sample is used to obtain the source domain category distribution vector using the feature extractor and the source domain classifier.

[0126] The target domain physiological data in the input target domain training sample is used to obtain the target domain category distribution vector using the feature extractor and the target domain classifier. Among them, s represents the source domain, t represents the target domain, and i represents the sequence number of the input target domain training sample in the batch.

[0127] b. Determine whether the training of the feature extractor and the source domain classifier is completed. If so, obtain the trained feature extractor and source domain classifier and proceed to step c. Otherwise, calculate the first joint loss, update the parameters of the feature extractor and the source domain classifier, and return to step a.

[0128] Specifically, based on the target domain category distribution vector and the source domain category distribution vector Calculate the first joint loss The first joint loss Including classification loss Adversarial Classification Loss and contrast loss Based on the first joint loss Update the parameters of the feature extractor and source domain classifier.

[0129] Contrastive loss

[0130] Contrastive loss It is the contrast loss between the two samples of each sample pair contained in the positive set. This loss term helps the model better distinguish the features of different categories and improve the classification accuracy by maximizing the similarity between sample pairs with high similarity and minimizing the similarity between sample pairs with low similarity. Its formula is expressed as follows:

[0131]

[0132] Among them, ξ(m1,m2) represents the contrast loss of the sample pair (m1,m2), s m1m2Indicates the similarity of sample pair (m1, m2); D[μ(n)>l(n)] represents the indicator function. If the upper bound of similarity μ(n) is greater than the lower bound of similarity l(n), the value is 1, otherwise it is 0; D(γ m1m2 =1) means that if the sample pair (m1, m2) is classified as a positive set, the value is 1, otherwise it is 0; μ(n) and l(n) are linear functions with the training round n as the independent variable; γ m1v Represents the eigenvalue of the subset of sample pair (m1,v). If the subset is a positive set, then γ m1v =1, if the subset is a negative set, then γ m1v =-1, if the subset is an uncertain set, then γ m1v =0;|γ m1v | means γ m1v The absolute value of , O represents the number of training samples in the input batch of this round of training.

[0133] The construction process of the “positive set” is:

[0134] (1) Target domain category distribution vector based on each target domain training sample and the source domain category distribution vector Based on the similarity judgment between the two, the target domain training samples are classified into source similarity set and source dissimilarity set respectively:

[0135] First, the target domain category distribution vector is fused and the source domain category distribution vector Get the source target mixed category distribution vector

[0136]

[0137] Among them, σ(·) represents the Softmax activation function, [·] represents concatenation, and T represents matrix transposition;

[0138] Then, based on the source target mixed category distribution vector Calculate the source domain voting score according to the following formula: and the target domain voting score

[0139]

[0140] Where K represents the number of preset mental stress levels. Represents the input of the i-th training sample The characteristics corresponding to the k-th mental stress level;

[0141] Finally, if Greater than If the source is similar to the source set, it is classified into the source similar set; otherwise, it is classified into the source dissimilar set.

[0142] (2) Each sample in the source similarity set is paired with each sample in the source similarity set; each sample in the source dissimilar set is paired with each sample in the source dissimilar set. Then, the category similarity of the two samples in each sample pair is calculated, and the sample pairs that meet the conditions are selected to form a positive set.

[0143] The calculation of category similarity and the screening process of sample pairs are as follows:

[0144] First, based on the source domain category distribution vector of the two samples (m1, m2) in the sample pair and According to the following formula, the probability distribution q of its source domain category is calculated m1 and q m2 :

[0145]

[0146] Where σ(·) represents the Softmax activation function;

[0147] Then, using the probability distribution q of the source domain category m1 and q m2 , calculate the similarity s between the two according to the following formula m1m2 :

[0148]

[0149] Where T represents matrix transpose;

[0150] Afterwards, the similarity s m1m2 Compare with the upper bound μ(n) and the lower bound l(n) respectively, and classify the values ​​greater than the upper bound μ(n) into the positive set, the values ​​less than the lower bound l(n) into the negative set, and the rest into the uncertain set;

[0151] The upper bound μ(n) and the lower bound l(t) are calculated according to the following formulas:

[0152] μ(n)=μ0-λ μ ·n

[0153] l(n)=l0+λ l ·n

[0154] 0≤l(n)≤μ(n)≤1

[0155] Among them, μ(n) and l(n) are linear functions with training round n as independent variables, μ0 and l0 are the initial upper and lower bounds respectively, and λ μ and λ l are the parameters that control the slopes of μ0 and l0 respectively.

[0156] μ0, l0, λ μ , l The specific value of can be set according to simulation tests and manual experience. In a preferred example, μ0 and l0 are set to 0.8 and 0.2, so as to more strictly distinguish source-similar and source-dissimilar sample pairs; λ μ and λ l Setting them to 0.05 and 0.025 means that at the end of each training batch, μ0 will decrease by 0.05 and l0 will increase by 0.025. The rate of decrease is slightly larger than the rate of increase. The hope is that the model will be stricter in certainty and more flexible in uncertainty.

[0157] Classification Loss

[0158] Classification Loss is the sum of the classification losses of the source domain classifier and the target domain classifier, which is used to measure the classification performance of the source domain classifier and the target classifier. It is calculated according to the following formula:

[0159]

[0160] Among them, σ(·) represents the Softmax activation function, and Represent the target domain category distribution vector and the source domain category distribution vector respectively, and They represent the source domain voting scores and the target domain voting scores respectively; I represents the number of training samples in the input batch of this round of training.

[0161] Adversarial Classification Loss

[0162] Adversarial Classification Loss It is the alignment loss between the source domain and the target domain of the features generated by the feature extractor. It improves the cross-domain generalization ability of the model by encouraging the features generated by the feature extractor to be aligned between the source domain and the target domain. It is calculated as follows:

[0163]

[0164] in, and They represent the source domain voting scores and the target domain voting scores respectively; K represents the number of levels of preset mental stress levels; Represents the input of the i-th training sample , which corresponds to the feature of the kth mental stress level; I represents the number of training samples in the input batch of this round of training.

[0165] c. Determine whether the training of the target classifier is completed. If so, obtain the trained target classifier. Otherwise, calculate the second joint loss, update the parameters of the target classifier, and return to step a.

[0166] Specifically, based on the target domain category distribution vector and the source domain category distribution vector Calculate the second joint loss The second joint loss Including classification loss and target classification loss

[0167] Among them, the classification loss The calculation method of has been specifically described in step b and will not be repeated here.

[0168] Object classification loss is the classification loss of the target domain classifier on the target domain physiological data. This loss term focuses on the classification performance of the target domain data. The purpose is to ensure that the target classifier can adapt to the feature distribution of the target domain data. It is calculated as follows:

[0169]

[0170] in, and They represent the source domain voting scores and the target domain voting scores respectively; K represents the number of levels of preset mental stress levels; Represents the input of the i-th training sample , which corresponds to the feature of the kth mental stress level; I represents the number of training samples in the input batch of this round of training.

[0171] S5. Construct a mental stress recognition model based on the trained feature extractor and target domain classifier

[0172] In this step, the trained feature extractor and target domain classifier are constructed into a mental stress recognition model, which can be applied in actual scenarios. When performing mental stress recognition tasks, the driver's electrocardiogram signal is collected through wearable devices and used as the input of the mental stress recognition model. The mental stress recognition model can be used to identify the mental stress category of the current driver.

[0173] Finally, it should be noted that the above embodiments are only preferred implementations and are not intended to limit the present invention. It should be pointed out that for those skilled in the art, several modifications, equivalent replacements, improvements, etc. can be made without departing from the scope of the present invention and the scope of protection of the claims, and all of these should be included in the protection scope of the present invention.

Claims

1. A method for identifying mental stress based on passive domain adaptation, characterized in that: A. Training the mental stress recognition model A1. Based on the preset mental stress level, construct an operation scenario; collect the original physiological data of the operator in various operation scenarios, wherein the original physiological data is various physiological index data including electrocardiogram signals; construct source domain physiological data using the collected original physiological data in various operation scenarios; A portion of the source domain physiological data is divided according to a preset ratio to form source domain training samples, which are constructed as a source domain training set; the electrocardiogram signals contained in the remaining source domain physiological data are extracted as target domain physiological data, and the target domain physiological data and its corresponding source domain physiological data constitute target domain training samples, which are constructed as a target domain training set; the source domain training set and the target domain training set both include training samples corresponding to all operation scenarios; A2. For each source domain physiological data contained in the source domain training set and the target domain training set, first, each physiological index data contained in the source domain physiological data except the electrocardiogram signal is resampled at the sampling frequency of the electrocardiogram signal; then, each physiological index data obtained by resampling is fused with the electrocardiogram signal to obtain source domain single-channel data with the same sampling rate and dimension as the electrocardiogram signal; A3. Based on the source domain single-channel data of the source domain training samples contained in the source domain training set, various transformation models are used to perform data transformation to obtain transformed data samples; based on the source domain single-channel data of the source domain training samples and the transformed data samples, an extended data set is constructed, and based on the data sources of the samples of the extended data set, pseudo labels are assigned to them; Using samples from the extended dataset and their pseudo labels, the feature extractor and source domain classifier are trained in a self-supervised manner. A4. Extract target domain training samples from the target domain training set as input for this round of training; The source domain single channel data of the source domain physiological data in the input target domain training sample is used to obtain the source domain category distribution vector using the feature extractor and source domain classifier The target domain physiological data in the input target domain training sample is used to obtain the target domain category distribution vector using the feature extractor and the target domain classifier. Among them, s represents the source domain, t represents the target domain, and i represents the sequence number of the input target domain training sample in the batch; The source domain classifier and the target domain classifier are classifiers with the same network structure; A5, determine whether the training of the feature extractor and the source domain classifier is completed, if so, execute step A6; Otherwise, based on the target domain category distribution vector and the source domain category distribution vector Calculate the first joint loss The first joint loss Including classification loss Adversarial Classification Loss and contrast loss Based on the first joint loss Update the parameters of the feature extractor and the source domain classifier, and return to step A4; A6, determine whether the training of the target domain classifier is completed, if so, proceed to step A7; Otherwise, based on the target domain category distribution vector and the source domain category distribution vector Calculate the second joint loss The second joint loss Including classification loss and target classification loss Based on the second joint loss Update the parameters of the target domain classifier and return to step A4; The classification loss is the sum of the classification losses of the source domain classifier and the target domain classifier; The target classification loss is the classification loss of the target domain classifier on the target domain physiological data; The adversarial classification loss The alignment loss between the source domain and the target domain for the features generated by the feature extractor; The contrast loss is the contrast loss between two samples of each sample pair included in the positive set; the construction process of the positive set includes: based on the target domain category distribution vector and the source domain category distribution vector Based on the similarity judgment between the two, the target domain training samples input in step A4 are classified into a source similar set and a source dissimilar set respectively; each sample in the source similar set is combined with each sample in the source similar set to form a sample pair; each sample in the source dissimilar set is combined with each sample in the source dissimilar set to form a sample pair; the category similarity of the two samples in each sample pair is calculated, and the sample pairs that meet the conditions are selected to form a positive set; A7. Construct a mental stress recognition model based on the trained feature extractor and target domain classifier; B. Perform mental stress identification tasks The operator's electrocardiogram signal is input, and the mental stress type of the current operator is identified using the mental stress recognition model.

2. The method for identifying mental stress based on passive domain adaptation as claimed in claim 1, characterized in that: Five preset mental stress levels, including no stress, slight stress, relatively tense, very stressful and extremely stressful; In step A1, an operation scenario is constructed based on a preset mental stress level; the original physiological data of each operator in various operation scenarios are collected using a chest-worn RespiBAN device and a wrist-worn EmpaticaE4 device, respectively. The original physiological data are various physiological index data including electrocardiogram signals, including: electrocardiogram signals, electromyogram signals, respiratory signals, galvanic skin response, photoplethysmogram, triaxial acceleration signals and skin temperature; Divide a portion of the source domain physiological data of the operators according to a preset ratio to construct a source domain training set; and use the remaining source domain physiological data of the operators to construct a target domain training set; In step B, the operator's electrocardiogram signal is input from a wearable device.

3. The method for identifying mental stress based on passive domain adaptation as claimed in claim 1, characterized in that: In step A2, for each source domain physiological data contained in the source domain training set and the target domain training set, first, each physiological index data except the electrocardiogram signal contained in the source domain physiological data is resampled at the sampling frequency of the electrocardiogram signal; Then, the resampled physiological index data are fused with the ECG signal to obtain source domain single-channel data with the same sampling rate and dimension as the ECG signal, including: A21, resampling the physiological index data except the electrocardiogram signal contained in the source domain physiological data by using an interpolation resampling method, and unifying the sampling rate of each physiological index data to the sampling rate of the electrocardiogram signal; A22. For each physiological index data and ECG signal obtained by resampling, the maximum and minimum values ​​of each physiological index data are used to normalize them respectively to obtain source domain multi-channel data; A23. For the data of each channel in the source domain multi-channel data, Fourier transform is performed respectively, and the real part of the Fourier transform result is extracted and standardized; the real part standardized results corresponding to each channel are averaged to form the source domain single-channel data.

4. A method for identifying mental stress based on passive domain adaptation as claimed in any one of claims 1 to 3, characterized in that: In step A2, before resampling, it also includes: using a full convolution denoising autoencoder to preprocess various physiological index data.

5. The method for identifying mental stress based on passive domain adaptation as claimed in claim 1, characterized in that: The feature extractor is a convolutional neural network, the source domain classifier and the target domain classifier are both fully connected layers; the target domain category distribution vector and the source domain category distribution vector It is expressed as: Among them, F represents the feature extractor, C s represents the source domain classifier, C t represents the target domain classifier; The source domain single-channel data representing the source domain physiological data of the target domain training samples, Represents the target domain physiological data of the target domain training samples.

6. A method for identifying mental stress based on passive domain adaptation as claimed in claim 1 or 5, characterized in that: The transformation models in step A3 include: noise transformation, scaling transformation, negative transformation, shuffle transformation and time warp transformation; the data sources of the samples include untransformed and transformed by each transformation model, and each data source is marked with a serial number, and unique hot encoding is performed based on the serial number to obtain a pseudo label for each data source; In step A3, using the samples of the extended dataset and their pseudo labels, the loss L of the feature extractor and source domain classifier is calculated according to the following formula: total , perform self-supervised training: Among them, L j is the loss of the jth data source, P j is the pseudo label of the jth data source, is the probability that the input extended dataset sample predicted by the source domain classifier is the jth data source, λ j is the loss weight of the jth data source, and J is the number of data sources.

7. A method for identifying mental stress based on passive domain adaptation as claimed in claim 1 or 5, characterized in that: In the process of constructing the positive set, based on the target domain category distribution vector and the source domain category distribution vector Based on the similarity judgment between the two, the target domain training samples input in step A4 are classified into a source similarity set and a source dissimilarity set, including: First, the target domain category distribution vector is fused according to the following formula: and the source domain category distribution vector Get the source target mixed category distribution vector Among them, σ(·) represents the Softmax activation function, [·] represents concatenation, and T represents matrix transposition; Then, based on Calculate the source domain voting score according to the following formula: and the target domain voting score Where K represents the number of preset mental stress levels. Represents the input of the i-th training sample The characteristics corresponding to the k-th mental stress level; Afterwards, if Greater than If the source is similar to the original source, it is classified into the source similar set; otherwise, it is classified into the source dissimilar set.

8. The method for identifying mental stress based on passive domain adaptation as claimed in claim 7, characterized in that: The classification loss is the sum of the classification losses of the source domain classifier and the target domain classifier, calculated as follows: The target classification loss is the classification loss of the target domain classifier on the target domain physiological data, calculated as follows; The adversarial classification loss The alignment loss between the source domain and the target domain of the features generated by the feature extractor is calculated as follows; Among them, σ(·) represents the Softmax activation function, and Represent the target domain category distribution vector and the source domain category distribution vector respectively, and denote the source domain voting score and the target domain voting score respectively, Represents the input of the i-th training sample , which corresponds to the feature of the kth mental stress level, I represents the number of training samples in the input batch of this round of training, and K represents the number of levels of the preset mental stress level.

9. A method for identifying mental stress based on passive domain adaptation as claimed in claim 1 or 5, characterized in that: Calculate the category similarity of the two samples in each sample pair, select the sample pairs that meet the conditions, and form a positive set, including: First, based on the source domain category distribution vector of the two samples (m1, m2) in the sample pair and According to the following formula, the probability distribution q of its source domain category is calculated m 1 and q m 2: Where σ(·) represents the Softmax activation function; Then, using the probability distribution q of the source domain category m 1 and q m 2. Calculate the similarity s between the two according to the following formula: m 1m2: Where T represents matrix transpose; Afterwards, the similarity s m 1m2 is compared with the upper bound μ(n) and the lower bound l(n) respectively, and the values ​​greater than the upper bound μ(n) are classified into the positive set, the values ​​less than the lower bound l(n) are classified into the negative set, and the rest are classified into the uncertain set; The upper bound μ(n) and the lower bound l(t) are calculated according to the following formulas: μ(n)=μ0-λμ·n l(n)=l0+λ l ·n 0≤l(n)≤μ(n)≤1 Among them, μ(n) and l(n) are linear functions with training round n as independent variables, μ0 and l0 are the initial upper and lower bounds respectively, and λ μ and λ l are the parameters that control the slopes of μ0 and l0 respectively.

10. The method for identifying mental stress based on passive domain adaptation as claimed in claim 9, characterized in that: The contrast loss is the contrast loss between the two samples of each sample pair included in the positive set; Among them, ξ(m1, m2) represents the contrast loss of the sample pair (m1, m2), s m1m2 Indicates the similarity of sample pair (m1, m2); D[μ(n)>l(n)] represents the indicator function. If the upper bound of similarity μ(n) is greater than the lower bound of similarity l(n), the value is 1, otherwise it is 0; D(γ m1m2 =1) means that if the sample pair (m1, m2) is classified as a positive set, the value is 1, otherwise it is 0; μ(n) and l(n) are linear functions with the training round n as the independent variable; γ m1v Represents the eigenvalue of the subset of sample pair (m1, v). If the subset is a positive set, then γ m1v =1, if the subset is a negative set, then γ m1v =-1, if the subset is an uncertain set, then γ m1v =0;|γ m1v | means γ m1v The absolute value of , I represents the number of training samples in the input batch of this round of training.