A method for detecting abnormal emotions based on physiological signals

Abstract

The present invention discloses an abnormal emotion detection method based on physiological signals, which belongs to the field of artificial intelligence technology and comprises the following steps: Step 1: using a stacking operation to increase the dimension of a one-dimensional physiological signal to N dimensions. Step 2: inputting the multidimensional data processed by the stacking operation into a variational autoencoder (VAE) to obtain reconstructed data corresponding to the input data. Step 3: calculating the reconstruction probability and the whitening distance from the input data and the corresponding reconstructed data, taking the weighted sum of the reconstruction probability and the whitening distance as an abnormal score, setting an abnormal threshold, and the data whose abnormal score exceeds the threshold is abnormal.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to an abnormal emotion detection method based on physiological signals. Background Art

[0002] Emotional states not only play an important role in people's daily lives and work, but also greatly affect human health. The experience of negative emotions is inevitable, but long-term negative emotions, such as stress and depression, will affect the body's hormone secretion, reduce the body's resistance and immunity, and even increase the risk of cardiovascular disease. Considering the serious losses caused by negative emotions, it is necessary to detect and pay attention to abnormal emotions.

[0003] In recent years, with the development of technologies such as artificial intelligence and sensors, emotion recognition has received widespread attention and has been widely used in the fields of intelligent medical care, human-computer interaction, robotics, health monitoring, and rehabilitation assistance for patients. The latest progress in machine learning and deep learning has proved that physiological signals are a reliable source of information for emotion detection. Compared with the conventional time series anomaly detection problem, the abnormal emotion detection problem based on physiological signals has its own unique characteristics: 1. A single data point does not have data characteristics, but the waveform composed of points contains emotional characteristics; 2. The diversity of normal and abnormal patterns; 3. There is obvious heterogeneity between individuals, so that the difference between normal waveforms of different individuals may be greater than the difference between normal and abnormal waveforms. In addition, most emotion detection schemes based on physiological signals are supervised methods, which greatly limits their application. Therefore, it is necessary to study an unsupervised abnormal emotion detection method that can solve the above characteristics. Summary of the invention

[0004] In view of this, the present invention aims at the problems existing in the prior art and provides an abnormal emotion detection method based on physiological signals. The method is based on a non-invasive wearable device to collect physiological signals of the human body to identify abnormal emotions. The wearable device can monitor the emotional state of an individual for a long time. When an individual is in a bad mood for a long time, a reminder will be issued for health status monitoring or auxiliary treatment.

[0005] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0006] A method for detecting abnormal emotions based on physiological signals comprises the following steps:

[0007] Step 1: Use a stacking operation to increase the dimension of a one-dimensional physiological signal to N dimensions to obtain an N-dimensional signal;

[0008] Step 2: Input the N-dimensional signal processed by the stacking operation in step 1 into a variational autoencoder (VAE) to obtain reconstructed data corresponding to the input N-dimensional signal;

[0009] Step 3: Calculate the reconstruction probability and whitening distance based on the N-dimensional signal obtained in step 1 and the corresponding reconstructed data obtained in step 2, and use the weighted sum of the calculated reconstruction probability and whitening distance as the abnormality score. When the abnormality score exceeds the set threshold, the one-dimensional physiological signal is abnormal.

[0010] In step 1, the stacking operation method is specifically as follows:

[0011] The one-dimensional physiological signal x is divided into multiple subsequences through a window of length T, represented as x t-T：t ={x t-T , x t-T+1 , …, x t-1}, where t = T, 2T ..., kT (kT < n + 1), n ​​is the length of the one-dimensional physiological signal, and the divided subsequences are stacked along the column direction to form the following matrix;

[0012]

[0013] On this basis, a sliding window with a step size p is formed on the one-dimensional physiological signal x to form multiple k×T matrices, thereby achieving dimensionality increase of the one-dimensional time series;

[0014] In step 1, in order to reduce the jump problem near the cutting point during cutting, when cutting the subsequence, that is, x s , at the beginning of each row of the T matrix except the first row, the data points of length q are repeated to form the following matrices:

[0015]

[0016]

[0017]

[0018]

[0019]

[0020]

[0021] Finally, the one-dimensional time series signal x is converted into dimensional signal, where N is the number of window sliding times, and kT-(k-1)q+Np≤n;

[0022] In step 2, the variational autoencoder consists of an encoder and a decoder Composition, to achieve the representation of N-dimensional signal as low-dimensional potential representation z, and then use z to reconstruct the original N-dimensional signal input; the space where the output data of the encoder of the variational autoencoder is located It is called latent space. 2D convolutional layers are introduced in both the encoder and decoder of the variational autoencoder to realize multi-scale periodic feature extraction. In the latent space of the variational autoencoder, a Gaussian mixture distribution prior is used to replace the traditional single Gaussian distribution. Assume that there are M clusters in the Gaussian distribution, and M is a predefined hyperparameter. is the category distribution matrix, Cat(π) is the prior category distribution;

[0023] The method to obtain the reconstructed data is:

[0024] First, the input N-dimensional signal is output through the encoder of the variational autoencoder to obtain M groups of mean and standard deviation vectors corresponding to the input N-dimensional signal. The M groups of mean and standard deviation vectors correspond to the M clusters in the Gaussian distribution. According to the classification prior distribution between the M clusters in the mixed Gaussian model, a component c is selected, c~Cat(π), to determine the potential Gaussian distribution corresponding to the N-dimensional signal. Furthermore, in the Gaussian distribution Upsample F times to obtain F potential vectors corresponding to the N-dimensional signal, and use f = 1, ..., F represents the fth potential vector, and the sampled potential vector is input into the decoder to obtain the corresponding reconstructed mean vector and reconstruct the standard deviation vector As the output of the decoder;

[0025] In step 3, the N-dimensional signal input by the encoder is fitted to the multivariate Gaussian distribution of the reconstructed mean vector and the reconstructed standard deviation vector obtained in step 2, and the reconstruction probabilities corresponding to all F potential vectors are averaged to obtain the reconstruction probability of the input N-dimensional signal, as shown in the following formula:

[0026]

[0027] In step 3, for each N-dimensional signal, the distance from the corresponding category center is measured, and all F potential vectors are averaged to obtain the whitening distance corresponding to the N-dimensional signal, as shown in the following formula:

[0028]

[0029] The weighted sum of the obtained reconstruction probability and whitening distance is taken as the anomaly score, and λ is the weighting coefficient, as shown in the following formula:

[0030]

[0031] Compared with the prior art, the present invention has the following significant advantages:

[0032] 1. This paper proposes a stacking operation that can increase the receptive field of the model and enable 2D convolution to extract the temporal dependency of physiological signals. The idea of ​​stacking operation is not only applicable to the anomaly detection task in this paper, but also to other time series tasks.

[0033] 2. This paper proposes a new framework for abnormal emotion detection. The framework is implemented using a variational autoencoder (VAE) model based on a convolutional neural network (CNN). In the latent space, we use a Gaussian mixture distribution prior to replace the traditional single Gaussian distribution to better describe multiple categories of normal samples. We use the whitening distance as the anomaly score, combined with the prior information learned by the Gaussian mixture model, so that the model measures the degree of outliers of samples under the condition of independent and identical distribution between dimensions of multidimensional data, thereby producing more accurate detection results.

[0034] 3. The present invention is based on deep learning technology and does not require any expert prior knowledge. It only needs to provide the data required for model training to detect abnormal emotions based on physiological signals online. At the same time, the model described in the present invention has good universality and does not require the collection of individual baseline data.

[0035] 4. The present invention discloses a method for detecting abnormal emotions based on physiological signals, which belongs to the field of artificial intelligence technology and includes the following steps: Step 1: Use a stacking operation to increase the dimension of a one-dimensional physiological signal to N dimensions. Step 2: Input the multidimensional data processed by the stacking operation into a variational autoencoder (VAE) to obtain reconstructed data corresponding to the input data. Step 3: Calculate the reconstruction probability and the whitening distance from the input data and the corresponding reconstructed data, and use the weighted sum of the reconstruction probability and the whitening distance as the abnormal score, set an abnormal threshold, and the data with an abnormal score exceeding the threshold is abnormal. The present invention proposes a framework for abnormal emotion detection. It is implemented using a variational autoencoder model based on a convolutional neural network (CNN). In the latent space, a Gaussian mixture distribution prior is used to replace the traditional single Gaussian distribution, so as to better describe multiple types of normal samples. We use the whitening distance as the abnormal score, combined with the prior information learned by the Gaussian mixture model, so that the model measures the degree of outliers of the sample under the condition that each dimension of the multidimensional data is independent and identically distributed, thereby producing a more accurate detection result. At the same time, the model described in the present invention has good universality and does not require the collection of individual baseline data. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a structural diagram of the abnormal emotion detection model based on physiological signals;

[0037] Figure 2 Schematic diagram of the physiological signal acquisition device. DETAILED DESCRIPTION

[0038] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0039] A method for detecting abnormal emotions based on physiological signals comprises the following steps:

[0040] Step 1: Use a stacking operation to increase the dimension of a one-dimensional physiological signal to N dimensions to obtain an N-dimensional signal;

[0041] Step 2: Input the N-dimensional signal processed by the stacking operation in step 1 into a variational autoencoder (VAE) to obtain reconstructed data corresponding to the input N-dimensional signal;

[0042] Step 3: Calculate the reconstruction probability and whitening distance based on the N-dimensional signal obtained in step 1 and the corresponding reconstructed data obtained in step 2, and use the weighted sum of the calculated reconstruction probability and whitening distance as the abnormality score. When the abnormality score exceeds the set threshold, the one-dimensional physiological signal is abnormal.

[0043] In step 1, the stacking operation method is specifically as follows:

[0044] The one-dimensional physiological signal x is divided into multiple subsequences through a window of length T, represented as x t-T：t ={x t-T , x t-T+1 , …, x t-1}, where t = T, 2T ..., kT (kT < n + 1), n ​​is the length of the one-dimensional physiological signal, and the divided subsequences are stacked along the column direction to form the following matrix;

[0045]

[0046] On this basis, a sliding window with a step size p is formed on the one-dimensional physiological signal x to form multiple k×T matrices, thereby achieving dimensionality increase of the one-dimensional time series;

[0047] In step 1, in order to reduce the jump problem near the cutting point during cutting, when cutting the subsequence, that is, x s , at the beginning of each row of the r matrix except the first row, the data points of length q are repeated to form the following matrices:

[0048]

[0049]

[0050]

[0051]

[0052]

[0053]

[0054] Finally, the one-dimensional time series signal x is converted into dimensional signal, where N is the number of window sliding times, and kT-(k-1)q+Np≤n;

[0055] In step 2, the variational autoencoder consists of an encoder and a decoder Composition, to achieve the representation of N-dimensional signal as low-dimensional potential representation z, and then use z to reconstruct the original N-dimensional signal input; the space where the output data of the encoder of the variational autoencoder is located It is called latent space. 2D convolutional layers are introduced in both the encoder and decoder of the variational autoencoder to realize multi-scale periodic feature extraction. In the latent space of the variational autoencoder, a Gaussian mixture distribution prior is used to replace the traditional single Gaussian distribution. Assume that there are M clusters in the Gaussian distribution, and M is a predefined hyperparameter. is the category distribution matrix, Cat(π) is the prior category distribution;

[0056] The method to obtain the reconstructed data is:

[0057] First, the input N-dimensional signal is output through the encoder of the variational autoencoder to obtain M groups of mean and standard deviation vectors corresponding to the input N-dimensional signal. The M groups of mean and standard deviation vectors correspond to the M clusters in the Gaussian distribution. According to the classification prior distribution between the M clusters in the mixed Gaussian model, a component c is selected, c~Cat(π), to determine the potential Gaussian distribution corresponding to the N-dimensional signal. Furthermore, in the Gaussian distribution Upsample F times to obtain F potential vectors corresponding to the N-dimensional signal, and use f = 1, ..., F represents the fth potential vector, and the sampled potential vector is input into the decoder to obtain the corresponding reconstructed mean vector and reconstruct the standard deviation vector As the output of the decoder;

[0058] In step 3, the N-dimensional signal input by the encoder is fitted to the multivariate Gaussian distribution of the reconstructed mean vector and the reconstructed standard deviation vector obtained in step 2, and the reconstruction probabilities corresponding to all F potential vectors are averaged to obtain the reconstruction probability of the input N-dimensional signal, as shown in the following formula:

[0059]

[0060] In step 3, for each N-dimensional signal, the distance from the corresponding category center is measured, and all F potential vectors are averaged to obtain the whitening distance corresponding to the N-dimensional signal, as shown in the following formula:

[0061]

[0062] The weighted sum of the obtained reconstruction probability and whitening distance is taken as the abnormality score, and is the weighting coefficient, as shown in the following formula:

[0063]

[0064] Example

[0065] like Figure 1 As shown, the present invention provides an abnormal emotion detection model based on physiological signals. The model increases the dimension of one-dimensional physiological signals into three-dimensional data. At this time, the time correlation of the signal is converted into spatial position correlation, so that each preprocessed three-dimensional data can cover all the information of a subsequence on the observed time series. The preprocessed data is sent to the model training module to learn the representation and distribution of normal patterns. In the proposed framework, this offline training can be performed regularly. The trained model will be stored for online anomaly detection. The preprocessed three-dimensional data is input into the trained model to obtain its anomaly score and then determine whether it is abnormal.

[0066] In step 1, in order to facilitate model calculation, the collected one-dimensional physiological signals are firstly subjected to simple data preprocessing, including deleting incomplete data and applying minimum-maximum normalization to make the collected physiological signals of each subject within the same range. Then, the window size is selected as 60 seconds, and then a stacking operation is used to generate a total of 39,249 three-dimensional data of size 3*224*224.

[0067] In step 2, the Adam optimizer is used to optimize the variational autoencoder model according to the objective function shown in the following formula.

[0068]

[0069] Among them, the first term is the reconstruction term, which encourages the method to better explain the data samples. The second term is the Kullback-Leibler divergence from the mixed Gaussian prior to the variational posterior, which regularizes the potential representation z on the mixed Gaussian distribution, thereby better learning the description of multimodal time series.

[0070] In step 3, we assume that the abnormality rate of the one-dimensional physiological signal is known, denoted as μ%, and sort the abnormal scores from small to large. The abnormal score ranked in μ% is taken as the threshold for detecting abnormalities. The abnormal score less than the threshold is an abnormal sample, and the abnormal score greater than the threshold is a normal sample.

[0071] In order to verify the feasibility of the proposed scheme in daily life, a low-cost flexible wearable wireless sensor was independently designed to capture ECG signals. ECG data was collected using an ECG detection module based on the AD8232 biochip design and Ag / AgCl medical electrodes, and the values ​​were transmitted to the terminal in ASCII form using Bluetooth communication to record the ECG. The LA electrode was placed on the back of the left hand, and the RA and LA were placed on the back of the right hand. The sampling frequency of the ECG acquisition module is 200Hz. The ADC module of the MCU was used to pre-process the collected signal, and a 1-100Hz bandpass filter was designed to retain the main signal. At the same time, a 50Hz notch filter was set to remove power frequency interference, which produced a filtered signal with a clear peak. The experimental scenario and ECG acquisition device are as follows: Figure 2 shown.

[0072] Electrocardiogram (ECG) signals of 12 participants during emotional arousal in an office environment were recorded by using work stress stimuli, as well as participants' self-assessment of their emotional state after each stimulation, including subjective experience in terms of task load, mental effort, mood, and perceived stress. We induced abnormal emotional responses by controlling the participants' task completion time and applying interference.

[0073] The experiment consists of three stages. First, there is a relaxation stage of about 10 minutes. Then, the subjects will receive a work task prompt and are required to enter a work state without time limit and without interference for about one hour, which is called the relaxed work state. After the end, the subjects are allowed to take a short break. Finally, the participants will work for about 1 hour under the mandatory work completion time and may be subject to work interference at any time. This stage is called the stress work state. At the end of each work state, the participants were invited to complete a questionnaire about their current subjective experience. We collected ECG signals three times in each work stage, each lasting 5 minutes, with an interval of 10 minutes.

[0074] We used the collected data to train the proposed abnormal emotion detection model, and the actual inference time of the model was 108 (± 30) ms. Finally, we obtained an AUC of 0.742, an F1 of 0.709, and a detection accuracy of 85% for the model to detect abnormal emotions in daily work environments, thus verifying the feasibility of the actual deployment of the proposed system solution and its good application prospects.

[0075] The proposed model was run on a real mobile device. The goal is to build a personal health system for detecting abnormal emotions. The trained model is saved as a .pth file, which we convert to a .pt file. The application runs on the Android 11 operating system.

[0076] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting abnormal emotions based on physiological signals, characterized in that The steps include: Step 1: Use a stacking operation to increase the dimension of a one-dimensional physiological signal to N dimensions to obtain an N-dimensional signal; Step 2: Input the N-dimensional signal processed by the stacking operation in step 1 into the variational autoencoder to obtain reconstructed data corresponding to the input N-dimensional signal; Step 3: Calculate the reconstruction probability and whitening distance according to the N-dimensional signal obtained in step 1 and the corresponding reconstructed data obtained in step 2, and use the weighted sum of the calculated reconstruction probability and whitening distance as the abnormality score. When the abnormality score exceeds the set threshold, the one-dimensional physiological signal is abnormal. In step 2, the method for obtaining the reconstructed data is: First, the input N-dimensional signal is output through the encoder of the variational autoencoder to obtain M groups of mean and standard deviation vectors corresponding to the input N-dimensional signal. The M groups of mean and standard deviation vectors correspond to the M clusters in the Gaussian distribution. According to the classification prior distribution between the M clusters in the mixed Gaussian model, a component c is selected, c~Cat(π), to determine the potential Gaussian distribution corresponding to the N-dimensional signal. Furthermore, in the Gaussian distribution Upsample F times to obtain F potential vectors corresponding to the N-dimensional signal, and use Represents the fth potential vector, and the sampled potential vector is input into the decoder to obtain the corresponding reconstructed mean vector and reconstruct the standard deviation vector As the output of the decoder; In step 3, the N-dimensional signal input by the encoder is fitted to the multivariate Gaussian distribution of the reconstructed mean vector and the reconstructed standard deviation vector obtained in step 2, and the reconstruction probabilities corresponding to all F potential vectors are averaged to obtain the reconstruction probability of the input N-dimensional signal, as shown in the following formula: In step 3, for each N-dimensional signal, the distance from the corresponding category center is measured, and all F potential vectors are averaged to obtain the whitening distance corresponding to the N-dimensional signal, as shown in the following formula: The weighted sum of the obtained reconstruction probability and whitening distance is taken as the anomaly score, and λ is the weighting coefficient, as shown in the following formula:

2. The abnormal emotion detection method based on physiological signals according to claim 1, characterized in that: In step 1, the stacking operation method is specifically as follows: In the first step, the one-dimensional physiological signal x is divided into multiple subsequences through a window of length T, and the divided subsequences are stacked along the column direction to obtain the matrix x s，T ; In the second step, the matrix x obtained in the first step is s，T Sliding the window on the one-dimensional physiological signal x with a step size p forms multiple k×T matrices; The third step is to repeat the data points of length q at the beginning of each row of the multiple k×T matrices obtained in the second step except the first row, and complete the conversion of the one-dimensional time series signal x into dimensional signal operations.

3. The abnormal emotion detection method based on physiological signals according to claim 2, characterized in that: The multiple sub-sequences into which the one-dimensional physiological signal x is divided are: x t-T:t ={x t-T ,x t-T+1 ,…,x t-1 } Wherein, t=T, 2T . . . kT (kT<n+1), and n is the length of the one-dimensional physiological signal.

4. The abnormal emotion detection method based on physiological signals according to claim 2 or 3, characterized in that: The matrix x s，T It is expressed as:

5. The abnormal emotion detection method based on physiological signals according to claim 4, characterized in that: In the third step, It is expressed as:

6. The abnormal emotion detection method based on physiological signals according to claim 5, characterized in that: In the N-dimensional signal, kT-(k-1)q+Np≤n.

7. The abnormal emotion detection method based on physiological signals according to claim 1, characterized in that: In step 2, the variational autoencoder consists of an encoder and a decoder Composition, to achieve the representation of N-dimensional signal as low-dimensional potential representation z, and then use z to reconstruct N-dimensional signal; the space where the output data of the encoder of the variational autoencoder is located It is called latent space. 2D convolutional layers are introduced in both the encoder and decoder of the variational autoencoder to realize multi-scale periodic feature extraction. In the latent space of the variational autoencoder, Gaussian mixture distribution is used. Assume that there are M clusters in the Gaussian distribution, and M is a predefined hyperparameter. is the category distribution matrix, Cat(π) is the prior category distribution.

Images