A method and system for monitoring stress and emotion based on electrocardiosignal

Abstract

The application provides a stress and emotion monitoring method and monitoring system based on an electrocardiosignal, and aims to solve the technical problem that the existing stress and emotion monitoring method based on an electrocardiosignal usually adopts shallow machine learning or rules for monitoring, resulting in low monitoring accuracy. The stress and emotion monitoring method comprises the following steps: a hybrid neural network model comprising an Input layer, a CEEMDAN layer, a FeatureExtract layer, a BiLSTM layer, a LayerNorm layer, an Attention layer, a FullConnection layer and a Softmax layer is built; the multi-scale features of the electrocardiosignal are mined through the CEEMDAN layer, and the BiLSTM layer is combined with the CEEMDAN layer, so that the time sequence dependence of the feature matrix is automatically learned by using the multi-scale features, the dynamic evolution process of the emotional and stress state is captured, and a powerful feature representation and fusion mechanism is constructed; and then the attention mechanism is introduced through the Attention layer, the importance weight between different features is learned, the key features highly related to the emotional and stress state are adaptively focused, and the influence of redundant and noise features is inhibited.

Description

TECHNICAL FIELD

[0001] The present application relates to stress and emotion monitoring, and in particular to a stress and emotion monitoring method and system based on electrocardiogram signals. BACKGROUND

[0002] The management of stress and emotion has an important impact on the physical and mental health of human beings. However, traditional stress and emotion monitoring methods mostly rely on the form of questionnaires, lacking convenience and real-time. The evaluation method based on physiological parameters often also needs to use professional equipment and technology, which is not convenient and practical. In recent years, more and more people have begun to use smart watches, smart bracelets and other wearable devices to track their health data, but most wearable devices can only monitor basic biological parameters such as heart rate, step count, etc., lacking monitoring of stress and emotion. Therefore, monitoring stress and emotion in daily work and life has become a key research direction.

[0003] Electrocardiogram (EEG) is a bioelectric signal that records the electrical activity of the heart, which reflects the weak current generated by the heart during the electrical physiological activity of the heart muscle cells. Electrocardiogram plays an important role in clinical medicine and health monitoring, which is used to assess heart health and detect heart disease. Electrocardiogram is a periodic electrical signal, and its frequency is usually represented by heart rate (i.e. the number of heartbeats in one minute), which is usually between 50 and 100 times per minute, and may vary according to age, gender and physical condition. At present, the existing stress and emotion monitoring method based on electrocardiogram signals usually uses shallow machine learning or rules for monitoring, which usually has low monitoring accuracy, resulting in low user acceptance. SUMMARY

[0004] The purpose of the present application is to solve the technical problem that the existing stress and emotion monitoring method based on electrocardiogram signals usually uses shallow machine learning or rules for monitoring, resulting in low monitoring accuracy, and to propose a new stress and emotion monitoring method and system based on electrocardiogram signals.

[0005] In order to achieve the above-mentioned purpose, the technical solution provided by the present application is as follows:

[0006] A stress and emotion monitoring method based on electrocardiogram signals, characterized in that it comprises the following steps:

[0007] Step 1, collecting original electrocardiogram signals of multiple users, and filtering out interference signals in the original electrocardiogram signals to obtain preprocessed electrocardiogram signals;

[0008] Step 2, building a hybrid neural network model;

[0009] The mixed neural network model comprises, in sequence according to input and output order, an Input layer, a CEEMDAN layer, a FeatureExtract layer, a BiLSTM layer, a LayerNorm layer, an Attention layer, a FullConnection layer, and a Softmax layer;

[0010] The Input layer is configured to receive the preprocessed electrocardio signals; the CEEMDAN layer is configured to perform CEEMDAN decomposition on the electrocardio signals input by the Input layer to obtain multi-order IMF components and a residual term; the FeatureExtract layer is configured to extract time domain statistical features, frequency domain features and nonlinear dynamic features from the multi-order IMF components to obtain a three-dimensional feature matrix; the BiLSTM layer comprises a forward LSTM layer and a backward LSTM layer, and is configured to learn the time sequence dependency of the three-dimensional feature matrix through the forward LSTM layer and the backward LSTM layer to obtain a hidden state sequence of the three-dimensional feature matrix; the LayerNorm layer is configured to perform layer normalization on the hidden state sequence of the three-dimensional feature matrix; the Attention layer is configured to focus on key features in the three-dimensional feature matrix through an attention mechanism to obtain an attention vector; the FullConnection layer is configured to map the attention vector to a category space of emotional and stress states; and the Softmax layer is configured to perform softmax normalization on the output of the FullConnection layer to obtain a probability distribution of the emotional and stress states.

[0011] Step 3: inputting the plurality of preprocessed electrocardio signals obtained in step 1 into the built mixed neural network model to train the mixed neural network model, and obtaining a trained mixed neural network model;

[0012] Step 4: collecting an original electrocardio signal of a current user, filtering out interference signals in the original electrocardio signal, and inputting the original electrocardio signal into the trained mixed neural network model in step 3, so that the mixed neural network model outputs a probability distribution of emotional and stress states, thereby realizing monitoring of user stress and emotion.

[0013] Further, in step 1, filtering out the interference signals in the original electrocardio signal specifically comprises:

[0014] performing low-pass filtering, downsampling and wavelet analysis on the collected original electrocardio signal in sequence to filter out the interference signals in the original electrocardio signal.

[0015] Further, in step 2, the BiLSTM layer adopts a tanh function as an activation function;

[0016] The Attention layer adopts a softmax function as an activation function;

[0017] The Softmax layer adopts a softmax function as an activation function.

[0018] The application further provides a stress and emotion monitoring system based on an electrocardiosignal, which is used for implementing the stress and emotion monitoring method based on the electrocardiosignal.

[0019] The signal acquisition device is used for collecting original electrocardiosignals of a user in real time.

[0020] The microprocessor is provided with a preprocessing module and a hybrid neural network model.

[0021] The input end of the preprocessing module is connected with the output end of the signal acquisition device, and is used for filtering interference signals in the original electrocardiosignals.

[0022] The hybrid neural network model comprises an Input layer, a CEEMDAN layer, a FeatureExtract layer, a BiLSTM layer, a LayerNorm layer, an Attention layer, a FullConnection layer and a Softmax layer which are sequentially arranged in an input-output order.

[0023] The input end of the Input layer is connected with the output end of the preprocessing module, and is used for receiving the preprocessed electrocardiosignals; the CEEMDAN layer is used for performing CEEMDAN decomposition on the electrocardiosignals input by the Input layer to obtain a plurality of IMF components and a residual term; the FeatureExtract layer is used for extracting time domain statistical features, frequency domain features and nonlinear dynamic features from the plurality of IMF components respectively to obtain a three-dimensional feature matrix; the BiLSTM layer comprises a forward LSTM layer and a backward LSTM layer, and is used for learning time sequence dependency of the three-dimensional feature matrix through the forward LSTM layer and the backward LSTM layer to obtain a hidden state sequence of the three-dimensional feature matrix; the LayerNorm layer is used for performing layer normalization on the hidden state sequence of the three-dimensional feature matrix; the Attention layer is used for focusing on key features in the three-dimensional feature matrix through an attention mechanism to obtain an attention vector; the FullConnection layer is used for mapping the attention vector to a category space of emotion and stress states; and the Softmax layer is used for performing softmax normalization on the output of the FullConnection layer to obtain a probability distribution of the emotion and stress states.

[0024] The client is in communication connection with the Softmax layer of the hybrid neural network model, is used for recording data, and displays the probability distribution of the emotion and stress states through the client.

[0025] Further, the signal acquisition device comprises an electrocardio electrode and a signal analysis module;

[0026] The electrocardio electrode is used for acquiring electrocardio signals of a user in real time.

[0027] The signal analysis module comprises a primary signal amplification module, a secondary amplification module, a filter circuit module and a high-precision sampling module.

[0028] The input end of the primary signal amplification module is connected with the output end of the electrocardio electrode, and is used for performing primary amplification on the acquired electrocardio signals and converting the current signals into input voltage signals.

[0029] The input end of the secondary amplification module is connected with the output end of the primary signal amplification module, and is used for performing secondary amplification on the input voltage signals.

[0030] The input end of the filter circuit module is connected with the output end of the secondary amplification module, and is used for filtering out interference signals in the secondary-amplified input signals.

[0031] The input end of the high-precision sampling module is connected with the output end of the filter circuit module, and is used for performing tertiary amplification on the filtered input voltage signals and realizing AD sampling, so as to extract original electrocardio signals, and the output end of the high-precision sampling module is connected with the input end of the preprocessing module.

[0032] Further, a data platform is further included.

[0033] The data platform is in communication connection with the client, and is used for transmitting data of the client to the data platform for storage and analysis.

[0034] Further, the input impedance of the primary signal amplification module is greater than 10MΩ.

[0035] The sampling frequency of the high-precision sampling module is greater than 256HZ, and the sampling bit number is greater than 32 bits.

[0036] Further, the BiLSTM layer adopts a tanh function as an activation function.

[0037] The Attention layer adopts a softmax function as an activation function.

[0038] The Softmax layer adopts a softmax function as an activation function.

[0039] Further, the primary signal amplification module, the secondary amplification module, the filter circuit module, the high-precision sampling module and the microprocessor are integrated on one chip.

[0040] The beneficial effects of the present application compared with the prior art are as follows:

[0041] 1. The pressure and emotion monitoring method based on electrocardiosignal provided by the application has the following advantages compared with the traditional emotion and pressure monitoring method based on shallow machine learning or rules:

[0042] A. The CEEMDAN layer fully excavates the multi-scale features of the electrocardiosignal, extracts more comprehensive and rich emotion and pressure features, and overcomes the limitations of single-scale features.

[0043] B. The BiLSTM layer is combined with the CEEMDAN layer innovatively, the multi-scale features extracted by the CEEMDAN layer are fully utilized, the time sequence dependence of the feature matrix is automatically learned, the dynamic evolution process of the emotion and pressure state is captured, a more powerful feature representation and fusion mechanism is constructed, the emotion and pressure monitoring performance is effectively improved, and more accurate emotion and pressure state discrimination is realized.

[0044] C. The attention mechanism is introduced through the Attention layer, the importance weight between different features is learned, the key features highly related to the emotion and pressure state are adaptively focused, the influence of redundant and noise features is suppressed, the model's interpretability and adaptability are enhanced, the emotion and pressure discrimination performance of the model is further improved, and it is helpful to cope with the emotion and pressure changes in different individuals and scenes.

[0045] D. An end-to-end hybrid neural network model is constructed, manual feature design and decision rules are not needed, the emotion and pressure state discrimination model is automatically learned through data-driven way, the emotion and pressure monitoring process is greatly simplified, and the model's generalization ability and practicability are improved.

[0046] 2. The pressure and emotion monitoring system based on electrocardiosignal provided by the application has high integration, convenient wearing and low cost, can monitor the emotion and pressure degree in the user's daily life and work, and will not interfere with the user's normal work and life. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 The flowchart of the pressure and emotion monitoring method based on electrocardiosignal of the application;

[0048] Figure 2 The structural diagram of the pressure and emotion monitoring system based on electrocardiosignal of the application.

[0049] The specific reference signs are as follows:

[0050] 1-signal acquisition device, 11-electrocardio electrode, 12-signal analysis module, 121-primary signal amplification module, 122-secondary amplification module, 123-filter circuit module, 124-high-precision sampling module; 2-microprocessor, 21-preprocessing module, 22-mixed neural network model; 3-client; 4-data platform. DETAILED DESCRIPTION

[0051] In order to make the advantages and characteristics of the present application clearer, the present application will be further described in detail below in combination with the drawings and specific embodiments.

[0052] A stress and emotion monitoring method based on electrocardio signals, as shown in the figure, specifically comprising the following steps: Figure 1

[0053] Step 1, collect original electrocardio signals of multiple users, and sequentially perform low-pass filtering, downsampling and wavelet analysis on the collected original electrocardio signals to filter out baseline drift signals, power frequency interference, noise interference and electromyographic interference in the original electrocardio signals, and obtain preprocessed electrocardio signals.

[0054] Step 2, build a mixed neural network model.

[0055] The mixed neural network model of the present application comprises Input layer, CEEMDAN layer, FeatureExtract layer, BiLSTM layer, LayerNorm layer, Attention layer, FullConnection layer and Softmax layer arranged in input-output order, the model uses CEEMDAN to adaptively decompose the original electrocardio signal x(t), and extracts multi-scale features, learns the time sequence dependence relationship through BiLSTM, and introduces the attention mechanism to focus on key emotional and stress features, and realizes end-to-end emotion and stress state recognition.

[0056] (1) Input layer: signal input.

[0057] The Input layer is used to receive the preprocessed electrocardio signals, the size is (batch_size, num_channels, signal_length), in the present model batch_size=32, num_channels=1, signal_length=7680, and no training parameters are needed.

[0058] (2) CEEMDAN layer: adaptive decomposition.

[0059] ​CEEMDAN (Complete Ensemble Empirical Mode Decomposition with Adaptive Noise) is an adaptive signal decomposition method that overcomes the mode mixing problem in EMD by introducing adaptive white noise. The core idea is to suppress mode mixing through multiple trials to obtain higher quality intrinsic mode functions (IMFs). The specific implementation steps of CEEMDAN layer are as follows:

[0060] A. Add different scale Gaussian white noise w_i(t) to the input ECG signal x(t) of the Input layer to obtain a set of perturbed signals x_i(t), which is specifically represented as follows:

[0061] x_i(t) = x(t) + β·w_i(t),

[0062] where i = 1, 2,..., I, I is the number of different scale Gaussian white noise, and β is the standard deviation of Gaussian white noise.

[0063] B. Perform EMD decomposition on each perturbed signal x_i(t) to obtain a series of IMF components.

[0064] C. Take the ensemble average of the same order IMF components in all IMF components to obtain multi-order IMF components IMF_j(t):

[0065]

[0066] where j is the order index of the IMF component, j = 1, 2,..., J, and J represents the total number of IMF components obtained by CEEMDAN decomposition.

[0067] For example, assume that EMD decomposition of each perturbed signal x_i(t) obtains 5 IMF components, then J = 5.

[0068] The first order IMF component IMF_1(t) of CEEMDAN decomposition is the ensemble average of IMF_{i,1}(t) obtained by decomposition of all x_i(t), i.e.:

[0069] IMF_1(t) = (1 / I)·[IMF_{1,1}(t) + IMF_{2,1}(t) +... + IMF_{I,1}(t)]

[0070] Similarly, IMF_2(t) to IMF_5(t) can be obtained by taking the ensemble average of IMF_{i,2}(t) to IMF_{i,5}(t), respectively.

[0071] This ensemble averaging of the IMF components of the same order effectively suppresses the mode mixing problem in EMD decomposition, resulting in more reliable and physically meaningful IMF components, thus better capturing the multi-scale characteristics of the original signal. This is a major advantage of CEEMDAN over EMD.

[0072] D. Subtract each order IMF component IMF_j(t) from the original ECG signal x(t) to obtain the residual term res(t).

[0073] Through adaptive decomposition of the CEEMDAN layer, the ECG signal is divided into multiple order IMF components and residual terms, effectively extracting the multi-scale features related to emotion and stress contained in the ECG signal, laying a foundation for subsequent feature learning. Compared with traditional wavelet transform or EMD-based ECG signal decomposition methods, CEEMDAN can better adapt to the non-stationary characteristics of ECG signals, obtain more stable and high-quality IMF components, and thus provide a more reliable feature basis for emotion and stress monitoring.

[0074] (3) FeatureExtract layer: obtain a three-dimensional feature matrix.

[0075] From the multi-order IMF components obtained by the CEEMDAN layer, three types of typical features are extracted: time-domain statistical features, frequency-domain features, and nonlinear dynamic features. These features describe the emotion and stress-related properties of ECG signals from different angles, including amplitude, frequency, complexity, etc. The extracted multi-scale features form a three-dimensional feature matrix X, where each slice X_t represents a feature vector at time t. Compared with traditional single-scale feature extraction methods, the model obtains more comprehensive and rich emotion and stress feature representation through the cooperation of the CEEMDAN layer and the FeatureExtract layer.

[0076] (4) BiLSTM layer: feature fusion.

[0077] To further tap the potential association between multi-scale features, the application proposes a feature fusion method based on BiLSTM (Bidirectional Long Short-Term Memory Network). The BiLSTM layer includes a forward LSTM layer and a backward LSTM layer, which can simultaneously utilize the forward and backward information of the feature matrix, learn the time sequence dependence of the feature matrix, i.e. learn the long and short term dependence between multi-scale features, and obtain all hidden state sequences. The output of the BiLSTM layer fuses the feature information at time t and its previous and subsequent time, forming a more abstract and advanced feature representation. Compared with traditional shallow feature fusion methods such as concatenation and weighted average, BiLSTM can automatically learn the complex time sequence relationship between multi-scale features and extract more discriminative high-level features, which helps to improve the performance of emotion and stress monitoring. Preferably, the BiLSTM layer in the embodiment adopts a tanh function as the activation function.

[0078] (5) LayerNorm layer: used for layer normalization of the hidden state sequence of BiLSTM to accelerate convergence and improve generalization ability.

[0079] (6) Attention layer: focusing on key features through attention mechanism.

[0080] The attention mechanism is introduced to adaptively focus on key features highly related to emotional and stress states by learning the importance weights between features at different times. The hidden state outputs of the forward LSTM layer and the backward LSTM layer at time t are denoted as h_(t,f) and h_(t,b) respectively, and the hidden states are concatenated to obtain the hidden state output h_t of the BiLSTM layer at each time step t h_t = [h_(t,f), h_(t,b)]. The hidden state output of the BiLSTM layer at the last time step T is denoted as h_T. In this embodiment, the Luong attention mechanism is adopted. First, the similarity e_t between the hidden state output h_T of the last step of the BiLSTM layer and the hidden state output h_t at each time step t is calculated. Then, the attention weight distribution α_t is obtained by normalizing through the softmax function. Finally, the attention vector c is obtained by weighting and summing the attention weight and the hidden state output h_t at the corresponding time step.

[0081] Through the attention mechanism, the model can adaptively adjust the importance of different features, suppress the influence of redundant and noisy features, and highlight the discriminative features of emotion and stress state, thereby improving the performance of emotion and stress monitoring. In addition, the attention weight can also serve as an explanation mechanism to reveal the contribution of different features to the discrimination of emotion and stress state, enhancing the explainability of the model. Compared with the traditional method of setting feature weights based on human experience, the attention mechanism can automatically learn the importance of features, avoiding the limitations of subjectivity and priori, and improving the adaptability and robustness of emotion and stress monitoring.

[0082] (7) FullConnection layer: maps the attention vector to the class space of emotion and stress state.

[0083] (8) Softmax layer: normalizes the output of the full connection layer through the softmax function as the activation function, to obtain the probability distribution of emotion and stress state.

[0084] The data of each layer of the hybrid neural network model in this embodiment is shown in Table 1, wherein the input size, output size and parameter number of each layer are calculated according to the actual electrocardio data and hyperparameter setting.

[0085] Table 1 Data of each layer of the hybrid neural network model

[0086] Layer Name Input Size Output Size Activation Function Input Layer (32，1，7680) (32，1，7680) - CEEMDAN Layer (32，1，7680) (32，9，7680) - FeatureExtract Layer (32，9，7680) (32，64，7680) - BiLSTM Layer (32，64，7680) (32，256，7680) tanh LayerNorm Layer (32，256，7680) (32，256，7680) - Attention Layer (32，256，7680) (32，256) softmax FullConnection Layer (32，256) (32，3) - Softmax Layer (32，3) (32，3) softmax

[0087] The calculation of the parameter number of each layer in the hybrid neural network model is as follows:

[0088] 1. Input layer: input the preprocessed electrocardio signal, the input size is (batch_size, num_channels, signal_length), in this model batch_size = 32, num_channels = 1, signal_length = 7680, the output size of the Input layer is the same as the input size, which is (32, 1, 7680); The Input layer does not need to be trained, it inputs the preprocessed electrocardio signal into the network through the pre-defined data input and tensor arrangement method, and adjusts the size and dimension order to match the input requirements of the subsequent layers.

[0089] 2. CEEMDAN layer: CEEMDAN decomposition is performed on the input signal to obtain IMF components and residual terms with a size of (batch_size, num_imfs+1, signal_length), in this model num_imfs = 8, i.e. the output size of the CEEMDAN layer is (32, 9, 7680).

[0090] 3、FeatureExtract layer: extract multi-scale features to get a feature matrix with size (batch_size, num_features, signal_length), num_features = 64 in this model, so the output size of FeatureExtract layer is (32, 64, 7680). CEEMDAN layer and FeatureExtract layer also do not need to train parameters, which are realized by pre-defined signal processing and feature engineering methods.

[0091] 4、BiLSTM layer: learn the time sequence dependence of the feature matrix to get a hidden state sequence with size (batch_size, 2*hidden_size, signal_length), hidden_size = 128 in this model, so the output size of BiLSTM layer is (32, 256, 7680). The number of parameters of BiLSTM layer is 4x[hidden_sizex(num_features+hidden_size+1)] = 4x[128x(64+128+1)] = 98816.

[0092] 5、LayerNorm layer: perform layer normalization on the input, and the output size is the same as the input size. The number of parameters of LayerNorm layer is 2xnormalized_shape, where

[0093] normalized_shape = hidden_sizex2 = 128x2 = 256, so the number of parameters is 2x256 = 512.

[0094] 6、Attention layer: after focusing on key features through attention mechanism, get an attention vector with size (batch_size, 2*hidden_size), i.e. the output size of Attention layer is (32, 256). The number of parameters of Attention layer is 4xhidden_size^2+2xhidden_size = 4x128^2+2x128 = 65792.

[0095] 7、FullConnection layer: its output size is (batch_size, num_classes), num_classes = 3 in this model, so the output size of FullConnection layer is (32, 3). The number of parameters of FullConnection layer is (2*hidden_size+1)xnum_classes = (2x128+1)x3 = 771.

[0096] 8、Softmax layer: the Softmax layer normalizes the output of the FullConnection layer into a probability distribution by a predefined Softmax function, obtaining the predicted probability of each class, and the output size is the same as the input size. The Softmax function is a fixed mathematical transformation and does not involve parameter training.

[0097] In summary, the total number of parameters of the hybrid neural network model is 98816+512+65792+771=165891. Among them, the parameter proportion of BiLSTM layer and Attention layer is 59.57% and 39.66% respectively, which is still the core component of the hybrid neural network model. The parameter proportion of LayerNorm layer and FullConnection layer slightly increases, which is 0.31% and 0.46% respectively, but it is still relatively small and plays a supporting role in optimization and output mapping.

[0098] The hybrid neural network model of the application is evaluated, and good results are obtained for the classification of emotions and stress, as shown in Tables 2 and 3.

[0099] Table 2: Emotion classification index of hybrid neural network model

[0100]

[0101] Table 3: Stress classification index of hybrid neural network model

[0102]

[0103] Step 3: input the plurality of preprocessed electrocardio signals obtained in step 1 into the hybrid neural network model built to train, and obtain the trained hybrid neural network model.

[0104] Step 4: collect the original electrocardio signal of the current user, filter out the interference signal in the original electrocardio signal, and input it into the hybrid neural network model trained in step 3, the hybrid neural network model outputs the probability distribution of the emotion and stress state, and realizes the monitoring of the user's stress and emotion.

[0105] The CEEMDAN-BiLSTM-Attention hybrid neural network model constructed by the application takes the ECG signals filtered from interference as input, and finally outputs the probability distribution of the emotional (happy, neutral, sad) and stress (high stress, medium stress, normal) states through CEEMDAN adaptive decomposition, multi-scale feature extraction, BiLSTM feature fusion and attention weight adjustment, through the full connection layer and the softmax layer. The whole process does not need artificial design of features and decision rules, and automatically learns the discrimination model of emotion and stress state through end-to-end data-driven mode, greatly simplifying the emotion and stress monitoring process.

[0106] In order to realize the above-mentioned stress and emotion monitoring method based on ECG signals, the application also provides a stress and emotion monitoring system based on ECG signals, as shown in the figure, comprising a signal acquisition device 1, a microprocessor 2, a client 3, a data platform 4, a communication module and a peripheral device indication control module. Figure 2

[0107] The signal acquisition device 1 comprises an ECG electrode 11 and a signal analysis module 12. The ECG electrode 11 is used to collect the ECG signals of the user in real time, which is usually made of good conductive material, and can be a dry electrode such as a metal, metal alloy electrode, conductive cloth, etc., or a wet electrode.

[0108] The signal analysis module 12 comprises a primary signal amplification module 121, a secondary amplification module 122, a filter circuit module 123 and a high-precision sampling module 124. The input end of the primary signal amplification module 121 is connected to the output end of the ECG electrode 11, which is used to amplify the collected physiological signals and convert the current signal into an input voltage signal. The primary signal amplification module 121 is the first stage of amplification circuit for weak signals, which has the characteristics of high input impedance, low noise, high common-mode rejection ratio, low drift, nonlinearity, etc., and can provide appropriate dynamic range; its input impedance should be above 10MΩ, which is much larger than the impedance between the electrode and the skin, so as to obtain higher amplitude signals. The input end of the secondary amplification module 122 is connected to the output end of the primary signal amplification module 121, which is used to amplify the input voltage signal twice to improve the gain. The input end of the filter circuit module 123 is connected to the output end of the secondary amplification module 122, which is used to filter out the interference signals in the input voltage signal after secondary amplification, such as power frequency interference, baseline drift and other external noise. The input end of the high-precision sampling module 124 is connected to the output end of the filter circuit module 123, which is used to amplify the input voltage signal filtered from interference three times and perform high-precision AD sampling to extract the original ECG signal. Preferably, the high-precision sampling module 124 adopts oversampling technology, and the sampling frequency is usually at least 200HZ or above, and the sampling bit is at least 16 bits or above, so as to ensure that the effective original ECG signal can be extracted. ​

[0109] The microprocessor 2 can adopt a single-chip microcomputer such as CC2640, MCS-51, DSP, etc., and the microprocessor 2 is provided with a preprocessing module 21 and a hybrid neural network model 22. The input end of the preprocessing module 21 is connected to the output end of the high-precision sampling module 124, for filtering out baseline drift signals, power frequency interference, noise interference and myoelectric interference in the original electrocardio signal, and obtaining a preprocessed electrocardio signal.

[0110] The hybrid neural network model 22 comprises, in sequence according to the input-output order, an Input layer, a CEEMDAN layer, a FeatureExtract layer, a BiLSTM layer, a LayerNorm layer, an Attention layer, a FullConnection layer and a Softmax layer. The input end of the Input layer is connected to the output end of the preprocessing module, for receiving the preprocessed electrocardio signal; the CEEMDAN layer is used for CEEMDAN decomposition of the electrocardio signal input by the Input layer, to obtain a plurality of IMF components and a residual term; the FeatureExtract layer is used for extracting time-domain statistical features, frequency-domain features and nonlinear dynamic features from the plurality of IMF components, to obtain a three-dimensional feature matrix;

[0111] The BiLSTM layer comprises a forward LSTM layer and a backward LSTM layer, for learning the time sequence dependency of the three-dimensional feature matrix through the forward LSTM layer and the backward LSTM layer, to obtain a hidden state sequence of the three-dimensional feature matrix; the LayerNorm layer is used for layer normalization of the hidden state sequence of the three-dimensional feature matrix; the Attention layer is used for focusing on key features in the three-dimensional feature matrix through an attention mechanism, to obtain an attention vector; the FullConnection layer is used for mapping the attention vector to a category space of emotions and stress states; and the Softmax layer is used for softmax normalization of the output of the full connection layer, to obtain a probability distribution of emotions and stress states.

[0112] The client 3 preferably adopts a mobile device such as a mobile phone, a tablet computer, a computer, etc., and is communicatively connected to the Softmax layer of the hybrid neural network model 22 through a communication module, for recording data and displaying the probability distribution of emotions and stress states through the client 3, i.e., the client 3 can acquire and save the trend of the user's emotional and stress state over time from the Softmax layer, to generate an intuitive historical record report, for facilitating review and analysis by the user. Preferably, the Softmax layer communicates with the communication module through a chip serial port, and the communication module communicates with the mobile client 3 through Bluetooth or WIFI.

[0113] The peripheral device indication control module is used for indicating the connection state of the communication module with other parts and the signal input state, and includes a connection indication lamp and a signal indication lamp in the embodiment, the connection indication lamp is prompted when the communication module is in a connection state with the client, and the signal indication lamp is prompted when the ECG signal quality acquired by the primary signal amplification module is unstable or no signal is acquired.

[0114] The data platform 4 and the client 3 are in communication connection, used for transmitting the data of the client 3 to the data platform 4 to display, store and analyze, so as to obtain the long-term emotion and stress degree data of the user, and provide guidance suggestions for the user.

[0115] In order to reduce the size and volume of the monitoring system, and make it convenient to wear, the primary signal amplification module 121, the secondary amplification module 122, the filter circuit module 123, the high-precision sampling module 124 and the microprocessor 2 in the signal analysis module 12 are preferably integrated in one chip in the embodiment.

[0116] In summary, the hybrid neural network model based on CEEMDAN-BiLSTM-Attention has significant innovation and superiority in the field of emotion and stress monitoring, provides a new idea and method for realizing continuous, real-time and personalized emotion and stress monitoring, and has wide application prospect and research value.

[0117] The above description is only used to illustrate the technical solutions of the present application, and not to limit them, and the ordinary skilled in the art can modify the specific technical solutions described in the above embodiments, or replace some technical features with equivalent ones, without changing the essence of the corresponding technical solutions out of the scope of the technical solutions protected by the present application.

Claims

1. A stress and emotion monitoring method based on electrocardiosignal, characterized in that, The method comprises the following steps: Step 1, collecting original electrocardio signals of multiple users, and filtering interference signals in the original electrocardio signals to obtain preprocessed electrocardio signals; Step 2, building a hybrid neural network model; The hybrid neural network model comprises, in sequence according to input and output, an Input layer, a CEEMDAN layer, a FeatureExtract layer, a BiLSTM layer, a LayerNorm layer, an Attention layer, a FullConnection layer and a Softmax layer; The Input layer is used for receiving the preprocessed electrocardio signals; the CEEMDAN layer is used for performing CEEMDAN decomposition on the electrocardio signals input by the Input layer to obtain multi-order IMF components and a residual term; the FeatureExtract layer is used for extracting time domain statistical features, frequency domain features and nonlinear dynamic features from the multi-order IMF components to obtain a three-dimensional feature matrix; the BiLSTM layer comprises a forward LSTM layer and a backward LSTM layer, and is used for learning time sequence dependency of the three-dimensional feature matrix through the forward LSTM layer and the backward LSTM layer to obtain a hidden state sequence of the three-dimensional feature matrix; the LayerNorm layer performs layer normalization on the hidden state sequence of the three-dimensional feature matrix; the Attention layer is used for focusing on key features in the three-dimensional feature matrix through an attention mechanism to obtain an attention vector; the FullConnection layer is used for mapping the attention vector to a category space of emotional and stress states; and the Softmax layer is used for performing softmax normalization on the output of the full connection layer to obtain a probability distribution of the emotional and stress states; Step 3, inputting the multiple preprocessed electrocardio signals obtained in step 1 into the built hybrid neural network model for training to obtain a trained hybrid neural network model; Step 4, collecting original electrocardio signals of a current user, filtering interference signals in the original electrocardio signals, and inputting the original electrocardio signals into the trained hybrid neural network model in step 3, so that the hybrid neural network model outputs a probability distribution of emotional and stress states, thereby realizing monitoring of stress and emotion of the user.

2. The method according to claim 1, wherein in step 1, filtering interference signals in the original electrocardio signals comprises: performing low-pass filtering, downsampling and wavelet analysis on the collected original electrocardio signals in sequence to filter interference signals in the original electrocardio signals.

3. The method according to claim 2, wherein in step 2, the BiLSTM layer adopts a tanh function as an activation function; the Attention layer adopts a softmax function as an activation function; and the Softmax layer adopts a softmax function as an activation function.

4. A stress and emotion monitoring system based on electrocardio signals, configured to implement the method according to any one of claims 1-3. ​ ​ The system comprises a signal acquisition device (1), a microprocessor (2) and a client (3); The signal acquisition device (1) is used for collecting original electrocardio signals of a user in real time; The microprocessor (2) is provided with a preprocessing module (21) and a hybrid neural network model (22); The input end of the preprocessing module (21) is connected with the output end of the signal acquisition device (1), and is used for filtering interference signals in the original electrocardio signals; The hybrid neural network model (22) comprises an Input layer, a CEEMDAN layer, a FeatureExtract layer, a BiLSTM layer, a LayerNorm layer, an Attention layer, a FullConnection layer and a Softmax layer arranged in sequence according to input and output orders; The input end of the Input layer is connected with the output end of the preprocessing module (21), and is used for receiving the preprocessed electrocardio signals; The CEEMDAN layer is used for performing CEEMDAN decomposition on the electrocardio signals input by the Input layer to obtain a plurality of IMF components and a residual term; the FeatureExtract layer is used for extracting time domain statistical features, frequency domain features and nonlinear dynamic features from the plurality of IMF components to obtain a three-dimensional feature matrix; the BiLSTM layer comprises a forward LSTM layer and a backward LSTM layer, and is used for learning time sequence dependency of the three-dimensional feature matrix through the forward LSTM layer and the backward LSTM layer to obtain a hidden state sequence of the three-dimensional feature matrix; the LayerNorm layer is used for performing layer normalization on the hidden state sequence of the three-dimensional feature matrix; the Attention layer is used for focusing on key features in the three-dimensional feature matrix through an attention mechanism to obtain an attention vector; the FullConnection layer is used for mapping the attention vector to a category space of emotion and stress state; and the Softmax layer is used for performing softmax normalization on the output of the FullConnection layer to obtain a probability distribution of the emotion and stress state. The client (3) is in communication connection with the Softmax layer of the hybrid neural network model (22), is used for recording data, and displays the probability distribution of the emotion and stress state through the client (3).

5. The system according to claim 4, characterized in that: The signal acquisition device (1) comprises an electrocardio electrode (11) and a signal analysis module (12); The electrocardio electrode (11) is used for collecting electrocardio signals of a user in real time; The signal analysis module (12) comprises a primary signal amplification module (121), a secondary amplification module (122), a filter circuit module (123) and a high-precision sampling module (124); The input end of the primary signal amplification module (121) is connected with the output end of the electrocardio electrode (11), and is used for performing primary amplification on the collected electrocardio signals, and converting a current signal into an input voltage signal; The input end of the secondary amplification module (122) is connected to the output end of the primary signal amplification module (121), for performing secondary amplification on the input voltage signal; The input end of the filter circuit module (123) is connected to the output end of the secondary amplification module (122), for filtering out interference signals in the secondary-amplified input voltage signal; The input end of the high-precision sampling module (124) is connected to the output end of the filter circuit module (123), for performing tertiary amplification on the interference-filtered input voltage signal and realizing AD sampling, so as to extract the original ECG signal, and the output end is connected to the input end of the preprocessing module (21).

6. The stress and emotion monitoring system based on ECG signals according to claim 5, characterized in that: It further comprises a data platform (4); The data platform (4) is in communication connection with the client (3), for transmitting the data of the client (3) to the data platform (4) for storage and analysis.

7. The stress and emotion monitoring system based on ECG signals according to claim 6, characterized in that: The input impedance of the primary signal amplification module (121) is greater than 10MΩ; The sampling frequency of the high-precision sampling module (124) is greater than 256HZ, and the sampling bit number is greater than 32 bits.

8. The stress and emotion monitoring system based on ECG signals according to any one of claims 4-7, characterized in that: The BiLSTM layer adopts a tanh function as an activation function; The Attention layer adopts a softmax function as an activation function; The Softmax layer adopts a softmax function as an activation function.

9. The stress and emotion monitoring system based on ECG signals according to claim 5, characterized in that: The primary signal amplification module (121), the secondary amplification module (122), the filter circuit module (123), the high-precision sampling module (124), and the microprocessor (2) are integrated on one chip.

Images