A wearable emotion monitoring and evaluation system based on brain-heart-intestine nervous system electrical signals

Abstract

The application discloses a wearable emotion monitoring and evaluation system based on brain-heart-intestine nervous system electrical signals, and belongs to the field of bioelectric signal processing and identification. The system can provide comprehensive emotion evaluation by comprehensively integrating electroencephalogram (EEG), electrocardiogram (ECG) and innovatively integrating electrogastrogram (EGG) signals closely related to nervous system activities. These signals respectively represent different emotional response mechanisms, and the fusion of multi-modal signals can more accurately capture emotional changes and improve the accuracy and reliability of emotion detection. The system can collect and analyze the physiological signals of users in real time, provide instant emotional feedback, help users identify emotional fluctuations in the early stage and take corresponding measures, and avoid emotional out-of-control or long-term emotional depression.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of bioelectric signal processing and recognition, particularly to the discipline of biomedical engineering and wearable technology. BACKGROUND

[0002] With increasing attention to health and emotional management, the importance of emotions on mental health is increasingly recognized. However, traditional wearable devices usually focus on monitoring physiological indicators such as heart rate, step count, etc., while the monitoring and management of emotions have not been fully valued. Emotions, as an important indicator of human mental health, are closely related to the brain-heart-gut nervous system. The brain is responsible for rational thinking and emotional processing, while the heart and gut nervous system plays an important role in emotions, representing emotional and intuitive responses, respectively.

[0003] Although wearable devices on the market generally have some physiological monitoring functions, they are not optimized specifically for emotion monitoring. Some smart bracelets and smart watches may integrate some emotion-related sensors, such as heart rate monitoring, electrodermal activity sensors, etc., but their emotion recognition accuracy and practicality are limited. These traditional devices often fail to fully consider the synergistic effects of the brain, heart, and gut nervous system in emotional expression, thus failing to fully capture the complexity of individual emotional states. SUMMARY

[0004] The present application proposes a wearable human emotion monitoring system based on multi-modal physiological signals, aiming to overcome the limitations of existing technology. By combining the information responses of the brain, heart, and gut nervous system, our system can more comprehensively assess individual emotional states and provide more accurate and comprehensive emotional management services to users. The present application considers signals related to emotions obtained from wearable devices, such as EEG (electroencephalogram), ECG (electrocardiogram), and EGG (electrogastrogram), extracts emotion-related information from these signals, and integrates them into an emotion monitoring and evaluation system that can detect emotions and provide feedback.

[0005] The technical solution of the present application is a wearable emotion monitoring and evaluation system based on brain-heart-gut nervous system electrical signals, which includes a signal acquisition and preprocessing unit, a feature extraction unit, a feature fusion unit, an emotion recognition unit, and an emotion evaluation unit. The signal acquisition and preprocessing unit includes EEG (electroencephalogram), ECG (electrocardiogram), and EGG (electrogastrogram) signal acquisition and preprocessing, and the preprocessed signals are transmitted to the feature extraction unit. The feature extraction unit extracts features from the received EEG, ECG, and EGG signals and transmits the extracted features to the feature fusion unit. The features extracted by the feature extraction unit include EEG features, ECG features, and EGG features.

[0006] The electroencephalogram features include: electroencephalogram time domain features, electroencephalogram frequency domain features, electroencephalogram nonlinear features;

[0007] The electrocardiogram features include: electrocardiogram time domain features, electrocardiogram frequency domain features, electrocardiogram nonlinear features;

[0008] The electrogastric features include: electrogastric time domain features, electrogastric frequency domain features, electrogastric nonlinear features;

[0009] The feature fusion unit is a multi-input deep learning network, which fuses all the features, and then inputs the fused features into the emotion recognition unit and the emotion evaluation unit;

[0010] The emotion recognition unit identifies the type of emotion, and the emotion evaluation unit identifies the intensity of emotion.

[0011] Further, the electroencephalogram time domain features include:

[0012] Mean potential: the average electroencephalogram activity level in a set time period,

[0013] Peak potential: the potential value of the highest and lowest points,

[0014] Waveform area: the area of the signal waveform within a certain time,

[0015] Electroencephalogram amplitude: the difference between the maximum amplitude and the minimum amplitude of the signal;

[0016] The electroencephalogram frequency domain features include:

[0017] Power spectral density (PSD): power distribution at each frequency band, commonly used for energy features of alpha wave, beta wave and theta wave, alpha wave is 8-12 Hz, beta wave is 13-30 Hz, and theta wave is 4-7 Hz,

[0018] Spectral centroid: the centroid position of the spectrum, representing the center frequency of the spectrum,

[0019] Frequency bandwidth: frequency range of alpha wave and beta wave;

[0020] The electroencephalogram nonlinear features include:

[0021] Approximate entropy (ApEn): measures the complexity and irregularity of the signal,

[0022] Sample entropy (SampEn): similar to approximate entropy, but the calculation is more stable,

[0023] Fractal dimension: self-similarity and complexity characteristics of the signal.

[0024] Further, the electrocardiogram time domain features include:

[0025] RR interval: the time interval between two adjacent R-wave peaks,

[0026] HRV time-domain indices: including standard deviation (SDNN), mean RR interval (mRR), standard deviation of adjacent RR intervals (SDSD), QT interval: the time interval from the beginning of Q wave to the end of T wave;

[0027] The ECG frequency-domain features include:

[0028] HRV frequency-domain indices: including low frequency (LF) of 0.04-0.15 Hz, high frequency (HF) of 0.15-0.4 Hz, and ratio LF / HF;

[0029] Spectral centroid: the centroid position of the ECG signal spectrum,

[0030] Band energy: the energy within the frequency band LF and HF;

[0031] The ECG nonlinear features include:

[0032] Approximate entropy (ApEn): measures the complexity of the ECG signal,

[0033] Sample entropy (SampEn): the irregularity of the ECG signal.

[0034] Further, the gastric electrical time-domain features include:

[0035] Fundamental frequency: the main frequency component of the gastric electrical signal, generally 3 times / minute (0.05 Hz),

[0036] Waveform amplitude: the maximum and minimum amplitude of the gastric electrical signal,

[0037] Slow wave period: the periodicity of the slow wave;

[0038] The gastric electrical frequency-domain features include:

[0039] Power spectral density (PSD): the energy distribution of the gastric electrical signal in the frequency domain,

[0040] Spectral peak: the main frequency peak of the gastric electrical signal,

[0041] Frequency bandwidth: the frequency range within the specified frequency band;

[0042] The gastric electrical nonlinear features include:

[0043] Approximate entropy (ApEn): the complexity of the gastric electrical signal,

[0044] Sample entropy (SampEn): the irregularity of the gastric electrical signal,

[0045] Fractal dimension: the self-similarity of the gastric electrical signal;

[0046] Further, the multi-input deep learning network of the feature fusion unit is composed of a convolutional neural network and a long short-term memory network.

[0047] Further, the emotion recognition unit is a deep learning network, and the emotion evaluation unit is a machine learning network.

[0048] Further, the deep learning network of the emotion recognition unit is a fully connected layer and an activation function, and the machine learning network of the emotion evaluation unit is a support vector machine (SVM).

[0049] By integrating electroencephalogram (EEG), electrocardiogram (ECG), and innovatively incorporating electrogastric (EGG) signals closely related to nervous system activity, the system can provide comprehensive emotion evaluation. These signals each represent different emotional response mechanisms, and the fusion of multi-modal signals can more accurately capture emotional changes, improving the accuracy and reliability of emotion detection. The system can collect and analyze users' physiological signals in real time, providing immediate emotional feedback, which can help users identify emotional fluctuations early and take appropriate measures to avoid emotional out-of-control or long-term emotional depression. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is a workflow diagram.

[0051] Figure 2 is a working principle diagram for identifying emotions using CNN-LSTM. DETAILED DESCRIPTION

[0052] To achieve the above purpose, the wearable electroencephalogram, electrocardiogram, and electrogastric acquisition module is used to measure the nervous system electrical signals of the human body to obtain electroencephalogram, electrocardiogram, and electrogastric signals, and measure the characteristic parameters related to emotions in the signals. Then, a multi-input deep learning algorithm is used for feature fusion, and finally, a machine learning (ML) and deep learning (DL) algorithm is used for emotion classification and evaluation.

[0053] The extracted features in the signals include electroencephalogram average potential: the average electroencephalogram activity level within a certain time.

[0054] The extracted features in the signals include electroencephalogram peak potential: the potential value of the highest and lowest points.

[0055] The extracted features in the signals include electroencephalogram waveform area: the area of the signal waveform within a certain time.

[0056] The extracted features in the signals include electroencephalogram wave amplitude: the difference between the maximum amplitude and the minimum amplitude of the signal.

[0057] The features extracted from the signals include electroencephalogram power spectral density (PSD): power distribution over various frequency bands, often used for energy features of alpha waves (8-12 Hz), beta waves (13-30 Hz), theta waves (4-7 Hz).

[0058] The features extracted from the signals include electroencephalogram spectral centroid: the centroid position of the spectrum, representing the center frequency of the spectrum.

[0059] The features extracted from the signals include electroencephalogram frequency bandwidth: the frequency range within a certain frequency band, such as the bandwidth of alpha waves, beta waves.

[0060] The features extracted from the signals include electroencephalogram approximate entropy (ApEn): measures the complexity and irregularity of the signal.

[0061] The features extracted from the signals include electroencephalogram sample entropy (SampEn): similar to approximate entropy, but the calculation is more stable.

[0062] The features extracted from the signals include electroencephalogram fractal dimension: self-similarity and complexity features of the signal.

[0063] The features extracted from the signals include electrocardiogram RR interval: the time interval between the peaks of two adjacent R waves.

[0064] The features extracted from the signals include electrocardiogram HRV time domain indicators: including standard deviation (SDNN), mean RR interval (mRR), standard deviation of adjacent RR interval differences (SDSD), etc.

[0065] The features extracted from the signals include electrocardiogram QT interval: the time interval from the start of the Q wave to the end of the T wave.

[0066] The features extracted from the signals include electrocardiogram HRV frequency domain indicators: including low frequency (LF, 0.04-0.15 Hz), high frequency (HF, 0.15-0.4 Hz) and LF / HF ratio.

[0067] The features extracted from the signals include electrocardiogram spectral centroid: the centroid position of the electrocardiogram signal spectrum.

[0068] The features extracted from the signals include electrocardiogram band energy: energy within a certain frequency band (such as LF, HF).

[0069] The features extracted from the signals include electrocardiogram approximate entropy (ApEn): measures the complexity of the electrocardiogram signal.

[0070] The features extracted from the signals include electrocardiogram sample entropy (SampEn): irregularity of the electrocardiogram signal.

[0071] The features extracted from the signal include gastric electrical basic frequency: the main frequency component of the gastric electrical signal, typically 3 cycles / minute (0.05 Hz).

[0072] The features extracted from the signal include gastric electrical waveform amplitude: the maximum and minimum amplitude of the gastric electrical signal.

[0073] The features extracted from the signal include gastric electrical slow wave period: the periodicity feature of the slow wave.

[0074] The features extracted from the signal include gastric electrical power spectral density (PSD): the energy distribution of the gastric electrical signal in the frequency domain.

[0075] The features extracted from the signal include gastric electrical spectral peak: the main frequency peak of the gastric electrical signal.

[0076] The features extracted from the signal include gastric electrical frequency bandwidth: the frequency range within a certain frequency band.

[0077] The features extracted from the signal include gastric electrical approximate entropy (ApEn): the complexity of the gastric electrical signal.

[0078] The features extracted from the signal include gastric electrical sample entropy (SampEn): the irregularity of the gastric electrical signal.

[0079] The features extracted from the signal include gastric electrical fractal dimension: the self-similarity of the gastric electrical signal.

[0080] Feature fusion using CNN-LSTM algorithm: (CNN part) Spatial feature extraction: Convert the preprocessed physiological signal into a two-dimensional form suitable for CNN processing (such as time-frequency graph). CNN is used to extract spatial features from these two-dimensional representations, such as local patterns, spectral features, etc. Convolutional layer: Use convolutional and pooling layers to extract and compress features to capture important patterns and features in the signal.

[0081] Time series modeling (LSTM part) Sequence modeling: Input the feature sequence obtained from the CNN part into the LSTM network. LSTM handles the temporal dependencies in these sequences and captures the dynamic features of the emotional state over time. Long-term dependencies: LSTM can remember and update long-term dependencies, which is very important for capturing changes in emotional state.

[0082] Emotion classification: Classification layer: After the LSTM layer, a fully connected layer and an activation function are usually added to map the extracted features to specific emotion categories. Prediction: Use the trained model to predict emotions, output the emotion category (such as happy, sad, angry, etc.) corresponding to the physiological signal.

[0083] Emotion evaluation using ML, DL algorithms (e.g. SVM). SVM is a supervised learning algorithm, particularly suitable for classification tasks. After selecting features, choose kernel function, SVM can use different kernel functions (e.g. linear kernel, polynomial kernel, radial basis function (RBF), etc.). Selecting a kernel function suitable for the data can improve classification performance. Train the model: Use labeled emotion data (containing physiological signals with known emotion labels) to train the SVM model. The model will learn how to predict emotion intensity based on input features.

[0084] Emotion recognition using machine learning models (ML) and deep learning (DL) models. Machine learning models: such as support vector machines (SVM) for emotion classification based on integrated features. Deep learning models: such as convolutional neural networks (CNN), recurrent neural networks (RNN), etc., for emotion recognition. Assisted by clinical psychological evaluation methods (such as psychological scale POMS, SCL90) to test the reliability of emotion recognition. Feedback to the user when abnormal or negative emotions and psychological states are detected.

[0085] Parameter tuning: Adjust the hyperparameters of SVM (such as C parameter and kernel function parameter) to optimize the performance of the model. Usually use cross-validation to select the best parameter combination. Use the trained SVM model to make emotion prediction on new physiological signal data. Input feature vector, model output predicted emotion category.

Claims

1. A wearable emotion monitoring and assessment system based on electrical signals from the brain-heart-gut nervous system, the system comprising: a signal acquisition and preprocessing unit, a feature extraction unit, a feature fusion unit, an emotion recognition unit, and an emotion assessment unit; The signal acquisition and preprocessing unit includes: The system acquires and preprocesses signals from electroencephalogram (EEG), electrocardiogram (ECG), and gastric electroencephalogram (GEG), and then transmits the preprocessed signals to the feature extraction unit. The feature extraction unit extracts features from the received EEG, ECG, and gastric electroencephalogram signals and transmits the extracted features to the feature fusion unit. The features extracted by the feature extraction unit include: electroencephalogram (EEG) features, electrocardiogram (ECG) features, and gastric electroencephalogram (GEG) features; The EEG characteristics include: EEG time-domain characteristics, EEG frequency-domain characteristics, and EEG nonlinear characteristics; The electrocardiogram (ECG) characteristics include: ECG time-domain characteristics, ECG frequency-domain characteristics, and ECG nonlinear characteristics; The gastric electrical characteristics include: gastric electrical time-domain characteristics, gastric electrical frequency-domain characteristics, and gastric electrical nonlinear characteristics; The feature fusion unit is a multi-input deep learning network that fuses all features and then inputs the fused features into the emotion recognition unit and the emotion evaluation unit. The emotion recognition unit identifies the type of emotion, and the emotion assessment unit identifies the intensity of the emotion.

2. The wearable emotion monitoring and assessment system based on brain-cardio-gut nervous system electrical signals as described in claim 1, characterized in that, The EEG temporal features include: Average potential: The average level of brain electrical activity over a given time period. Peak potential: The potential values ​​at the highest and lowest points. Waveform area: The area of ​​a signal waveform over a given time period. Brainwave amplitude: The difference between the maximum and minimum amplitude of a signal; The EEG frequency domain features include: Power spectral density: the power distribution across various frequency bands, commonly used to characterize the energy of alpha, beta, and theta waves. Alpha waves are 8-12 Hz, beta waves are 13-30 Hz, and theta waves are 4-7 Hz. Spectral centroid: The location of the centroid of the spectrum, representing the center frequency of the spectrum. Frequency bandwidth: the frequency range of alpha and beta waves; The nonlinear characteristics of the EEG include: approximate entropy, sample entropy, and fractal dimension.

3. The wearable emotion monitoring and assessment system based on brain-cardio-gut nervous system electrical signals as described in claim 1, characterized in that, The electrocardiogram time-domain features include: RR interval: the time interval between two consecutive R wave peaks. HRV time-domain metrics include standard deviation, mean RR interval, and standard deviation of the difference between adjacent RR intervals. QT interval: The time interval from the beginning of the Q wave to the end of the T wave; The electrocardiogram frequency domain features include: HRV frequency domain specifications: including low frequency (LF) of 0.04-0.15Hz, high frequency (HF) of 0.15-0.4Hz, and the ratio LF / HF; Centroid of the spectrum: The location of the centroid of the electrocardiogram (ECG) signal spectrum. Bandwidth energy: Energy within the LF and HF bands; The nonlinear characteristics of the electrocardiogram include: approximate entropy and sample entropy.

4. The wearable emotion monitoring and assessment system based on brain-cardio-gut nervous system electrical signals as described in claim 1, characterized in that, The gastric electrical time-domain characteristics include: fundamental frequency, waveform amplitude, and slow wave period; The gastric electrical frequency domain characteristics include: power spectral density, spectral peak value, and frequency bandwidth; The nonlinear characteristics of gastric electrophysiology include: approximate entropy, sample entropy, and fractal dimension.

5. The wearable emotion monitoring and assessment system based on brain-cardio-gut nervous system electrical signals as described in claim 1, characterized in that, The multi-input deep learning network of the feature fusion unit consists of a convolutional neural network and a long short-term memory network.

6. The wearable emotion monitoring and assessment system based on brain-cardio-gut nervous system electrical signals as described in claim 1, characterized in that, The emotion recognition unit is a deep learning network, and the emotion evaluation unit is a machine learning network.

7. The wearable emotion monitoring and assessment system based on brain-cardio-gut nervous system electrical signals as described in claim 1, characterized in that, The deep learning network of the emotion recognition unit consists of a fully connected layer and an activation function; the machine learning network of the emotion evaluation unit is a support vector machine (SVM).

Images