Anxiety state recognition method based on multi-mode signal analysis

Abstract

The invention provides an anxiety state recognition method based on a multi-modal signal. A convolutional neural network (CNN), a gated recursive unit (GRU) and a deep learning model of a frequency band attention mechanism are combined. According to the method, spatial and temporal features are extracted from EEG and HR signals and weighted, key features related to anxiety emotion can be effectively captured, and accurate anxiety state classification is carried out. The method comprises the following steps: firstly, extracting each frequency band characteristic of an EEG signal through wavelet transform or short-time Fourier transform, and calculating information quantity indexes such as Shannon entropy and the like; the heart rate signal provides physiological information of emotional fluctuation by calculating heart rate variability (HRV) characteristics. Then, the CNN is used for extracting spatial features of the EEG signals, a frequency band attention mechanism dynamically weights features of all frequency bands, the GRU captures time sequence dependence of the EEG and the heart rate signals, and finally the anxiety state is output through a classification layer. According to the method, accurate real-time monitoring and intervention of the anxiety state can be realized, and powerful technical support is provided for emotion health management.

Description

Technical Field

[0001] This invention belongs to the field of biomedical signal processing and emotion recognition technology, specifically relating to a method for anxiety state recognition based on multimodal signal analysis, particularly a deep learning model combining convolutional neural networks (CNN), gated recurrent units (GRU), and bandgap attention mechanisms. This method can efficiently extract spatiotemporal features from multimodal physiological signals and accurately determine anxiety states, making it suitable for emotion monitoring and intervention in adolescents. Background Technology

[0002] With the rapid development of modern society, adolescents face pressure from academics, family, and social aspects, making anxiety a significant factor affecting their physical and mental health. If anxiety is not identified and effectively intervened in a timely manner, it can lead to serious consequences such as mental disorders and depression. Traditional methods for identifying anxiety typically rely on self-reporting, questionnaires, or clinical interviews, which suffer from high subjectivity and low efficiency. In recent years, automated methods for identifying anxiety states based on biomedical signals (such as electroencephalography (EEG) and heart rate (HR)) have been extensively studied.

[0003] Electroencephalography (EEG) is a non-invasive method for monitoring brain function, capable of recording the electrical activity of the cerebral cortex in real time, with high spatial and temporal resolution. Different frequency bands in EEG signals (such as theta waves, alpha waves, beta waves, and gamma waves) exhibit different patterns of change under different emotional states. Heart rate (HR) is closely related to changes in the autonomic nervous system; anxiety is typically accompanied by an increased heart rate and decreased variability. Therefore, combining EEG and heart rate signals and utilizing deep learning methods to accurately classify anxiety states in adolescents has significant research and application value.

[0004] Currently, anxiety state identification based on EEG and heart rate signals mostly employs traditional machine learning methods, such as Support Vector Machines (SVM) and Random Forests (RF). However, these methods rely on manual feature extraction and struggle to fully uncover deep-seated features when dealing with complex time-series data and multimodal signals. With the development of deep learning, fusion methods based on models such as Convolutional Neural Networks (CNN) and Long Short-Term Memory Networks (LSTM) have gradually become a research hotspot, but existing technologies still face the challenge of effectively capturing the spatiotemporal features of EEG and heart rate signals.

[0005] Therefore, the goal of this invention is to propose a deep learning model based on CNN, GRU and bandgap attention mechanism for the accurate identification of anxiety states in adolescents, and to overcome the limitations of existing technologies. Summary of the Invention

[0006] (a) Technical problems to be solved

[0007] While research on emotion recognition based on multimodal physiological signals such as EEG and heart rate has made some progress, current technologies are inefficient in signal fusion, often suffering from insufficient feature extraction and fusion, and excessive computational overhead. Existing emotion recognition methods often fail to effectively capture the temporal dependencies in EEG and heart rate signals. Emotional states (especially anxiety) typically change over time, and traditional methods struggle to model this temporal dynamics. Models lacking temporal modeling capabilities perform poorly when processing complex physiological signals and cannot accurately identify key moments in emotional fluctuations. To address these issues, this invention proposes a deep learning method based on CNN, GRU, and bandgap attention mechanisms, aiming to improve the accuracy and stability of anxiety state recognition while enhancing the efficiency of multimodal data fusion and fully exploiting the spatiotemporal features of EEG and heart rate signals.

[0008] (II) Technical Solution

[0009] This invention provides a method for anxiety state recognition based on multimodal signal analysis. This method can automatically extract spatiotemporal features from EEG and heart rate signals, and dynamically weight feature information from different frequency bands through a frequency band attention mechanism, thereby achieving accurate classification of anxiety states.

[0010] To achieve the above objectives, the present invention adopts the following technical solution:

[0011] An anxiety state recognition method based on multimodal signal analysis includes the following steps:

[0012] 1. Physiological signal preprocessing

[0013] The EEG signals were denoised and artifact removed. Bandpass filters were used to remove low-frequency drift and high-frequency noise, and independent component analysis was used to remove artifacts such as electrooculography and electromyography to ensure data quality.

[0014] Wavelet transform is used to remove artifacts from the heart rate signal, resulting in stable heart rate data.

[0015] 2. Feature Extraction

[0016] For the preprocessed signal, chaotic features at three time scales are extracted:

[0017] Short-time Fourier transform (STFT) is performed on the EEG signal to extract the power spectral density (PSD) of different frequency bands of the theta, alpha, beta, and gamma waves.

[0018] The Shannon entropy and differential entropy features of each frequency band are calculated and combined with the heart rate V feature of the heart rate signal to obtain multimodal features.

[0019] 3. Convolutional Neural Networks: Convolutional Neural Networks (CNNs) are used to extract spatial features from the frequency band features of EEG signals.

[0020] 4. Frequency band attention mechanism: Introducing a frequency band attention mechanism, automatically assigning weights to different frequency bands, focusing on frequency band features related to anxiety.

[0021] 5. GRU Temporal Modeling: Input the weighted features into the GRU to model the temporal changes of EEG and heart rate signals.

[0022] 6. Anxiety State Classification and Output: The features output by GRU are fed into a fully connected layer for classification, and the final output is the predicted result of anxiety state.

[0023] (III) Beneficial Effects

[0024] Compared with the prior art, the present invention has the following advantages:

[0025] The method for identifying adolescent anxiety states based on CNN, GRU, and frequency band attention mechanism provided by this invention has the following significant beneficial effects:

[0026] 1. Improve the accuracy of anxiety state recognition:

[0027] This invention introduces a frequency band attention mechanism, which automatically weights features from different frequency bands (such as theta, alpha, beta, and gamma waves), allowing the model to focus more on frequency bands closely related to anxiety. This dynamic weighting mechanism significantly improves the accuracy of anxiety state recognition, especially for potential differences in frequency band features among adolescents, and can adaptively adjust the focus.

[0028] 2. Effective fusion of multimodal physiological signals:

[0029] This invention combines EEG and heart rate signals, fully utilizing the complementary information provided by these two signals in anxious states. EEG signals primarily reflect changes in brain activity, while heart rate signals reflect the regulatory state of the heart and autonomic nervous system. By extracting spatial features from EEG signals using CNN and modeling the temporal dependencies of the signals using GRU, the model can effectively fuse these multimodal physiological signals, improving the comprehensiveness and accuracy of emotion recognition.

[0030] 3. Enhanced ability to model temporal dependencies:

[0031] The advantages of GRU (Gated Recurrent Unit) in time-series data modeling enable this invention to capture long-term dependencies in EEG and heart rate signals, especially during changes in anxiety, where mood fluctuations often exhibit clear time-series characteristics. Through GRU, the model can gain a deeper understanding of the dynamic evolution of anxiety states, thereby improving the stability and robustness of emotion classification. Attached Figure Description

[0032] Figure 1 A flowchart illustrating the overall framework of an anxiety state recognition method based on multimodal signal analysis, provided for the implementation of this invention; Detailed Implementation

[0033] As attached Figure 1 As shown, the overall process of an anxiety state recognition method based on multimodal signal analysis is as follows:

[0034] 1. Signal preprocessing

[0035] (1) Preprocessing of electroencephalogram (EEG) signals

[0036] Bandpass filtering: EEG signals are bandpass filtered from 0.5 to 50 Hz to remove power frequency noise and other high-frequency noise. The formula for a bandpass filter is:

[0037]

[0038] Where, ω c s is the cutoff frequency, and s is the complex frequency domain, which can preserve the effective frequency band of brain electrical activity.

[0039] Artifact removal: Independent component analysis (ICA) is used to remove common artifact signals such as electrooculography (EOG) artifacts and electromyography (EMG) artifacts.

[0040] (2) Heart rate signal preprocessing

[0041] First, artifact removal is performed on the heart rate signal to remove noise caused by motion or equipment errors. Then, the heart rate signal is standardized to ensure it meets the requirements for subsequent analysis.

[0042] 2. Feature Extraction

[0043] Power spectral density calculation:

[0044] After preprocessing, the EEG signal is decomposed into multiple frequency bands, such as theta waves, alpha waves, beta waves, and gamma waves. Each frequency band represents the brain's activity pattern in different states. This is achieved by performing STFT decomposition on the EEG signal to separate different frequency bands.

[0045]

[0046] Where x(t) is the time-domain signal, w(t) is the window function, and τ and f are the time and frequency, respectively.

[0047] Next, power spectral density calculations were performed for each frequency band. Common frequency bands include: theta wave: 4-8Hz, alpha wave: 8-13Hz, beta wave: 13-30Hz, and gamma wave: 30-40Hz.

[0048] Calculate the power spectral density for each frequency band:

[0049]

[0050] in, For Fourier transform, f is the frequency, and PSD(f) is the spectral density.

[0051] Shannon entropy calculation:

[0052] Shannon entropy is used to measure the complexity of a signal; a higher entropy value indicates a more complex signal. The formula is:

[0053]

[0054] Where p(x) i ) represents the probability of the i-th value in the signal, and n represents the number of discrete values ​​in the signal.

[0055] Differential entropy calculation:

[0056] Differential entropy is used to calculate the complexity of continuous signals; a higher differential entropy value indicates a more complex signal. The formula is:

[0057]

[0058] Where p(x) is the probability density function of the signal.

[0059] The extracted EEG features are combined into a one-dimensional vector, which serves as the input to the CNN. The input data has an 8×12×1 shape, representing the feature matrix for each time window. Here, 8 represents the 8 channels of the EEG signal, 12 represents the number of features per channel (4 frequency bands × 3 features), and 1 represents the grayscale image of a single channel.

[0060] Heart rate variability:

[0061] The R wave is detected from the heart rate signal, and the time difference between adjacent R waves is calculated to obtain the RR interval. Assume the acquired R wave positions are R1, R2, R3, ... R n Then the RR interval is:

[0062] RR i =R i+1 -R i

[0063] Among them, R i Let RR be the time position of the i-th R wave. i Let be the i-th RR interval.

[0064] Short-term heart rate variability is represented by the root mean square depth (RMSSD) of the difference between adjacent heartbeats. Lower RMSSD is often associated with emotional states such as anxiety and stress.

[0065]

[0066] Where ΔRR i It is the difference between the i-th and (i+1)-th RR intervals, and N is the number of R interval samples.

[0067] The overall variability of heart rate is then represented by the standard deviation (SDNN).

[0068]

[0069] in, is the average of the RR intervals, and N is the sample size of the RR intervals.

[0070] 3. Convolutional Neural Network (CNN)

[0071] Spatial features of EEG signals are extracted using convolutional neural networks (CNNs). CNNs automatically learn local features in EEG signals through multiple convolutional layers, pooling layers, and ReLU activation functions.

[0072] The first convolutional layer uses 5×5 convolutional kernels, with a total of 16 kernels, and performs non-linear mapping using the ReLU activation function; the second convolutional layer uses 3×3 convolutional kernels, with a total of 32 kernels.

[0073]

[0074] Where x(t) is the input signal, w(n) is the convolution kernel, and * denotes the convolution operation.

[0075] Max pooling downsamples the feature map size, reducing computation while preserving important spatial features. After convolutional and pooling layers, the CNN extracts 32-dimensional features.

[0076] CNNs extract high-level features from EEG signals through multiple layers of convolution and pooling, effectively capturing spatial patterns and potential emotional features in the signals.

[0077] 4. Bandwidth attention mechanism

[0078] After the features of the EEG signal are input into the CNN, the band attention mechanism calculates the weight of each band, enabling the model to adaptively focus on the band features related to anxiety.

[0079] Attention weight calculation: The band attention mechanism adaptively learns the weights of each band feature based on the training data. The input feature vector is , where X ∈ [0, 1] ∈ [0, 2] ∈ [0, 3] ∈ [0, 4] ∈ [0, 5] ∈ [0, 6] ∈ [0, 7] ∈ [0, 8] ∈ [0, 9] ∈ [0, 1] ∈ [0, 1] ∈ [ i For the features of the i-th frequency band, the model will calculate an attention weight 'a' for each frequency band. i :

[0080]

[0081] Among them, W f and b f These are the learned weights and biases, and the Softmax activation function ensures the normalization of the weights.

[0082] Weighted features: through attention weights a i The frequency band features are weighted to obtain the weighted feature representation X. att The formula is:

[0083] X att =a·X

[0084] Where a is the attention weight vector, X is the input feature, and X att This represents the weighted frequency band characteristics.

[0085] 5. GRU Temporal Modeling and Classification

[0086] After features are extracted and weighted using CNN and bandgap attention mechanisms, GRU (Gated Recurrent Unit) further models the temporal dependence of the signal, capturing the temporal evolution of emotional states. The extracted HRV features, representing the variability of the heart rate signal, are input as temporal features into the GRU network to capture its dynamic patterns over time and generate a 32-dimensional feature vector. The EEG spatial features, processed by convolutional and pooling layers, are also fed into a GRU network to capture the temporal variations of the EEG signal, further revealing the dynamic changes in anxiety states.

[0087] The update formula for the GRU model is as follows:

[0088] r t =σ(W r ·[h t-1 ,x t ]+b r )

[0089] z t =σ(W z ·[ht-1 ,x t ]+b z )

[0090]

[0091] Where, r t To reset the door, z t To update the gate, h t Let x be the hidden state at the current moment. t The input is the current moment. Through GRU modeling, the time dependence of EEG and heart rate signals is captured, further improving the accuracy of emotion state recognition.

[0092] The time-series features (32-dimensional) of the EEG signal processed by GRU and the time-series features (32-dimensional) of the HRV signal are concatenated to form a 64-dimensional feature vector. The concatenated 64-dimensional feature vector is then input into a fully connected layer, which further processes the features for classification or regression tasks.

[0093] Anxiety state classification and output:

[0094] After extracting spatial features using a CNN, weighting features using a frequency band attention mechanism, and modeling temporal features using a GRU, the output is finally classified using a fully connected layer and a softmax activation function to determine the anxiety state. The predicted anxiety state is either "anxious" or "non-anxious".

[0095] Class = Softmax(W out ·h t +b out )

[0096] Among them, W out and b out The output layer's weights and biases are used, and the Softmax activation function is used to map the results to classification probabilities.

[0097] Through the above steps, the features of EEG and heart rate signals are first extracted spatially using a CNN, then adaptively weighted using a frequency band attention mechanism, and finally temporal modeled using a GRU. This entire process accurately captures the spatiotemporal features related to anxiety. The final classification result is either "anxious" or "non-anxious," thus achieving efficient and accurate identification of anxiety states in adolescents.

Claims

1. A method for anxiety state recognition based on multimodal signal analysis, characterized in that, The method includes the following steps: (1) Signal preprocessing: Denoising and artifact removal were performed on the electroencephalogram (EEG) signal. Bandpass filters were used to remove low-frequency drift and high-frequency noise, and independent component analysis (ICA) was used to remove artifacts such as electrooculography and electromyography to ensure data quality. Wavelet transform was used to remove artifacts from the heart rate (HR) signal to obtain stable heart rate data. (2) Feature extraction of EEG and heart rate signals: including short time Fourier transform (STFT) or wavelet transform (WT) to extract the power spectral density (PSD) of the theta wave, alpha wave, beta wave and γ wave, and calculate Shannon entropy and differential entropy to extract heart rate variability (HRV) features. The root mean square (RMSSD) and standard deviation (SDNN) of the difference between adjacent heartbeats are used to obtain heart rate features related to emotional state. (3) Convolutional Neural Network Feature Learning: Using a convolutional neural network (CNN) to extract spatial features from the frequency band features of the signal after feature extraction, thereby focusing on the frequency band features related to anxiety; (4) Band attention mechanism: Introduce a band attention mechanism to dynamically weight the band characteristics of the EEG signal according to the contribution of each band; (5) Gated Recursive Unit: The weighted features are input into the gated recursive unit (GRU) for time series modeling to capture the time dependence of EEG and heart rate signals; (6) Classification output: The temporal features output by GRU are fed into the fully connected layer for anxiety state classification, and the prediction results of anxiety state are output.

2. The method according to claim 1, wherein, The convolutional neural network (CNN) uses convolutional layers with kernel sizes of 5×5 and 3×3, and uses max pooling to downsample features.

3. The method according to claim 1, wherein, The anxiety state classification uses the Softmax activation function, and the output results include "anxious" or "non-anxious" states.

4. The method according to claim 1, wherein, The feature extraction of the heart rate signal includes calculating heart rate variability (HRV) metrics, including SDNN and RMSSD. SDNN measures the standard deviation of heart rate intervals, and RMSSD measures the root mean square of the difference between adjacent heartbeat intervals.

5. The method according to claim 1, wherein, Feature extraction of the EEG signal includes calculating the power spectral density (PSD), Shannon entropy, and differential entropy for each frequency band.

Images