A method for measuring psychological stress by analyzing EEG signals based on bidirectional long short-term memory network and attention mechanism

Abstract

The present invention discloses a method for measuring psychological stress by analyzing electroencephalogram (EEG) signals based on a bidirectional long short-term memory (LSTM) network and an attention mechanism. First, the EEG signal data set is preprocessed, including sliding window segmentation, frequency band setting, and removal of artifacts and noise to improve the purity and accuracy of the signal. These steps lay the foundation for subsequent feature extraction and model training. Then, a new psychological stress measurement model is constructed, and the model components are a bidirectional long short-term memory (BiLSTM) module and an attention mechanism (Attention) module. Through experiments, the effectiveness of the model on the public data set DREAMER data set is verified, and its recognition accuracy is significantly improved compared with other methods. In addition, the present invention also tests the robustness and generalization ability of the model on the public data set Feeling Emotions and a self-built data set, and the results also prove the effectiveness of the model.

Description

Technical Field

[0001] The invention involves deep learning models and psychological stress measurement, which belongs to the intersection of psychology and computers. Specifically, it is a method for measuring psychological stress by analyzing EEG signals based on a bidirectional long short-term memory network and attention mechanism. Background Art

[0002] Among many physiological signals, EEG is a signal that records changes in scalp potential, which can better reflect a person's mental state and emotional changes. Various studies have shown that there is a certain correlation between EEG and emotions. Given that EEG signals have the ability to present rich information about an individual's mental state related to stress, the exploration of using EEG signals to identify and measure the stress level of subjects has been widely practiced in the academic field. In the study of psychological stress recognition based on EEG and voice signals, it was confirmed that the nonlinear characteristics of EEG can reflect the psychological stress level of the human body, and psychological stress has a significant impact on the characteristics of sound. The vibration frequency and jitter of the vocal cords increased significantly when the tester performed the task, and decreased slightly after the task was completed, but it was difficult to fully recover to the state before the task in a short time. In the study of psychological stress measurement system based on EEG physiological signals, stress induction and test experiments were designed at the same time, and verified for specific populations. Combined with visual analog quantization tables, the test was conducted. The results show that the system can objectively, comprehensively and accurately measure the psychological stress level. As machine learning and deep learning have shown excellent classification task performance in various fields, scholars have begun to apply these advanced frameworks to EEG-based psychological stress measurement classification research. Summary of the invention

[0003] This paper constructs a new psychological stress measurement classification model, which introduces an attention mechanism based on a bidirectional long short-term memory network (BiLSTM). This method captures the temporal information in the electroencephalogram (EEG) signal more comprehensively, thereby more accurately measuring psychological stress. Through experiments, we verified the effectiveness of the model on the DREAMER dataset, and its recognition accuracy was significantly improved compared with other methods. In addition, we also used the FeelingEmotions public dataset and self-built dataset to test the robustness and generalization ability of the model, and the results also proved the superior performance of the model.

[0004] The present invention is achieved through the following technical solutions:

[0005] A method for measuring psychological stress by analyzing EEG signals based on a bidirectional long short-term memory network and an attention mechanism comprises the following steps:

[0006] Step 1: First, preprocess the EEG signal data set and use a 2-second window to perform sliding window segmentation to ensure that the data in each window has relatively independent meaning, which is convenient for subsequent feature extraction and analysis. Next, filter the segmented data to filter out low-frequency noise and high-frequency interference.

[0007] Step 2: Extract features from the EEG signal after filtering, including time domain features, frequency domain features and differential entropy features.

[0008] Step 3: Construct a psychological stress measurement model. The model introduces the attention mechanism (Attention) based on the bidirectional long short-term memory network (BiLSTM) to form the final psychological stress measurement classification model.

[0009] Step 4: Input the preprocessed EEG data and extracted features into the model for testing, and use precision, recall, and F1-Score as the main evaluation indicators to obtain the model test results.

[0010] Step 5: Test the robustness and generalization ability of the model on another public dataset and a self-built dataset.

[0011] Furthermore, the datasets are the public datasets DREAMER, Feeling Emotions, and self-built datasets. DREAMER is a multimodal database consisting of electroencephalogram (EEG) and electrocardiogram (ECG) signals recorded during emotional stimulation through audio-visual stimulation. The signals of 23 participants were recorded, as well as the participants' self-assessment of their emotional states after each stimulation, including valence, arousal, etc. This dataset divides the subjects' mental states into relaxation, tension, and normal.

[0012] The public data set Feeling Emotions included in the database Kaggle was collected from two subjects (1 male and 1 female). The researchers used a Muse EEG headband to record the EEG signals of the EEG placement positions (TP9, AF7, AF8 and TP10) through electrodes. 3 minutes of data were collected for each stress level (low extreme, medium level, high level), and six minutes of rest neutral data were also recorded. The self-built data set collection process of the present invention is as follows: 20 industrial field workers were invited to fill in the human psychological stress measurement questionnaire based on the Stress Perception Scale (PSS), and the psychological stress situation was preliminarily obtained through the questionnaire scores. Two workers were selected from the three intervals of the questionnaire results "low psychological stress level", "medium psychological stress level" and "high psychological stress level" for experiments (considering the differences between men and women, the two people participating in the experiment in each interval were one man and one woman). Six industrial workers were asked to wear EEG acquisition equipment while working. The equipment software displayed the changes of the five EEG channel signals (TP9, AF7, AF8, TP10, Right AUX) of the subjects in real time during the test, and the experimental data set was exported by the software background.

[0013] Furthermore, the model design chooses to introduce the attention mechanism based on the bidirectional long short-term memory network (BiLSTM) to construct the final psychological stress measurement model. The long short-term memory recurrent network (LSTM) is a variant of the traditional recurrent neural network. The introduction of this network overcomes the problem that the gradient may disappear or explode when the time series is longer and the feedback is adjusted. LSTM defines the cell state variable. The core idea is to allow information to pass selectively and continuously remove or add information to the cell state. This goal is achieved through three gating units: input gate, output gate and forget gate.

[0014] The core idea of ​​the bidirectional LSTM model is that it not only considers the current state of the input sequence, but also the past state of the input sequence. Specifically, the bidirectional LSTM has two main components: a forward LSTM and a backward LSTM. The forward LSTM receives the original input of the input sequence, starts from the first element of the input sequence, processes each element in turn, and outputs the forward representation of each element. The backward LSTM receives the reversed copy of the input sequence, starts from the last element of the input sequence, processes each element in turn, and outputs the backward representation of each element. The outputs of the two LSTMs are connected to form a context vector, which contains all the information of the input sequence. This context vector can be provided to the fully connected layer or other types of output layers. The core idea of ​​the attention mechanism is to assign different weights to each part in order to process different parts of the input data so that they can be weighted and summarized in the calculation of the model. This enables the model to better handle complex sequence data, images and other types of information, and improve the performance and generalization ability of the model.

[0015] Furthermore, the preprocessed EEG signal data is input into the BiLSTM+Attention model, and the output out and hidden state h are obtained through the bidirectional LSTM layer. The output out is paid attention to by the attention layer to obtain the attention weights, which are multiplied with the output out and summed to obtain the attention output. Finally, the attention output is processed by the fully connected layer and the activation function relu to obtain the final measured three-category output of psychological stress.

[0016] Compared with the prior art, the main advantages of the present invention include:

[0017] 1. A new psychological stress measurement and classification model was constructed by introducing the attention mechanism based on the bidirectional long short-term memory network, and the model achieved excellent results.

[0018] 2. Test the effectiveness of the model on the public datasets DREAMER, Feeling Emotions and self-built datasets.

[0019] 3. A self-built EEG signal dataset was constructed based on the EEG signals collected by a portable EEG device. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for describing the embodiments are briefly introduced below.

[0021] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0022] Figure 2 is an architecture diagram of the attention mechanism module of an embodiment of the present invention;

[0023] Figure 3 is a structural diagram of a psychological stress measurement model according to an embodiment of the present invention;

[0024] Figure 4 is a flow chart of pressure identification of industrial field workers according to an embodiment of the present invention; DETAILED DESCRIPTION

[0025] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only provided for the convenience of explanation, and the embodiments described below are only a part of the present invention and should not be construed as being limited to all examples of the above-mentioned manner.

[0026] like Figure 1-4 As shown, a method for measuring psychological stress based on analyzing EEG signals based on a bidirectional long short-term memory network and an attention mechanism includes the following specific steps:

[0027] Step 1: First, the EEG signal data set is preprocessed. The present invention verifies the model algorithm based on the public emotion data sets DREAMER, Feeling Emotions and self-built data sets.

[0028] Data preprocessing plays an important role in obtaining meaningful information about the signal, involving the problem of noise and artifacts. Typically, brain waves and external artifacts (such as electrode bursts, motion artifacts, environmental noise, and wiring noise in EEG sensors) have different frequencies. Most external artifacts can be effectively removed by filtering outside the range of the EEG signal. For example, a bandpass filter with a higher cutoff frequency of 64 Hz and a lower cutoff frequency of 0.5 Hz can remove most external artifacts that cause slow and fast changes in the EEG signal. The criteria for designing the cutoff frequency of the bandpass filter are: based on the frequency range of rhythmic brain potentials detected by surface electrodes placed on the head (e.g., delta waves 0.5-4 Hz, theta waves 4-7.5 Hz, alpha waves 7.5-13 Hz, low beta waves 13-15 Hz, beta waves 15-20 Hz, high beta waves 20-38 Hz, gamma waves 38-high Hz). The present invention uses a 1Hz-40Hz filter to divide the original EEG signal, removes high-frequency and low-frequency noise in the experimental data set, and uses independent component analysis (ICA) to identify and remove inherent artifacts in the EEG recording signal. The ICA method can separate the inherent artifacts in the EEG signal by identifying the artifact components and removing the artifact-related parts while retaining the integrity of the original EEG signal, thereby obtaining a pure EEG signal. The data set is preprocessed by the above method to remove a large amount of artifacts and noise in the data set, and the subsequent model psychological stress classification research is carried out.

[0029] Step 2: EEG signal feature selection and extraction. The main goal of feature extraction is to obtain information that can effectively reflect the emotional state of an individual. These extracted features play an important role in the accuracy of emotion recognition. EEG signal features are divided into four categories: time domain, frequency domain, time-frequency domain, and nonlinear features.

[0030] At present, commonly used time domain feature analysis methods include histogram analysis, event-related potentials, higher-order crossover (HOC), and Higuchi's fractal dimension method for measuring signal self-similarity and complexity. In addition, statistical features such as mean, standard deviation, skewness, variance, kurtosis, etc. can also be used for analysis; frequency domain features are obtained by calculating the time domain signal using discrete Fourier transform, among which typical features include power spectral density (PSD), logarithmic energy spectrum, higher-order spectrum (HOS), etc.; time-frequency domain features generally segment the signal by windowing, and perform time-frequency domain signal transformation on each sub-signal to combine the advantages of time domain and frequency domain, thereby improving the processing ability of unstable signals. One way to divide data into different windows is to use a fixed window method, that is, to define the entire action sequence data as a continuous single segment. Then, features are extracted from these window segments and used in machine learning algorithms to classify test segments with fixed lengths. This ensures that each feature can capture the information of the entire action sequence during training. Another alternative to the fixed window method is to use a sliding window method to segment the data.

[0031] EEG signal is a non-stationary signal with nonlinear and non-periodic characteristics. Using traditional linear analysis methods for analysis may lead to the loss of original information. Therefore, nonlinear dynamic analysis methods are used to study EEG signals to reveal their more comprehensive characteristics. Commonly used nonlinear features of EEG signals include various entropies such as permutation entropy, approximate entropy, singular value decomposition entropy, power spectrum entropy and sample entropy, as well as correlation dimension and fractal dimension. The present invention extracts DE features, frequency domain features-power spectrum density, time domain features-macro average and weighted average to classify psychological stress.

[0032] Step 3: Construction of BiLSTM+Attention model for psychological stress measurement. This model introduces the attention mechanism (Attention) based on the bidirectional long short-term memory network (BiLSTM) to form the final psychological stress measurement classification model.

[0033] The attention mechanism is a technology that simulates the human visual and cognitive system, which enables the neural network to selectively focus on the task-related parts when processing input data. After the introduction of the attention mechanism, the neural network can automatically learn and focus on the key information in the input in a targeted manner, improving the performance and generalization ability of the model. The framework diagram is as follows Figure 2As shown in the figure, with the wide use of the Attention mechanism in computer vision, natural language processing and other fields and the good results achieved, this paper attempts to add the attention mechanism on the basis of the bidirectional long short-term memory network model to process the EEG signal sequence and extract features that more effectively reflect stress, and conduct experiments. The overall construction architecture of the psychological stress measurement model BiLSTM+Attention is shown in the figure. Figure 3 As shown, the EEG data is first preprocessed, and a 2-second window is used for sliding window segmentation to ensure that the data in each window has relatively independent meaning, which is convenient for subsequent feature extraction and analysis. Next, the segmented data is filtered to filter out low-frequency noise and high-frequency interference. In this process, the frequency band of 1-40Hz is focused on because it is a frequency band closely related to EEG activity. Through filtering, the required frequency band can be extracted, and most artifacts and noise can be removed at the same time, thereby improving the purity and accuracy of the data. After that, feature extraction is performed on the EEG signal after filtering. The present invention extracts features from the time domain and the frequency domain respectively to form a feature table. The time domain features mainly include the amplitude and waveform of the signal; the frequency domain features include the frequency components and energy distribution of the signal. These features can reflect the activity of the brain in different states and provide strong support for subsequent classification and recognition. In order to better represent and process these features, we input them into a deep model for further processing.

[0034] The deep model of the present invention uses a bidirectional LSTM (Bidirectional LSTM, BiLSTM). The core idea of ​​the bidirectional LSTM model is that it not only considers the current state of the input sequence, but also the past state of the input sequence. Specifically, the bidirectional LSTM has two main components: a forward LSTM and a backward LSTM. The forward LSTM receives the original input of the input sequence, starts from the first element of the input sequence, processes each element in turn, and outputs the forward representation of each element. The backward LSTM receives a reversed copy of the input sequence, starts from the last element of the input sequence, processes each element in turn, and outputs the backward representation of each element. The outputs of the two LSTMs are connected to form a context vector, which contains all the information of the input sequence. This context vector can be provided to the fully connected layer or other types of output layers. After the bidirectional LSTM layer is processed to obtain the output out and the hidden state h, the output is then calculated by the attention layer to obtain the attention weights. The attention weights are multiplied and summed with the output out to obtain the attention output. This process can further enhance the key information that the model pays attention to, while suppressing those information that are considered less relevant. Finally, the attention output is processed by the fully connected layer and the activation function relu to obtain the final three-category output.

[0035] Step 4: Model validity verification. In the current emotion recognition classification task, researchers usually use accuracy (ACC) and standard deviation (STD) as the main evaluation indicators. In addition to accuracy and standard deviation, precision, recall, F1-Score and confusion matrix can also be used to evaluate the emotion recognition model.

[0036] The present invention selects multiple evaluation indicators in machine learning to evaluate model performance, and compares the emotion classification results of LSTM model with and without attention mechanism and BiLSTM model on the DREAMER dataset. The LSTM model without attention mechanism has an average precision and recall rate of 73% and 72% respectively for positive emotion (Positive Valence, HV) and negative emotion (Negative Valence, LV) classification in the valence dimension, and the F1-score is 73%. In terms of arousal, the precision and recall rate of HV and LV classification are 73% and 74% respectively, and the F1-source is 74%.

[0037] Table 1 Experimental results of LSTM on the DREAMER dataset

[0038]

[0039] The LSTM model with attention mechanism achieves 85% and 85% precision and recall for HV and LV classification respectively, and 85% F1-score for valence, while 84% and 86% precision and 86% recall for HV and LV classification respectively, and 85% F1-score for arousal.

[0040] Table 2 Experimental results of LSTM+Attention on the DREAMER dataset

[0041]

[0042]

[0043] The experimental results of the present invention on the BiLSTM model with the Attention module are as follows: in the valence dimension, the precision and recall rate of HV and LV classification are 86% and 86% respectively, and the F1-score is 86%, while in terms of arousal, the precision and recall rate of HV and LV classification are 86% and 87% respectively, and the F1-score is 86%, as shown in Table 3.

[0044] Table 3 Experimental results of BiLSTM+Attention on the DREAMER dataset

[0045]

[0046] From the above comparative experimental results, we can see that the bidirectional LSTM model with attention mechanism performs better than other control groups on the DREAMER dataset, and its precision and recall F1-score in terms of valence and arousal are better than other control models.

[0047] Step 5: Test the robustness and generalization ability of the model on other datasets. The first is the public dataset Feeling Emotions included in the database Kaggle. This data was collected from two subjects (1 male and 1 female). The researchers used a Muse EEG headband to record the EEG signals of the EEG placement positions (TP9, AF7, AF8 and TP10) through electrodes, and collected 3 minutes of data for each stress level (low extreme, medium level, high level). Six minutes of resting neutral data were also recorded. The stimuli used to stimulate emotions were: Negative label movies-"Death scene" of the characters in "Marleyand Me", "Funeral scene" in "My Girl", Positive label movies-"Opening musical" in "La LaLand", "Time goes back and good life" in "Slow Life" and warm scenes in "Funny Dogs". The precision, recall and F1-score results of the LSTM model without attention on this dataset for the classification test of low hydrostatic pressure (LHP), medium hydrostatic pressure (MHP) and high hydrostatic pressure (HHP) are shown in Table 4:

[0048] Table 4 Experimental results of LSTM on the Feeling Emotion dataset

[0049]

[0050] The performance of the BiLSTM+Attention model on the Feeling Emotions dataset is shown in the following table:

[0051] Table 5 Experimental results of BiLSTM+Attention on the Feeling Emotion dataset

[0052]

[0053] From Tables 4 and 5, we can see that on the public dataset Feeling Emotions, the classification accuracy of the bidirectional LSTM model with attention mechanism is better than that of other control models.

[0054] Step 6: The present invention also tests the model results on a self-built data set. Figure 4The process of stress identification for industrial workers is outlined, aiming to assess the psychological stress of workers in severe working environments. In the first step, a head-mounted EEG device is used to collect EEG signals from five channels: AF3, AF4, T7, T8, and Pz. Based on the results of the Chinese version of the Perceived Stress Scale (CPSS) questionnaire filled out by industrial workers, their EEG signals are marked ( Figure 4 A), then extract the time domain and frequency domain features of the EEG signal (B), select the features that best reflect the individual's emotional state information, and combine the deep learning algorithm (BiLSTM+Attention) to classify the psychological stress of industrial site workers (C).

[0055] In this part of the work, pre-training was performed on the Feeling Emotions dataset, and fine-tuning was performed on the training set of the self-built dataset to better adapt to the data characteristics of specific fields. Finally, the model was verified and evaluated using the test set of the dataset to ensure its effectiveness and generalization ability in real scenarios. The test results of each model on the self-built dataset are shown in Table 6. The classification accuracy, recall rate, and F1-score of the BiLSTM+Attention model are better than those of other models.

[0056] Table 6 Comparison of experimental results of each model on the self-built data set

[0057]

[0058]

[0059] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art of the present invention, several equivalent substitutions or obvious variations can be made without departing from the concept of the present invention, and the performance or use is the same, which should be regarded as belonging to the protection scope of the present invention.

Claims

1. A method for measuring psychological stress by analyzing EEG signals based on a bidirectional long short-term memory network and an attention mechanism, characterized in that: The method includes the following steps: Step 1: First, preprocess the EEG signal data set and use a 2-second window to perform sliding window segmentation to ensure that the data in each window has relatively independent meaning, which is convenient for subsequent feature extraction and analysis; then filter the segmented data to filter out low-frequency noise and high-frequency interference; Step 2: Extract features of the EEG signal after filtering, including time domain features, frequency domain features and differential entropy features; Step 3: Construct a psychological stress measurement model, which consists of a bidirectional long short-term memory network module and an attention mechanism module; Step 4: Input the preprocessed EEG data and extracted features into the model for testing, use precision, recall and F1-Score as the main evaluation indicators, obtain the model test results, and verify the effectiveness of the model; Step 5: Test the robustness and generalization ability of the model on another public dataset and a self-built dataset.

2. According to claim 1, a method for measuring psychological stress by analyzing EEG signals based on a bidirectional long short-term memory network and an attention mechanism is characterized in that: The EEG signal datasets were selected from the public datasets DREAMER, FeelingEmotions and self-built datasets; data preprocessing plays an important role in obtaining meaningful information about the signal, involving the problems of noise and artifacts; a 1Hz-40Hz filter was used to divide the original EEG signal, remove high-frequency and low-frequency noise in the experimental dataset, and use independent component analysis (ICA) to identify and remove inherent artifacts in the EEG recording signal; the ICA method can retain the integrity of the original EEG signal while identifying the artifact components and removing the artifact-related parts, thereby separating the inherent artifacts in the EEG signal and finally obtaining a pure EEG signal; the dataset was preprocessed using the above method to remove a large amount of artifacts and noise in the dataset, and subsequent research on the psychological stress classification of the model was carried out.

3. According to claim 1, a method for measuring psychological stress by analyzing EEG signals based on a bidirectional long short-term memory network and an attention mechanism is characterized in that: Feature extraction is performed on the preprocessed EEG signal data; EEG signal feature selection and extraction, the main goal of feature extraction is to obtain information that can effectively reflect the individual's emotional state, and these extracted features play an important role in the accuracy of emotion recognition; DE features, frequency domain features-power spectral density, time domain features-macro average, and weighted average are extracted respectively for psychological stress measurement and classification.

4. According to claim 1, a method for measuring psychological stress by analyzing EEG signals based on a bidirectional long short-term memory network and an attention mechanism is characterized in that: The construction of the bidirectional long short-term memory network and attention mechanism of the psychological stress measurement model; the model introduces the attention mechanism in the bidirectional long short-term memory network to form the final psychological stress measurement classification model; the preprocessed EEG signal data is input into the bidirectional long short-term memory network and attention mechanism model, and the output out and hidden state h are obtained through the bidirectional LSTM layer. The output out is calculated through the attention layer to obtain the attention weights, which are multiplied with the output out and summed to obtain the attention output. Finally, the attention output is processed through the fully connected layer and the activation function relu to obtain the final measured three-category output of psychological stress.

5. According to claim 4, a method for measuring psychological stress by analyzing EEG signals based on a bidirectional long short-term memory network and an attention mechanism is characterized in that: The deep learning model uses a bidirectional LSTM module; Specifically, the bidirectional LSTM has two main components: a forward LSTM and a backward LSTM; The forward LSTM receives the original input sequence, starts from the first element of the input sequence, processes each element in turn, and outputs the forward representation of each element; The backward LSTM receives a reversed copy of the input sequence, starts from the last element of the input sequence, processes each element in turn, and outputs the backward representation of each element; The outputs of the two LSTMs are concatenated to form a context vector that contains all the information of the input sequence. This context vector can be provided to a fully connected layer or other types of output layers. The output and hidden state are obtained after processing by a bidirectional LSTM layer.

6. The method for measuring psychological stress by analyzing EEG signals based on a bidirectional long short-term memory network and an attention mechanism according to claim 4, characterized in that: The core idea of ​​the attention mechanism module is to process different parts of the input data and assign different weights to each part so that they can be weighted and summarized in the model calculation; this enables the model to better handle complex sequence data, images, and other types of information, improving the performance and generalization ability of the model; The attention mechanism is an important component of the psychological stress measurement model. It is used to process EEG signal sequences, extract features that more effectively reflect stress, and conduct experiments.

7. The method for measuring psychological stress by analyzing EEG signals based on bidirectional long short-term memory network and attention mechanism according to claim 1, characterized in that: The experiment uses precision, recall and F1-Score as the main evaluation indicators, obtains the test results of different methods on the same dataset, and compares the effectiveness and superiority of the models.

8. The method for measuring psychological stress by analyzing EEG signals based on bidirectional long short-term memory network and attention mechanism according to claim 1, characterized in that: The robustness and generalization ability of the model were tested on another public dataset and a self-built dataset. First, experiments were conducted on the public dataset Feeling Emotions included in the Kaggle database to measure and classify psychological stress. Secondly, the model was pre-trained on the Feeling Emotions dataset and fine-tuned on the training set of the dataset to better adapt to the data characteristics of specific fields. Finally, the test set of the self-built dataset was used to verify and evaluate the model.

Images