An electroencephalogram depression detection method based on multi-feature sequence spatio-temporal feature fusion

Abstract

Beneficial results of the application: the current deep learning model does not fully utilize the complementary relationship between spatial, frequency domain and time characteristics, which affects the classification accuracy of the model. In order to solve this problem, this paper proposes a spatio-temporal feature fusion network (SFFNet) based on feature image sequence and time sequence for EEG depression recognition. In terms of EEG data, the difference between subjects is fully considered, and the robust features with discriminative ability are identified from the EEG. After windowing the data, the signal channel is used to construct a feature matrix graph in the topological structure of the brain space and its voltage density, and the multi-band feature image is stacked in the channel to form a multi-channel feature image sequence. The windowed EEG time sequence corresponding to it is used as a time feature extraction. The SCCN and LGTN network modules in SFFNet are used to extract related features, and the PAM and SWAM attention modules are used for feature fusion. The multi-feature complementary fusion of EEG is realized, and the learning ability of the network and the accuracy of network recognition are improved.

Description

Technical Field

[0001] This invention relates to the field of medical signal processing, and more particularly to a method for detecting depression using electroencephalogram (EEG) signals based on the fusion of spatiotemporal features of multiple feature sequences. Background Technology

[0002] Depression is a serious mental health disorder, typically characterized by persistent, severe depressive mood and loss of interest in daily life. Diagnosis of depression often relies on patient subjective reports and physician clinical judgment, which can lead to misdiagnosis or missed diagnosis. In contrast, electroencephalography (EEG) is a non-invasive neurophysiological technique used to measure electrical activity in the brain. It is used to detect neurological disorders. EEG signals provide necessary features for depression identification, indicating that EEG-based depression identification is feasible. To date, most EEG-based depression identification technologies have focused on addressing two main problems: 1) how to extract discriminative depression-related features from EEG signals; and 2) how to construct more efficient depression identification models.

[0003] Early researchers built various machine learning models for depression identification, using algorithms such as random forests, logistic regression, and support vector machines to construct binary classification models. However, traditional machine learning methods typically require manual feature design and extraction, which can demand domain expertise and experience. Feature engineering can be time-consuming and may not capture potentially complex features. This can lead to inconsistent model performance in real-world clinical applications.

[0004] Deep learning has emerged in biomedical signal recognition due to its more powerful feature learning, ability to handle large-scale data, and flexibility. However, despite being a powerful machine learning technique, its internal workings remain difficult to explain. Current research on electroencephalogram (EEG) signals is limited to extracting single features from the time, frequency, or spatial domains, failing to comprehensively reflect psychological states such as depression and thus restricting in-depth understanding.

[0005] To address the aforementioned problems, this invention learns more effective feature representations based on electroencephalography (EEG) for brain decoding. Inspired by the advantages of combining multiple features in 3D convolutional attention neural networks and 4D convolutional recurrent neural networks, we propose a novel method for depression identification. Summary of the Invention

[0006] This invention proposes a method for detecting depression using EEG signals through multi-feature sequence spatiotemporal feature fusion learning. This method inputs multi-band feature map representations and EEG time series data into an SFFNet network model that considers multiple feature information for classification. This approach comprehensively considers multiple aspects of EEG signal features, integrating temporal, spatial, frequency, and left / right hemisphere differential feature information through a feature map constructed with voltage density as the core. This enhances information related to depression in the spatial, temporal, and frequency dimensions.

[0007] A method for detecting depression using electroencephalogram (EEG) signals based on the fusion learning of multi-feature sequence spatiotemporal features, characterized by the following steps:

[0008] Step 1: Preprocess the data and extract features using the designed algorithm;

[0009] Step 2: Construct the spatiotemporal feature fusion network structure SFFNet for the sequence weight attention module designed in this invention;

[0010] Step 3: Use SFFNet to train the processed data and verify the performance of SFFNet.

[0011] Step 1 includes the following steps:

[0012] Step 1.1: First, use a 50 Hz notch filter to perform initial filtering on the signal;

[0013] Step 1.2: A fourth-order Butterworth bandpass filter is used to filter the signal in the second stage;

[0014] Step 1.3: Fast Independent Component Analysis (fast ICA) removes artifacts from electrooculography (EOG) and electromyography (EMG) signals;

[0015] Step 1.4: Divide the signal into 60 non-overlapping time series segments of 1 second each;

[0016] Step 1.5: Use the designed algorithm to convert the signal into a multi-band feature image;

[0017] Step 2 includes the following steps:

[0018] Step 2.1: Use the LGTN module to learn the time-series features of EEG;

[0019] Step 2.2: Use the SCCN module to receive the input multi-band feature image;

[0020] Step 2.3: Use the PAM module and the improved SWAM module to process the feature inputs from the SCCN module and the LGTN module respectively;

[0021] Step 3 includes the following steps:

[0022] Step 3.1: Train the processed data using SFFNet;

[0023] Step 3.2: Compare performance with recent state-of-the-art methods.

[0024] The beneficial results of this invention are as follows: This invention comprehensively considers that current deep learning models do not fully utilize the complementary relationship between spatial, frequency, and temporal features, which affects the classification accuracy of the models. To solve this problem, this paper proposes a spatiotemporal feature fusion network (SFFNet) based on feature image sequences and time series for EEG depression identification. In terms of EEG data, it fully considers the differences in EEG among subjects, identifies robust features with discriminative ability from EEG, and after windowing the data, constructs a feature matrix map using the topological structure of signal channels in brain space and their voltage density. Multi-frequency feature maps are then superimposed to form a multi-channel feature map sequence, and the corresponding windowed EEG time series is used for temporal feature extraction. The SCCN and LGTN network modules in SFFNet are used to extract relevant features, and the PAM and SWAM attention modules are used to fuse the extracted features, achieving multi-feature complementary fusion of EEG, improving the network's learning ability on multiple features and the accuracy of network recognition. Attached Figure Description

[0025] Figure 1 This is the overall flowchart of the method.

[0026] Figure 2 This is the overall framework diagram of the method.

[0027] Figure 3 This is a diagram of the SFFNet model structure.

[0028] Figure 4 This is a structural diagram of the PAM module and the improved SWAM module. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0030] Figure 1 This is an overall flowchart of the method of the present invention. Figure 2 This is the overall framework diagram of the method. Figure 3 This is the SFFNet model structure diagram. Figure 4This is a structural diagram of the PAM module and the improved SWAM module, as shown below. Figure 1 Figure 2 Figure 3 Figure 4 As shown, the method includes:

[0031] Step 1.1: Select a 50 Hz notch filter to perform the initial filtering of the signal;

[0032] Step 1.2: Select a fourth-order Butterworth bandpass filter to filter the signal in the second stage;

[0033] Step 1.3: Fast Independent Component Analysis (fast ICA) is used to remove artifacts from electrooculography (EOG) and electromyography (EMG) signals;

[0034] Step 1.4: Select sample points from the first minute of 19 channels and window the EEG signals into 60 1-second time series without overlap;

[0035] Step 1.5: Using the Discrete Fourier Transform (DFT) in the frequency domain, the time-domain EEG signal is converted into relevant information in the frequency domain. The frequency domain is then divided into frequency bands, and the voltage density of each channel in each band is derived. During the voltage density calculation, we provide the signal of a given channel. The discrete Fourier transform result is:

[0036]

[0037] Where N is the length of the signal. Frequency domain signal. The amplitude spectrum (absolute value spectrum) It can be represented as:

[0038]

[0039] In the above formula, [X(0), X(1), X(2), ..., X(N / 2)] are the complex values ​​corresponding to the frequency components, while [|X(0)|, |X(1)|, |X(2)|, ..., |X(N / 2)|] are the corresponding amplitude values, i.e., the amplitude spectrum. Finally, the voltage density of the five frequency bands of δ wave, θ wave, α wave, β wave and γ wave is calculated as follows:

[0040]

[0041] In the above formula, j is the index of the corresponding frequency band. For voltage density, ' 'and' ' represents the beginning and end of the corresponding frequency band;

[0042] What has been obtained The data is arranged (where V is the voltage density, i is the corresponding channel number, and j is the corresponding frequency band index). After feature extraction, it is mapped to the feature image of the multi-band channels of the nth EEG test. The transformation function for converting the feature image into a multi-band channel is:

[0043]

[0044] Step 2 includes the following steps:

[0045] Step 2.1: Input the EEG time series into the multi-scale temporal convolutional layer in the LGTN. The length of the temporal convolutional kernel in the layer is set to the EEG sampling rate ( Different proportions of ). This proportion coefficient is denoted as . , where k represents the level of the temporal convolutional layer, and the value of k varies from 1 to k (a = 0.5, k = 3). Therefore, the size of the T kernels in the k-th layer can be expressed as The specific formula is as follows:

[0046]

[0047] The EEG signal is input into a long short-term memory recurrent neural network after undergoing multi-scale convolution.

[0048] Step 2.2: Use the Spatial Component Intelligent Convolutional Network (SCCN) to learn the temporal and spatial correlations in EEG signals, with the structure as follows: Figure 3 The diagram shows data filtering in the spatial domain.

[0049] Step 2.3: After the SCCN module, use the Spatial Attention Module (PAM) to process local features and establish a contextual relationship model. Specifically, let the local features of the given multi-band feature image be... The input is then fed into a convolutional layer to generate two new feature maps. Then reshape them into Next, matrix multiplication is performed between the transposes of A and B, and a softmax layer is applied to compute the spatial attention map, as shown in the following formula:

[0050]

[0051] in This calculates the influence of the i-th position on the j-th position. The feature P is also input into the convolutional layer to generate a new feature map. Multiply it by the scaling factor α, and perform element-wise summation on feature A to obtain the final output. The specific formula is:

[0052]

[0053] The LGTN module is followed by the Sequence Weighted Attention (SWAM) module, an improvement on the dual attention module. Let L be the set of sequence inputs, which can be defined as:

[0054]

[0055] Where D represents the EEG sequence channel. Permutation layer implementation set Dimensional transformation to obtain a set . Set The sequence weight attention matrix is ​​obtained through a softmax layer. A matrix is ​​achieved through a permutation layer. Dimensional transformation yields a matrix. Then, multiply the result by the scaling parameter β and compare it with... Perform element-wise summation to obtain the final output:

[0056]

[0057] Step 3 includes the following steps:

[0058] Step 3.1: Train the processed data using SFFNet and evaluate the model's performance using the average result of 10 cross-validations;

[0059] Step 3.2: This experiment was compared with previous similar work that used hybrid neural networks and EEG signals to automatically detect several neurological diseases. The results are shown in Table 1.

[0060] Table 1 compares the results of this experiment with other similar experiments.

[0061] Acharya et al.

[49] , 2018 5 Normal, 5 Epileptic SeizureDetection CNN (13 Layers) 88.7 Ay et al.

[51] , 2019 15 Normal, 15 Depressed DepressionDetection CNN-LSTM 97.66 Li et al.

[48] , 2019 27 Normal, 24 Depressed DepressionDetection ConvNet 85.62 Geetanjali Sharma et al.

[47] , 2021 24 Normal, 21 Depressed DepressionDetection CNN (6 Layers) 99.1 JiaoZhang et al.

[50] , 2023 24Normal, 24Depressed (MODMA) DepressionDetection 2DCNN-LSTM 95.1 ZhuozhengWang et al.

[52] , 2022 24 Normal, 24 Depressed (MODMA) DepressionDetection 1D-CNN-GRU-ATTN 99.33 This invention 24 Normal, 24 Depressed (MODMA) DepressionDetection SFFNet 99.64

[0062] It should be understood that any parts not described in detail in this specification belong to the prior art.

[0063] The above description, in conjunction with the accompanying drawings, is merely a specific embodiment and process of the present invention. However, the scope of protection of the present invention is not limited thereto. Any person skilled in the art should understand that this is only an illustrative example, and various changes and substitutions can be made to this embodiment without departing from the essence of the present invention. The scope of the present invention is defined only by the appended claims.

[0064] The embodiments described with reference to the accompanying drawings are exemplary and are only used to explain the invention. They should not be construed as limiting the invention. The specific scope of the embodiments of the invention is not limited thereto. On the contrary, all embodiments of the invention include all variations and modifications that fall within the spirit and scope of the appended claims.

Claims

1. A method for detecting depression via electroencephalogram (EEG) based on the fusion of spatiotemporal features of multiple feature sequences, characterized in that, Includes the following steps: Step 1: Preprocess the data and extract features using the designed algorithm; Step 2: Construct the spatiotemporal feature fusion network structure SFFNet for the sequence weight attention module designed in this invention; Step 3: Use SFFNet to train the processed data and verify the performance of SFFNet; Step 1 specifically includes the following steps: Step 1.1: Select a 50 Hz notch filter to perform the initial filtering of the signal; Step 1.2: Select a fourth-order Butterworth bandpass filter to filter the signal in the second stage; Step 1.3: Rapid independent component analysis was used to remove artifacts from electrooculography (EOG) and electromyography (EMG) signals; Step 1.4: Window the selected 19-channel EEG signals into 60 1-second time series segments without overlap; Step 1.5: Using Discrete Fourier Transform in the frequency domain, the EEG signal is converted into relevant information in the frequency domain. The frequency domain is divided into frequency bands, and the voltage density of each channel in each band is derived. In the voltage density calculation process, we provide the signal of a given channel. The discrete Fourier transform result is: Where N is the length of the signal, a frequency domain signal. The amplitude spectrum (absolute value spectrum) It can be represented as: ， In the above formula, [X(0), X(1), X(2), ..., X(N / 2)] are the complex values ​​corresponding to the frequency components, while [|X(0)|, |X(1)|, |X(2)|, ..., |X(N / 2)|] are the corresponding amplitude values, i.e., the amplitude spectrum; finally, the voltage density of the five frequency bands of δ wave, θ wave, α wave, β wave and γ wave is calculated as follows: ， In the above formula, j is the index of the corresponding frequency band. For voltage density, ' 'and' ' represents the beginning and end of the corresponding frequency band; What has been obtained The data is arranged as follows: V represents voltage density, i represents the corresponding channel number, and j represents the corresponding frequency band index; after feature extraction, it is mapped to the feature image of the multi-band channels of the nth EEG test. The transformation function for converting the feature image into a multi-band channel is: ， Step 2 specifically includes the following steps: Step 2.1: Input the EEG time series into the multi-scale temporal convolutional layer in the LGTN. The length of the temporal convolutional kernel in the layer is set to the EEG sampling rate. Different proportions; this proportion coefficient is denoted as Where k represents the level of the temporal convolutional layer, and the value of k varies from 1 to k (a = 0.5, k = 3); therefore, the size of the T kernels in the k-th layer can be expressed as The specific formula is as follows: ， The EEG signal is input into a long short-term memory recurrent neural network after undergoing multi-scale convolution. Step 2.2: Use the Spatial Component Intelligent Convolutional Network (SCCN) to learn the temporal and spatial correlations in EEG signals and filter the data in the spatial domain; Step 2.3: After the SCCN module, the Spatial Attention Module (PAM) is used to process local features and establish a contextual relationship model; after the LGTN module, the Sequence Weighted Attention Module (SWAM), an improvement on the dual attention module, is used; let L be the sequence input set, which can be defined as: ， Where D represents the EEG sequence channel; the permutation layer implementation set Dimensional transformation to obtain a set ; set The sequence weight attention matrix is ​​obtained through a softmax layer. A matrix is ​​achieved through a permutation layer. Dimensional transformation yields a matrix. Then, multiply the result by the scaling parameter β and combine it with... Perform element-wise summation to obtain the final output: ， Step 3 includes the following steps: Step 3.1: Train the model on the processed data using SFFNet and evaluate its performance using the average result of 10 cross-validations; Step 3.2: Compare this experiment with previous similar work that uses hybrid neural networks and EEG signals to automatically detect several neurological diseases.

Images