Electroencephalogram fatigue detection method based on fusion of graph convolutional network and Transform

Abstract

The invention discloses an electroencephalogram fatigue detection method based on fusion of a graph convolutional network and Transform, and belongs to the technical field of artificial intelligence and electroencephalogram signal analysis. The method comprises the steps that multichannel electroencephalogram signals are collected, and band-pass filtering, power frequency notch, independent component analysis, standardization and other preprocessing are conducted on the signals; constructing an inter-channel graph structure based on a Pearson's correlation coefficient, and extracting spatial features by using a graph convolutional network; inputting the spatial features of the plurality of continuous time windows into Transform to carry out time sequence modeling; and finally, realizing fatigue state recognition through a full-connection network and a Softmax classifier. According to the method, the spatial topological structure and the time dynamic evolution of the EEG signal can be modeled at the same time, the accuracy and the real-time performance of fatigue detection are remarkably improved, and the method has good generalization ability and edge deployment ability and is suitable for traffic driving monitoring, intelligent health and other scenes.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of biomedical signal processing and fatigue detection, and specifically relates to an electroencephalogram (EEG) fatigue detection method fusing a graph convolutional network (GCN) and a Transformer model. The method is suitable for scenarios requiring real-time evaluation of brain fatigue state, such as traffic safety monitoring, medical health management, and human-computer interaction, through multi-channel EEG data analysis. Unlike traditional fatigue detection methods that use single time / frequency domain feature extraction and shallow classifiers, the method uses the advantages of graph neural networks and Transformers to simultaneously capture the spatial dependence and temporal dynamic characteristics of EEG signals, improving the accuracy and generalization ability of fatigue state recognition. BACKGROUND

[0002] Long hours of work by drivers or operators can lead to brain fatigue, which can affect attention and decision-making ability, thereby increasing the risk of accidents. Existing EEG fatigue detection methods are mainly based on time domain, frequency domain or traditional machine learning techniques, such as extracting single-channel EEG frequency band energy, relative energy ratio, and other indicators through statistical features or Fourier transform. However, since EEG signals are time-varying, non-stationary, and multi-channel complex signals, simple time-frequency features often cannot fully reflect the coordinated changes between different brain regions. Traditional methods usually ignore the modeling of functional connections between brain regions (spatial dependence) and are also difficult to fully utilize long-term evolution information. For example, most existing models only analyze a single brain region or a single time segment, and cannot capture the dynamic evolution characteristics of the entire brain network, resulting in insufficient recognition accuracy and real-time performance.

[0003] In recent years, deep learning-based EEG analysis methods have made significant progress. Graph convolutional networks (GCN) can effectively model the spatial topology between multi-channel EEG electrodes and realize the representation learning of functional connections between brain regions; Transformers have strong sequence modeling capabilities and can mine long-term sequential dependencies. Existing research has shown that networks combining graph models and attention mechanisms have achieved an accuracy of over 90% in driving fatigue detection. Therefore, designing a fusion model that simultaneously uses GCN to capture spatial features and Transformer to capture temporal dynamics can be expected to compensate for the shortcomings of existing methods and improve EEG fatigue recognition performance. SUMMARY

[0004] (I) Technical problems to be solved

[0005] In order to solve the problem that it is difficult to simultaneously model the spatial structure and time dynamics of electroencephalogram signals in the prior art, the present application proposes an electroencephalogram fatigue detection method based on the fusion of graph convolution network and Transformer. The method performs filtering, independent component analysis (ICA) for removing artifacts, standardization and other preprocessing steps on multi-channel EEG data, constructs a graph structure describing the correlation between channels, and sequentially uses GCN and Transformer to extract spatial and temporal features, and finally outputs the fatigue state through feature fusion and a classification network.

[0006] (II) Technical solutions

[0007] To achieve the above object, the present application adopts the following technical solutions:

[0008] A kind of electroencephalogram fatigue detection method based on the fusion of graph convolution network and Transformer, comprising the following steps:

[0009] 1. Signal preprocessing:

[0010] The band-pass filter (e.g. 0.5-45 Hz) is first performed on the collected EEG signal of each channel to filter out the DC drift and high-frequency noise, and the 50 Hz power grid interference is removed using a notch filter. The transfer function is:

[0011]

[0012] Where ω0=2Π*50, Q=30 is the quality factor. Then the independent component analysis (ICA) algorithm is used to remove artifacts such as electrooculogram and electromyogram. Then the preprocessed multi-channel signals are divided into sliding windows with a fixed time length (e.g. 2 seconds, 512 sampling points), with 50% overlap (step length 1 second) between windows, and the data in each window is standardized using Z-score to eliminate individual differences and improve model robustness.

[0013] 2. Graph structure construction (time connection based on Pearson correlation coefficient):

[0014] For the multi-channel signals in each sliding window, the Pearson correlation coefficient between each pair of channels is calculated to obtain the channel correlation matrix A. The matrix is added with a unit self-loop and normalized

[0015]

[0016] D is the degree matrix, which is used as the adjacency matrix of the graph convolution network. This graph structure reflects the similarity of signals in different brain regions (functional connectivity) and provides a topological basis for spatial feature extraction.

[0017] 3. GCN spatial feature extraction:

[0018] The normalized adjacency matrix above and the multi-channel signal data of the current window Input the two-layer graph convolution network.

[0019] The first layer of graph convolution extracts the preliminary spatial features by

[0020]

[0021] Extract the deep spatial features by the second layer

[0022]

[0023] Extract the deep spatial features. Among them, W (0) , W (1) are learnable parameter matrices, and ReLU is a nonlinear activation function. The feature vector H (2) output by the GCN represents the comprehensive features of each window in the brain region spatial structure.

[0024] 4. Transformer time modeling:

[0025] Concatenate the feature sequences output by multiple consecutive sliding windows in the GCN layer to form a sequence input where L is the time step length, and d is the GCN output dimension. Add the sine position encoding to the sequence to retain the time information, and then input it into the two-layer Transformer encoder module for time-dependent modeling. The Transformer module is set to have 2 layers of multi-head attention mechanism and 4 heads, and the hidden layer dimension is 64. The core calculation is:

[0026]

[0027] Among them, the query matrix Q, the key matrix K, and the value matrix V are obtained by linear mapping from the input sequence, and d k is the key vector dimension. Through multiple layers of self-attention and feedforward networks, the Transformer module can capture long-term dependencies and dynamic changes in the sequence. The output obtains the time series feature representation of each window after fusing the context information.

[0028] 5. Feature fusion and classification:

[0029] The spatial features extracted by the GCN and the time features output by the Transformer are spliced and fused to form a comprehensive feature vector. The feature is input into a fully connected neural network (MLP) for classification. The MLP includes several fully connected layers, and uses a ReLU activation function and dropout (0.3) after each layer to prevent overfitting. Finally, the probability values of the fatigue state belonging to each class (awake, fatigue) are calculated by the Softmax output layer. According to the maximum value of the output probability, the current brain fatigue level of the subject is determined.

[0030] (III) Beneficial effects:

[0031] The functional connection graph within the time window is constructed using the Pearson correlation coefficient, and the spatial collaborative relationship between channels is modeled using the GCN.

[0032] The Transformer can effectively capture the time trend of fatigue evolution, and enhance the continuity and stability of recognition.

[0033] The fusion model can realize efficient and real-time brain fatigue state detection, and has application potential in actual deployment. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 A general framework flowchart of an EEG fatigue detection method based on the fusion of graph convolution network and Transformer provided by the present application; DETAILED DESCRIPTION

[0035] As shown in the accompanying Figure 1 The overall process of the EEG fatigue detection method based on the fusion of graph convolution network and Transformer is as follows:

[0036] 1. Signal preprocessing

[0037] For eight-channel EEG signals, first, band-pass filter (0.5-45Hz) each channel signal to filter out low-frequency drift and high-frequency noise; At the same time, use a notch filter to remove 50Hz power frequency interference, and its transfer function is:

[0038]

[0039] Where ω0=2Π*50, Q=30 is the quality factor. Then apply independent component analysis (ICA) algorithm to remove electrooculogram / electromyogram artifacts, and set the EEG matrix as (N is the number of sampling points, T is the time step), the separation matrix W is solved by FastICA algorithm, and the signal A = WX is reconstructed. Then, the purified signal is segmented by time: 2 seconds (512 sample points) as the window length, 50% overlap way to divide the time series data, that is, every 1 second to generate a sample window containing 2 seconds of data. For each sample window, the Z-score standardization (mean removal, divided by standard deviation) is performed on each channel signal respectively to ensure that the features of different subjects and different channels have comparable numerical ranges and improve the generalization performance of the subsequent model.

[0040] 2. Graph structure construction:

[0041] Graph convolutional network (GCN) is a neural network that extends the convolution operation to non-Euclidean graph structure. In traditional CNN, the convolution kernel slides on regular grid data (such as image pixel matrix), relying on local spatial structure and shift-invariant features; while GCN targets graph structure data, aggregating features of nodes and their neighbors through adjacency matrix. Specifically, let the node feature matrix of the graph be H (l) , and the adjacency matrix plus the self-loop is normalized as

[0042]

[0043] Then the propagation of a layer of GCN can be represented as:

[0044]

[0045] where W (l) is the learnable weight matrix of the l-th layer, and σ is the nonlinear activation function. The formula shows that the new feature of each node is the weighted average (multiplied by the weight matrix and activated) of its own and its neighbor node features. Unlike two-dimensional CNN which relies on fixed convolution window to extract local grid features, GCN aggregates features of irregular neighborhood through the topology of the graph (adjacency matrix), thus adapting to arbitrary connection relationships among multiple channels.

[0046] For each standardized multi-channel window data, first calculate the Pearson correlation coefficient between each pair of channels:

[0047]

[0048] where x i,t represents the signal value of the i-th channel at time t, is the mean value of the i-th channel. The correlation matrix (N is the number of channels) describes the functional connectivity strength of each brain region. In order to stabilize the training of the graph convolutional network, A is added with a unit self-loop and parallel row symmetric normalization: Where D is the degree matrix, the final result is... As an adjacency matrix of a graph neural network, the temporal correlation between channels within the entire window is used to construct the spatial structure of brain regions.

[0049] 3. GCN spatial feature extraction:

[0050] A two-layer graph convolutional network is used to extract spatial features from the EEG structure for each time window. The input is a channel signal matrix. (The time-series feature vector for each channel can be simply taken as the instantaneous potential or pre-extracted spectral features of that channel; d) in (for input dimension), and adjacency matrix The first layer of graph convolution is passed through... Extracting preliminary spatial features, the second layer through Extract deep spatial features. Among them, W (0) W (1) Let be a learnable parameter matrix, and ReLU be a non-linear activation function.

[0051] GCN output Each row in the GCN is a feature vector of a brain region node, which can represent the spatial information of the entire window. Through convolution operations, the GCN integrates the channel dependencies reflected in the adjacency matrix, allowing the features of interconnected brain regions to influence and strengthen each other, thereby extracting more discriminative spatial feature representations.

[0052] 4. Transformer temporal modeling:

[0053] The output features of the GCN from several consecutive windows are serialized into a time series input to the Transformer. After extracting K = 15 consecutive windows, the sequence of the GCN output feature matrix can be represented as:

[0054]

[0055] Concatenate them along the time dimension to obtain the feature sequence To incorporate temporal information, a fixed sine and cosine positional encoding is added to each time step of sequence S. The sequence with positional encoding is input to a two-layer standard Transformer encoder. Each encoder layer contains a multi-head self-attention mechanism and a feedforward network. In this invention, the number of multi-heads is set to 4, and the feedforward layer dimension is set to 64, enabling the network to capture global temporal dependencies while maintaining computational efficiency. Through self-attention computation, the Transformer can dynamically adjust the weight distribution between features at different time steps, thereby uncovering long-term dependencies in the time series and suppressing irrelevant information. The Transformer output is a sequence representation that incorporates temporal context information, with its last time step or the sequence pooling result used as the overall temporal feature to capture fatigue development trends.

[0056] 5. Feature fusion and classification:

[0057] The spatial feature vector of the last window extracted by GCN is concatenated with the temporal feature vector output by Transformer to form the final feature representation. The integrated feature is input into a multi-layer perceptron (MLP) classification network composed of several fully connected layers. Each layer of MLP uses ReLU activation and adds Dropout (such as 0.3) to prevent overfitting. The final output layer uses the Softmax function to map the feature to three classification probability values (awake, mild fatigue, and severe fatigue). The category corresponding to the maximum probability is taken as the final detection result. During the model training process, the cross-entropy loss function can be used and the parameters of GCN, Transformer, and MLP are iteratively updated using gradient descent and other optimization algorithms.

Claims

1. A brainwave fatigue detection method based on the fusion of graph convolutional networks and Transformer, characterized in that, Includes the following steps: 1) Perform the following preprocessing operations sequentially on the raw signal of each channel: Bandpass filtering was performed from 0.5Hz to 45Hz, notch filter was used to suppress 50Hz power frequency interference, independent component analysis (ICA) was applied to remove eye movement and electromyography artifacts, the signal was divided into 2-second windows and 50% overlap sliding windows, and Z-score normalization was performed on each window signal. 2) Construct a graph structure between channels for the signal within each sliding window segment: The Pearson correlation coefficient between each channel is calculated to form an adjacency matrix. Unit self-loops are added to the adjacency matrix and normalized to generate the input graph structure of the graph convolutional network. 3) Input the signal of each window into the graph convolutional network and use two layers of GCN to extract spatial features; 4) Serialize the GCN outputs of multiple consecutive time windows into a time input sequence, add position encoding, and input it into a Transformer model containing two encoder layers to extract time-dependent features; 5) The GCN output and the Transformer output are concatenated and fused, and then input into a multilayer perceptron (MLP) classifier to output fatigue state classification labels, including two categories: awake and fatigued.

2. The method according to claim 1, characterized in that: The graph convolutional network consists of two graph convolutional layers, each with an output dimension of 64 and an activation function of ReLU.

3. The method according to claim 1, characterized in that: The Transformer model contains two encoder layers, each encoder uses four multi-head self-attention mechanisms, and the hidden layer dimension is 64.

4. The method according to claim 1, characterized in that: The classifier is an MLP network with a Dropout layer and a Dropout probability of 0.

3. The output uses the Softmax function to generate two types of fatigue state probabilities.

Images