Fatigue detection method and system based on forehead sparse electroencephalogram signals

Abstract

The invention discloses a fatigue detection method and system based on forehead sparse electroencephalogram signals, and relates to the technical field of biomedical signal processing. The method comprises the steps that a forehead electroencephalogram signal data set is obtained and preprocessed; dividing the forehead electroencephalogram signal, and performing frequency domain analysis on each frequency band; constructing a fatigue detection model based on a progressive fusion strategy, performing effective feature extraction on the time sequences of the power spectral density and the differential entropy features to obtain power spectral density time features and differential entropy time features, performing deep feature extraction on the power spectral density time features and the differential entropy time features, and performing fusion to obtain fusion features; and carrying out classification identification on the fusion features to obtain a fatigue detection result. According to the invention, an asymptotic hierarchical fusion strategy is adopted, different model modules are effectively combined, and feature information of each level is gradually optimized and fused, so that efficient and accurate driver fatigue state detection is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of biomedical signal processing, and in particular to a fatigue detection method and system based on forehead sparse electroencephalogram signals. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] When driving for a long time or without sufficient rest, drivers are prone to problems such as inattention and slow reaction, which greatly increases the risk of traffic accidents. Therefore, finding an efficient and reliable driver fatigue detection method can effectively reduce the incidence of road traffic accidents, which has great social significance and practical application value.

[0004] At present, fatigue driving detection methods are mainly divided into two categories: subjective methods and objective methods. Since the driver's fatigue state is unstable, has large individual differences and is easily affected by the external environment, traditional subjective detection methods have obvious limitations in judging the driver's fatigue state. In contrast, objective fatigue detection methods have gradually become the focus of industry research because they do not require human participation and have high detection accuracy. Common objective fatigue detection methods mainly rely on facial feature recognition and human physiological signal evaluation technology. Physiological signals mainly include electromyography signals, electrocardiogram signals, electrooculogram signals and electroencephalogram signals. In particular, electroencephalogram signals are widely regarded as the "gold standard" for human fatigue detection because of their high accuracy, real-time and precision in fatigue detection, and have been studied by many scholars.

[0005] Although fatigue detection technology based on EEG signals has made some progress in fatigue detection, most of them are based on multi-channel EEG signals and still face some significant limitations. First, the acquisition of multi-channel EEG signals usually requires complex equipment and fine electrode arrangement, which is easily affected by loose electrodes, signal noise and environmental interference, resulting in insufficient stability and reliability of the data. Secondly, when processing multi-channel signals, existing detection methods usually rely on a single model or a simple feature fusion strategy, which makes it difficult to fully explore and utilize the correlation information between different signal channels and the related information between features, thereby affecting the accuracy and robustness of detection. In addition, these methods often show insufficient generalization ability when facing complex and changeable driving environments, and it is difficult to effectively deal with individual differences and dynamic fatigue states. In contrast, using fewer EEG channels, especially frontal EEG, can greatly reduce the complexity of wearable devices and have stronger application value.

[0006] However, since frontal EEG signals have fewer channels, it is difficult to extract rich and representative features from their spatial distribution. Therefore, in order to obtain more features that can be used for fatigue driving detection, how to deeply mine the effective information in frontal EEG signals and achieve high fatigue driving detection accuracy and convenience with fewer channel EEG signals has become a technical problem that needs to be urgently solved in existing technologies. Summary of the invention

[0007] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a fatigue detection method and system based on forehead sparse EEG signals. For forehead sparse EEG channel signals, the power spectral density and differential entropy features of the forehead EEG signals are extracted and feature smoothing is performed. An asymptotic hierarchical fusion strategy is adopted to effectively combine different model modules, and the feature information of each level is gradually optimized and fused, thereby realizing efficient and accurate driver fatigue status detection.

[0008] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0009] The first aspect of the present invention provides a fatigue detection method based on forehead sparse EEG signals, comprising the following steps:

[0010] Get the frontal EEG signal dataset;

[0011] Preprocess the frontal EEG signal data;

[0012] The frontal EEG signal is divided by using the frequency band division method, and each frequency band is analyzed in the frequency domain to obtain the frequency domain features of the frontal EEG signal, wherein the frequency domain features include power spectrum density and differential entropy features;

[0013] A fatigue detection model is constructed based on the progressive fusion strategy. Effective feature extraction is performed on the time series of power spectral density and differential entropy features respectively to obtain power spectral density time features and differential entropy time features. After deep feature extraction, the power spectral density time features and differential entropy time features are fused to obtain fused features. The fused features are classified and identified to obtain fatigue detection results.

[0014] Furthermore, the steps of preprocessing the frontal EEG signal data include filtering, denoising and artifact removal.

[0015] Furthermore, the specific steps of dividing the frontal EEG signal using the frequency band division method and performing frequency domain analysis on each frequency band are as follows:

[0016] Using the 5-band division method, the EEG signal of each channel is divided into 5 frequency bands;

[0017] The power spectral density and differential entropy characteristics of each frequency band are calculated by short-time Fourier transform.

[0018] Furthermore, after obtaining the frequency domain features of the frontal EEG signal, the extracted power spectral density and differential entropy features are smoothed using a linear dynamic system.

[0019] Furthermore, the specific steps for extracting effective features from the time series of power spectrum density and differential entropy features are as follows:

[0020] The dual-branch architecture and multi-level convolution module are used to deeply mine the time series of power spectral density and differential entropy features, respectively, learn the correlation between frequency domain and time domain, and obtain the power spectral density time features and differential entropy time features.

[0021] Furthermore, the specific steps of fusing the power spectrum density time feature and the differential entropy time feature after deep feature extraction are as follows:

[0022] The power spectrum density time characteristic and the differential entropy time characteristic are added to obtain the intermediate state;

[0023] A convolution module with residual connection is used to extract deeper features from the power spectrum density time features and differential entropy time features after multi-layer convolution, and a cross-attention fusion module is used to update the intermediate state to obtain the updated intermediate state.

[0024] Perform convolution operations with residual connections on the deeper power spectrum density time features and differential entropy time features to extract higher-level semantic features, thereby obtaining power spectrum density semantic features and differential entropy semantic features.

[0025] The power spectral density semantic features, differential entropy semantic features and the updated intermediate states are input into the cross attention fusion module for fusion to obtain the fused features.

[0026] Furthermore, the specific steps of classifying and identifying the fused features and obtaining fatigue detection results are as follows:

[0027] The fused features pass through the self-attention module and enter the classification module composed of two fully connected layers and normalization processing to achieve three-category recognition of fatigue status.

[0028] The second aspect of the present invention provides a fatigue detection system based on forehead sparse EEG signals, comprising:

[0029] A data acquisition module, configured to acquire a frontal EEG signal dataset;

[0030] A signal and processing module, configured to pre-process the frontal EEG signal data;

[0031] The feature extraction module is configured to divide the frontal EEG signal by using a frequency band division method, and perform frequency domain analysis on each frequency band to obtain frequency domain features of the frontal EEG signal, wherein the frequency domain features include power spectrum density and differential entropy features;

[0032] The multimodal asymptotic hierarchical network module is configured to build a fatigue detection model based on a progressive fusion strategy, perform effective feature extraction on the time series of power spectral density and differential entropy features respectively to obtain power spectral density time features and differential entropy time features, perform deep feature extraction on the power spectral density time features and differential entropy time features and then fuse them to obtain fused features; classify and identify the fused features to obtain fatigue detection results.

[0033] The third aspect of the present invention provides a medium having a program stored thereon, which, when executed by a processor, implements the steps of the fatigue detection method based on forehead sparse EEG signals as described in the first aspect of the present invention.

[0034] The fourth aspect of the present invention provides a device, including a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the fatigue detection method based on forehead sparse EEG signals as described in the first aspect of the present invention are implemented.

[0035] One or more of the above technical solutions have the following beneficial effects:

[0036] The present invention discloses a fatigue detection method and system based on sparse frontal EEG signals, which adopts asymptotic hierarchical fusion strategy and uses a dual-branch architecture to effectively extract PSD and DE features and learn the relevant information between the features. An attention mechanism module is introduced into the multi-level convolution module, and the cross-modal hierarchical semantic features of PSD and DE are fused by the attention mechanism, and different features are weighted, so that the model pays more attention to the features useful for the classification task, thereby improving the accuracy of the method model.

[0037] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0039] Figure 1 This is a signal processing flow chart of a fatigue driving detection method based on forehead sparse EEG signals in Embodiment 1 of the present invention;

[0040] Figure 2This is a model structure diagram based on a multimodal asymptotic hierarchical fusion network in Embodiment 1 of the present invention;

[0041] Figure 3 This is a structural diagram of a multi-level convolution module in Embodiment 1 of the present invention;

[0042] Figure 4 This is a structural diagram of the attention mechanism module in Embodiment 1 of the present invention;

[0043] Figure 5 This is a structural diagram of the classification module in Example 1 of the present invention. DETAILED DESCRIPTION

[0044] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0045] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "include" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or their combinations;

[0046] Embodiment 1:

[0047] Embodiment 1 of the present invention provides a fatigue detection method based on forehead sparse EEG signals, such as Figure 1 As shown in the figure, by obtaining the forehead EEG signal, the original EEG signal is preprocessed by filtering and noise reduction to remove environmental and equipment interference. The power spectrum density and differential entropy features are extracted based on short-time Fourier transform and smoothed. The feature fusion is performed based on the multimodal asymptotic hierarchical fusion network to complete the three-classification task. It is realized that only the sparse EEG of the forehead can be used for fatigue detection tasks and obtain higher detection performance, which improves the accuracy and convenience of fatigue driving detection.

[0048] The specific steps include:

[0049] Step 1: Obtain the frontal EEG signal dataset.

[0050] In a specific implementation, the frontal EEG signal dataset includes sample EEG signals of multiple fatigue driving experiment subjects and label information of each sample EEG signal.

[0051] Specifically, a public data set is used or a simulated fatigue driving experiment is designed to collect EEG signals in the frontal area through an EEG acquisition device, and each sample frontal EEG signal includes sample channel signals of the frontal 4 channels. The label information includes frontal EEG signals in the awake state, frontal EEG signals in the fatigue state, and frontal EEG signals in the sleepy state.

[0052] Step 2: Preprocess the frontal EEG signal data.

[0053] In a specific embodiment, the step of preprocessing the frontal EEG signal data includes filtering, denoising and artifact removal to improve the quality and stability of the signal.

[0054] First, the EEG signal is filtered through bandpass filtering to remove high-frequency and low-frequency noise and retain the effective signal in the target frequency band. At the same time, independent component analysis (ICA) or other noise suppression techniques are used to denoise the signal and reduce noise interference. In this way, purer EEG data can be obtained to ensure the accuracy of subsequent processing.

[0055] Step 3: Use the frequency band division method to divide the frontal EEG signal, and perform frequency domain analysis on each frequency band to obtain the frequency domain characteristics of the frontal EEG signal.

[0056] Among them, the frequency domain features include power spectrum density and differential entropy features.

[0057] Step 3.1: Use the 5-band division method to perform frequency domain analysis on the frontal EEG signal and divide the EEG signal of each channel into 5 frequency bands.

[0058] Using the 5-band division method, the EEG signal of each channel is divided into five frequency bands: δ (1-4 Hz), θ (4-8 Hz), α (8-14 Hz), β (14-31 Hz), and γ (31-50 Hz).

[0059] Step 3.2: Perform spectrum analysis on the signals of each frequency band through short-time Fourier transform, and calculate the power spectral density (PSD) and differential entropy characteristics (DE) of each frequency band.

[0060] PSD is used to reflect the change of signal power with frequency. By analyzing the signal within a certain frequency range, the average power value of the frequency band can be obtained, and the corresponding PSD can be calculated. The calculation formula of PSD is as follows:

[0061]

[0062] In the formula, X(x n ) is the Fourier transform value of n, X* (x n ) is X(x n ) is the conjugate function of .

[0063] Differential entropy is used to measure the complexity of continuous random variables and can reflect the difference and complexity of signal distribution. By calculating the differential entropy of the extracted EEG signal features, the dynamic changes of the signal can be better described. The specific calculation formula of differential entropy is as follows:

[0064]

[0065] Where p(x) is the probability density function of a continuous variable. If a segment of EEG follows a Gaussian distribution Then the differential entropy can be expressed as:

[0066]

[0067] Step 3.3: After obtaining the frequency domain features of the frontal EEG signal, in order to further improve the stability of the features and reduce the interference of irrelevant features, the extracted power spectral density and differential entropy features are smoothed using a linear dynamic system to reduce the influence of irrelevant features. This smoothing process can effectively eliminate the influence of noise or fluctuations, making the features related to the classification task more stable and improving the accuracy of the subsequent classification process.

[0068] Step 4: Based on the progressive fusion strategy, a fatigue detection model is constructed. Effective feature extraction is performed on the time series of power spectrum density and differential entropy features to obtain power spectrum density time features and differential entropy time features. After deep feature extraction, the power spectrum density time features and differential entropy time features are fused to obtain fusion features. The fusion features are classified and identified to obtain fatigue detection results.

[0069] In a specific implementation, in order to further improve the accuracy, the present invention proposes a new network model, such as Figure 2 As shown, the network model constructed by the present invention based on the asymptotic fusion strategy, the model adopts the asymptotic hierarchical fusion strategy, and the model includes a convolution module, a cross-attention and self-attention mechanism module, and a classification module.

[0070] This embodiment introduces an attention mechanism in the multi-level convolution module, and uses the cross-attention mechanism and the self-attention mechanism to fuse the cross-modal hierarchical semantic features of the PSD and DE features, so that the model pays more attention to important feature information. At the same time, a dual-branch architecture is adopted to effectively extract features and related information between the two feature domains, and the intermediate state of the two features is input into the fusion module, making it easier for the model to find high-order semantic features to solve the problem of mode collapse that may occur when the two features are directly fused. The final fused features are identified by the classification module for three categories.

[0071] Step 4.1: Extract effective features from the time series of power spectrum density and differential entropy features respectively to obtain power spectrum density time features and differential entropy time features.

[0072] The dual-branch architecture and multi-level convolution module are used to deeply mine the time series of power spectral density and differential entropy features, respectively, learn the correlation between frequency domain and time domain, and obtain the power spectral density time features and differential entropy time features.

[0073] This embodiment adopts this dual-branch framework, which can effectively extract two features and learn the correlation between different feature domains. The multi-level convolution module is used to extract multi-level semantic information of the two features layer by layer, and combined with the asymptotic hierarchical fusion strategy, multi-level semantic information is extracted layer by layer in each convolution layer, so that the model can obtain richer feature expressions. This design not only enhances the network's ability to learn complex features, but also lays a solid foundation for subsequent feature fusion, thereby significantly improving the performance and classification accuracy of the overall model.

[0074] In this embodiment, the convolution module in the network model based on the gradual hierarchical fusion strategy is as follows: Figure 3 As shown in the figure, it is mainly composed of a multi-level convolution module and a convolution module with residual connection, which aims to improve the overall performance and feature extraction ability of the model. Among them, the multi-level convolution module extracts features layer by layer through the combination of multiple convolution layers and ReLU activation functions, enhances the expression ability of features, and effectively helps the model to deeply understand and model the original features, thereby improving the accuracy of classification tasks. In the multi-level convolution module, a one-dimensional convolution operation is used, and the output dimension is set to 32, 64 and 128, and the corresponding convolution kernel sizes are 1×1, 2×2 and 2×2 respectively. Through this setting, the complexity of features and the ability to extract details can be gradually increased. The mathematical expression of the convolution layer is as follows:

[0075]

[0076] Where: the activation function used is f, * represents the convolution operation, N j is the input feature map of the convolutional layer, Represents the weight matrix of the convolution kernel, is the bias matrix.

[0077] Step 4.2: After deep feature extraction, the power spectrum density time feature and the differential entropy time feature are fused to obtain the fused feature.

[0078] Step 4.2.1: Add the power spectrum density time characteristics and the differential entropy time characteristics to obtain the intermediate state.

[0079] Step 4.2.2: Use the convolution module with residual connection to perform deeper feature extraction on the power spectral density time features and differential entropy time features after multi-layer convolution.

[0080] In a specific implementation, the convolution module with residual connection adopts a 2×2 convolution kernel design, which effectively reduces the computational complexity while retaining the key feature information in the input data, helping the network to better learn complex feature representations. In addition, the module combines batch normalization (BatchNorm) and ReLU activation functions to accelerate the training process of the model and further improve the learning efficiency and generalization ability of the model. By introducing residual connections, not only the convergence speed of the model is accelerated, but also the gradient vanishing problem can be effectively alleviated in the deep network structure, reducing the risk of overfitting, and ensuring the stability and accuracy of the model.

[0081] Step 4.2.3: Use the cross-attention fusion module to update the intermediate state and obtain the updated intermediate state.

[0082] Specifically, the DE temporal features and PSD temporal features are used as the query vector (Q) and the value vector (V), respectively, and the intermediate state is input into the cross-attention fusion module as the key vector (K) to obtain the updated intermediate state.

[0083] The attention module in the model of the present invention plays a key role in the overall model. The structure is as follows: Figure 4 As shown in the figure. This module includes a cross-attention module and a self-attention module. By weighting different features in the EEG signal, it can effectively capture the relationship between features, learn and strengthen the correlation between features. This design highlights the features that are useful for classification tasks and suppresses irrelevant or redundant features, thereby significantly improving the classification accuracy and overall performance of the model.

[0084] The present invention improves the traditional cross-attention mechanism. The traditional cross-attention mechanism refers to the query vector (Q) and the key-value vector (K, V) coming from different feature sets, which is a special form of multi-head attention. When the query vector and the key-value vector come from different sources, it is cross-attention. In the present invention, the DE features and PSD features processed by the convolution module are used as the query vector Q and the value vector V respectively, and the intermediate state of the PSD and DE features after adding the multi-layer convolution module is used as the key vector K, which are input into the improved cross-attention mechanism module together. The attention score is obtained by calculating the dot product of the query vector Q and the key vector K, and is normalized by the Softmax function to obtain the attention weight. Subsequently, the weight is multiplied by the value vector V to generate the final output. Its mathematical formula is as follows:

[0085]

[0086] In this embodiment, the intermediate state is the result of adding the DE and PSD temporal features, representing the initial fusion of the two features. Using it as a key vector can provide a global feature benchmark for the cross-attention module, so that the query (Q) and value (V) can interact based on more comprehensive information in the attention mechanism. Using the intermediate state as a key can avoid directly using unfused original features, reduce interference in the feature space, and make the attention calculation more focused; the intermediate state contains both DE feature information and PSD feature information. Through interaction with queries and values ​​in the cross-attention module, it can effectively capture the deep correlation between feature domains, thereby improving the model's ability to understand multimodal information.

[0087] In the cross-attention mechanism of the present invention, the query vector Q is firstly combined with the key vector K T Multiply them together to get the score, and then divide it by the dimension of the key vector K. To maintain the stability of the gradient. Then, the calculation result is input into the Softmax activation function to obtain the weight value. Finally, the weight is multiplied by the value vector V to calculate the final output vector score. Since the cross attention mechanism is a special form of multi-head attention, the module of the present invention has the characteristics of the multi-head attention mechanism, that is, multiple weight matrices (multiple heads) are used to process different features simultaneously, and each head corresponds to a weight matrix W of the query vector, key vector and value vector Q , W K and W V By processing multiple heads in parallel (such as head1, head2, head3, etc.), the model can capture richer feature representations. The specific calculation formula for each head is as follows:

[0088]

[0089] head n =Attention(Q n ,K n ,V n ).

[0090] In the formula and Represents the weight transformation matrices of the query vector, key vector, and value vector of the nth head, respectively, and head n Represents the output of the nth head. The outputs of multiple heads are concatenated to form a high-dimensional feature vector, which is then transformed through a linear projection layer to obtain the final output result. The specific calculation formula is as follows:

[0091] MultiHead(Q,K,V)=Concat(head1,head2,…,head n )W o .

[0092] In the above formula, n represents the number of heads in the multi-head attention mechanism, W o is the output transformation matrix. Through the linear projection operation of this matrix, the output dimension of the multi-head attention mechanism can be kept consistent with the dimension of the original input, thereby ensuring the coherence and effectiveness of the model structure.

[0093] The cross attention mechanism and multi-head attention mechanism in the present invention are both variants based on the self-attention mechanism. However, the calculation method of the self-attention module is different from them. In the self-attention module of the present invention, the query vector Q, the key vector K and the value vector V are all derived from the output sequence of the previous cross attention module. The input of the self-attention module completely inherits the output of the cross attention module, so that the feature flow is consistent, and the discontinuity of the feature space caused by the introduction of new feature sources is avoided. The module first obtains the attention score by the dot product operation of the query vector Q and the key vector K, and then the score is normalized by the Softmax activation function to generate the attention weight. Finally, the attention weight is multiplied by the value vector V to generate the final output of the self-attention module.

[0094] In this embodiment, the cross-attention module has fused multimodal features (such as DE, PSD, and intermediate states), and its output sequence contains cross-modal correlation information and deep features. Directly using this sequence as Q, K, and V ensures that the self-attention module further focuses and optimizes on the fused features. Since the cross-attention module has captured the correlation between different modalities, the self-attention module further acts on these fused features, and captures the global correlation within the features through its own mechanism, which can extract higher-order semantic information. This gradually optimized hierarchical design (cross first and then self-attention) enables the model to simultaneously focus on the correlation between modalities (cross-attention) and the deep dependency within the modality (self-attention).

[0095] In the attention module of the present invention, in order to further alleviate the overfitting problem of the model, a Dropout layer is specially introduced. By randomly discarding the output of some neurons, the Dropout layer effectively enhances the generalization ability of the model. In addition, residual connections are added to the cross-attention module. This design not only alleviates the gradient vanishing problem, but also significantly improves the learning ability and generalization performance of the model, making the model more stable in the extraction and fusion process of complex features.

[0096] Step 4.2.4: Perform convolution operations with residual connections on the deeper power spectral density time features and differential entropy time features extracted in step 4.2.2 to extract higher-level semantic features and obtain power spectral density semantic features and differential entropy semantic features.

[0097] This convolution operation with residual connection is the same as before. This embodiment adopts two convolution designs with residual connection for multi-level feature extraction, feature optimization and fusion, and stability of model training. This progressive feature extraction strategy can ensure that the model can focus on detail features and extract deep semantic features when processing EEG signals, alleviate the gradient vanishing problem of deep networks, and enable features to retain important information after multi-layer convolution, thereby improving classification performance and model robustness.

[0098] Step 4.2.5: Input the power spectral density semantic features, differential entropy semantic features and the updated intermediate states into the cross-attention fusion module for fusion to obtain the fused features.

[0099] Specifically, the DE features, PSD features and updated intermediate states after convolution are input into the cross-attention fusion module again as Q, V and K respectively.

[0100] This embodiment uses a cross-attention fusion module with the same structure as the previous one to process step 4.2.5. The advantage of this setting is that the first cross-attention fuses information from different feature domains (such as DE, PSD features, and intermediate states) and updates the relationship between features. In this way, the model can capture the preliminary correlation between modalities. The second cross-attention further deepens the fusion processing of the results output by the first cross-attention, further strengthening the relationship between features. Through two cross-attentions, the model can more comprehensively capture the deep dependencies between different modalities, thereby improving the fusion effect of information.

[0101] Step 4.3: Classify and identify the fused features. The specific steps to obtain fatigue detection results are as follows:

[0102] The fused features pass through the self-attention module and enter the classification module composed of two fully connected layers and normalization processing to achieve three-category recognition of fatigue status.

[0103] In a specific embodiment, Figure 5 As shown, the model of the present invention obtains the final fusion features after processing the first few modules, and inputs them into the classification module for classification and prediction. First, the classification module reduces the dimension of the features through a maximum pooling layer with a pooling window size of 2 and a step size of 1. Subsequently, the multidimensional features are flattened into a one-dimensional vector using the Flatten layer to facilitate processing by the fully connected layer. The first fully connected layer maps the flattened features to 64 neurons and enhances the nonlinear expression through the ReLU activation function. Finally, the features are mapped to the classification label space through another fully connected layer to complete the classification task.

[0104] Among them, the three-classification identification of fatigue state refers to the three states of wakefulness, fatigue and sleepiness. The sleepiness state is usually manifested as extreme fatigue, decreased attention, and possible dozing off, but not necessarily completely closing eyes to fall asleep. This is a transitional state between wakefulness and complete sleep. The fatigue state refers to the driver feeling physically and mentally exhausted, but has not yet reached the level of sleepiness. Fatigue can lead to decreased attention, slower reaction speed, and may feel sleepy, but usually there will be no dozing off. Or the driver can still stay awake, but the alertness of driving is greatly reduced, and it may not be possible to concentrate for a long time. The difference between fatigue and sleepiness is that fatigue usually refers to physical and mental fatigue. The driver may feel sleepy, but still maintain a certain degree of alertness. Sleepiness is a further aggravation of fatigue. The driver may show signs of dozing off and extreme sleepiness, close to falling asleep, and his alertness and responsiveness are significantly reduced, which poses a great safety hazard. The method of this embodiment can detect the fatigue state before the driver enters the sleepiness state, reduce the driving risk as much as possible, and achieve safe driving.

[0105] The present invention performs a preprocessing operation on the original EEG signal; then extracts PSD and DE features from the preprocessed EEG signal and smoothes it; then extracts deeper features of PSD and DE through a multimodal asymptotic hierarchical fusion network, adopts asymptotic fusion strategy to fuse multiple modules, and better fuses the two features; finally, through a classification module, and uses K-fold cross validation to verify the performance of the model, an accuracy rate of 95.77% is achieved, which fully demonstrates the high efficiency and convenience of the method of the present invention.

[0106] Embodiment 2:

[0107] Embodiment 2 of the present invention provides a fatigue detection system based on forehead sparse EEG signals, including:

[0108] A data acquisition module, configured to acquire a frontal EEG signal dataset;

[0109] A signal and processing module, configured to pre-process the frontal EEG signal data;

[0110] The feature extraction module is configured to divide the frontal EEG signal by using a frequency band division method, and perform frequency domain analysis on each frequency band to obtain frequency domain features of the frontal EEG signal, wherein the frequency domain features include power spectrum density and differential entropy features;

[0111] The multimodal asymptotic hierarchical network module is configured to build a fatigue detection model based on a progressive fusion strategy, perform effective feature extraction on the time series of power spectral density and differential entropy features respectively to obtain power spectral density time features and differential entropy time features, perform deep feature extraction on the power spectral density time features and differential entropy time features and then fuse them to obtain fused features; classify and identify the fused features to obtain fatigue detection results.

[0112] Embodiment three:

[0113] Embodiment 3 of the present invention provides a medium on which a program is stored. When the program is executed by a processor, the steps of the fatigue detection method based on forehead sparse EEG signals as described in Embodiment 1 of the present invention are implemented.

[0114] Embodiment 4:

[0115] Embodiment 4 of the present invention provides a device, including a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps of the fatigue detection method based on forehead sparse EEG signals as described in Embodiment 1 of the present invention are implemented.

[0116] The steps involved in the above embodiments 2, 3 and 4 correspond to the method embodiment 1. For the specific implementation methods, please refer to the relevant description part of embodiment 1.

[0117] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0118] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A fatigue detection method based on forehead sparse EEG signals, characterized in that: The following steps are involved: Get the frontal EEG signal dataset; Preprocess the frontal EEG signal data; The frontal EEG signal is divided by using the frequency band division method, and each frequency band is analyzed in the frequency domain to obtain the frequency domain features of the frontal EEG signal, wherein the frequency domain features include power spectrum density and differential entropy features; A fatigue detection model is constructed based on a progressive fusion strategy. Effective feature extraction is performed on the time series of power spectrum density and differential entropy features to obtain power spectrum density time features and differential entropy time features. After deep feature extraction, the power spectrum density time features and differential entropy time features are fused to obtain fusion features. The fused features are classified and identified to obtain fatigue detection results.

2. The fatigue detection method based on forehead sparse EEG signals as claimed in claim 1, characterized in that: The steps of preprocessing the frontal EEG signal data include filtering, denoising and artifact removal.

3. The fatigue detection method based on forehead sparse EEG signals as claimed in claim 1, characterized in that: The specific steps of dividing the frontal EEG signal using the frequency band division method and performing frequency domain analysis on each frequency band are as follows: Using the 5-band division method, the EEG signal of each channel is divided into 5 frequency bands; The power spectral density and differential entropy characteristics of each frequency band are calculated by short-time Fourier transform.

4. The fatigue detection method based on forehead sparse EEG signals as claimed in claim 3, characterized in that: After obtaining the frequency domain features of the frontal EEG signal, the extracted power spectrum density and differential entropy features are smoothed using a linear dynamic system.

5. The fatigue detection method based on forehead sparse EEG signals as claimed in claim 1, characterized in that: The specific steps for effective feature extraction of the time series of power spectrum density and differential entropy features are as follows: The dual-branch architecture and multi-level convolution module are used to deeply mine the time series of power spectral density and differential entropy features, respectively, learn the correlation between frequency domain and time domain, and obtain the power spectral density time features and differential entropy time features.

6. The fatigue detection method based on forehead sparse EEG signals as claimed in claim 1, characterized in that: The specific steps of fusing the power spectrum density time feature and the differential entropy time feature after deep feature extraction are as follows: The power spectrum density time characteristic and the differential entropy time characteristic are added to obtain the intermediate state; A convolution module with residual connection is used to extract deeper features from the power spectrum density time features and differential entropy time features after multi-layer convolution, and a cross-attention fusion module is used to update the intermediate state to obtain the updated intermediate state. Perform convolution operations with residual connections on the deeper power spectrum density time features and differential entropy time features to extract higher-level semantic features, thereby obtaining power spectrum density semantic features and differential entropy semantic features. The power spectral density semantic features, differential entropy semantic features and the updated intermediate states are input into the cross attention fusion module for fusion to obtain the fused features.

7. The fatigue detection method based on forehead sparse EEG signals as claimed in claim 1, characterized in that: The specific steps for classifying and identifying the fused features and obtaining fatigue detection results are as follows: The fused features pass through the self-attention module and enter the classification module composed of two fully connected layers and normalization processing to achieve three-category recognition of fatigue status.

8. A fatigue detection system based on forehead sparse EEG signals, characterized in that: include: A data acquisition module, configured to acquire a frontal EEG signal dataset; A signal and processing module, configured to pre-process the frontal EEG signal data; The feature extraction module is configured to divide the frontal EEG signal by using a frequency band division method, and perform frequency domain analysis on each frequency band to obtain frequency domain features of the frontal EEG signal, wherein the frequency domain features include power spectrum density and differential entropy features; The multimodal asymptotic hierarchical network module is configured to build a fatigue detection model based on a progressive fusion strategy, perform effective feature extraction on the time series of power spectral density and differential entropy features respectively to obtain power spectral density time features and differential entropy time features, perform deep feature extraction on the power spectral density time features and differential entropy time features and then fuse them to obtain fused features; classify and identify the fused features to obtain fatigue detection results.

9. A computer-readable storage medium, characterized in that: A plurality of instructions are stored therein, and the instructions are suitable for being loaded by a processor of a terminal device and executing the fatigue detection method based on forehead sparse EEG signals as described in any one of claims 1-7.

10. A terminal device, characterized in that: It includes a processor and a computer-readable storage medium, the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, and the instructions are suitable for being loaded by the processor and executing the fatigue detection method based on forehead sparse EEG signals described in any one of claims 1-7.

Images