Deep learning-based cross-subject driving fatigue detection system

Abstract

This invention discloses a deep learning-based cross-subject driving fatigue detection system, comprising: a data processing module for loading and processing EEG signals from a subject's driving process to obtain driving fatigue EEG data of uniform size and identical features; a model training module for designing and building an ensemble network combining local and global features, training it using the processed driving fatigue EEG data, and obtaining the trained ensemble network as the main model; a model fine-tuning module for fine-tuning the main model using EEG data from a portion of a new subject's driving process to obtain the optimal detection model for that subject; and a cross-subject driving fatigue detection module for performing cross-driving fatigue detection tasks using the optimal detection model and saving the optimal model parameters. This invention combines deep learning technology with brain-computer interface technology to improve the performance of cross-subject driving fatigue detection, enhancing its accuracy and generalization.

Description

Technical Field

[0001] This invention relates to the technical fields of deep learning and brain-computer interfaces, and in particular to a deep learning-based cross-subject driving fatigue detection system. Background Technology

[0002] Fatigue can severely impair a driver's mental state, attention, and judgment, thereby increasing the risk of traffic accidents. Driver fatigue detection plays a crucial role in reducing traffic accidents. Among these methods, electroencephalography (EEG) signals are considered the gold standard for fatigue detection because they directly reflect a driver's mental state. However, due to the high noise levels and significant individual differences in EEG signals, cross-subject fatigue detection based on EEG still faces many challenges.

[0003] In the research field, machine learning methods have been widely used for fatigue detection using EEG signal features. These methods include, but are not limited to, wavelet transform, kernel principal component analysis, and support vector machines, to improve classification accuracy and efficiency. On the other hand, advances in deep learning technology have also revolutionized EEG signal processing. Deep learning methods, such as autoencoders, convolutional neural networks, and long short-term memory networks, have demonstrated superior performance in driver fatigue detection, especially when processing large and high-dimensional data.

[0004] However, these methods still face challenges in cross-subject studies, primarily due to inter-individual variability in EEG signals. To address this challenge, transfer learning has emerged as a prominent research direction in recent years. By reducing inter-subject variability, transfer learning promises to improve the generality and accuracy of models. Although existing transfer strategies have shown potential, they typically rely on large amounts of new subject data, limiting their feasibility for online use. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings and disadvantages of the prior art and provide a cross-subject driving fatigue detection system based on deep learning. This system combines deep learning technology with brain-computer interface technology to improve the performance of cross-subject driving fatigue detection, and enhance the accuracy and generalization of cross-subject driving fatigue detection.

[0006] To achieve the above objectives, the technical solution provided by this invention is: a deep learning-based cross-subject driving fatigue detection system, comprising:

[0007] The data processing module is used to load the EEG signals of the test subjects during driving and to perform feature processing on the EEG signals to obtain driving fatigue EEG data of uniform size and the same characteristics.

[0008] The model training module designed and built an integrated network combining local and global features, called LFP-GFPNet. This integrated network consists of two parts: a local feature extraction network, LFPNet, which uses a residual convolutional neural network, BiLSTM layers, additive attention mechanisms, and fully connected layers to extract local features from driving fatigue EEG data; and a global feature extraction network, GFPNet, which uses a one-dimensional convolutional neural network, positional encoding, a Transformer encoder, and fully connected layers to extract global features from driving fatigue EEG data. The LFP-GFPNet was trained using driving fatigue EEG data processed by the data processing module, and the trained LFP-GFPNet was used as the main model for cross-subject driving fatigue detection tasks.

[0009] The model fine-tuning module uses EEG data from a portion of the new subject's driving process to fine-tune the main model, obtaining the optimal detection model for that subject, improving the generalization ability of the main model in cross-subject driving fatigue detection tasks and improving the cross-subject driving fatigue detection effect for new subjects.

[0010] The cross-subject driving fatigue detection module uses the optimal detection model to perform cross-driving fatigue detection tasks and saves the optimal model parameters.

[0011] Furthermore, the data processing module includes a data loading module, a data preprocessing module, and a feature extraction module, wherein:

[0012] The data loading module reads multiple sets of EEG signals from the test subjects during their driving process from the local machine. The EEG signals include signal values ​​and label values. The signal values ​​are composed of channel values ​​from A channels, with each channel having a sampling frequency of F and a sampling time of t. The label values ​​reflect the degree of driving fatigue. Based on the different degrees of fatigue, the label values ​​are divided into three levels: 0 represents a conscious state, 1 represents a fatigued state, and 2 represents a drowsy state.

[0013] The data preprocessing module is used to perform bandpass filtering of 1-70Hz on the divided signal values ​​to reduce noise interference and obtain preprocessed signal values.

[0014] The feature extraction module is used to extract DE features, also known as differential entropy features, from the preprocessed signal values. The specific operation is as follows:

[0015] The DE feature is obtained using the logarithmic spectral feature, and the formula is as follows:

[0016]

[0017] In the formula, DE r σ r and P rThese represent the DE characteristics, variance, and spectral characteristics of the EEG signal in the r-band, respectively, with const being a constant.

[0018] For the spectral feature P, the short-time Fourier algorithm is used to calculate:

[0019]

[0020]

[0021] In the formula, j is the imaginary sign; w[nm] is the window function of the short-time Fourier transform, where a Hamming window is chosen; m represents the translation parameter of the window function w[nm]; n = 1, 2, ..., N is the number of sample points in the discrete-time index, and N is the total number of sample points; v is the angular frequency; x[n] is the EEG time-series signal at time index n, and X(m,ω) is the result obtained by the short-time Fourier transform;

[0022] First, feature extraction is performed on the preprocessed signal values. A differential entropy feature is extracted every 2 Hz in the frequency range from 1 Hz to 50 Hz, with the last feature ranging from 49 Hz to 50 Hz. Next, a linear dynamic system is used to smooth the extracted features. Finally, the features are normalized to obtain the final signal values ​​after feature extraction.

[0023] Furthermore, the model training module performs the following operations:

[0024] 1) Train the local feature extraction network LFPNet, which includes a residual convolutional neural network, BiLSTM layers, additive attention mechanism, and fully connected layers;

[0025] The signal values ​​after feature extraction are used as input to LFPNet. LFPNet first uses a residual convolutional neural network to extract deep local features. The core components of the residual convolutional neural network include residual blocks and pooling layers.

[0026] Above the residual block are two 3×3 convolutional kernels with predetermined input and output channel numbers; below, a 1×1 convolutional kernel is used to adjust the number of input channels so that the number of output channels matches the number of output channels above the residual block; by adding the outputs above and below the residual block during forward propagation, the gradient vanishing problem can be effectively prevented and the training speed of the residual convolutional neural network can be accelerated.

[0027] The pooling layer uses max pooling, which compresses features by removing redundant information, simplifies network complexity, and reduces computational cost. Max pooling helps retain key features while reducing the dimensionality of the feature map, thus improving computational efficiency.

[0028] After the input of LFPNet passes through the residual convolutional neural network, the BiLSTM layer is introduced to effectively solve the long-term dependency problem in the neural network, thereby achieving effective memory of long-term information.

[0029] Subsequently, an additive attention mechanism is employed to assign different weights to different input positions. The core idea of ​​the additive attention mechanism is that for the decoding of each output unit, a corresponding weight is assigned to each position in the input sequence, ensuring that LFPNet can focus its attention on the input position most relevant to the current output position. This design enables LFPNet to process information in the input sequence more flexibly and accurately, improving LFPNet's learning ability on long-distance dependencies. The specific calculation steps are as follows: First, for the output of each BiLSTM layer, a random query q and key k are assigned, and the additive attention score E is defined as:

[0030] E(q,k)=v T tanh(W q q+W k k)

[0031] In the formula, v is the weight vector of the additive attention score E, v T W represents the transpose of v. q and W k These are the weight matrices for the query and the key, respectively; next, the softmax function is used to calculate the attention weight α:

[0032]

[0033] In the formula, e E(q,k) It is the exponent of E(q,k), and k′ is the index of all elements in the set of key k vectors;

[0034] We obtain the context vector c, which is a weighted average of all keys:

[0035] c = ∑ j α(q,k j V j

[0036] In the formula, V j Is with key k j The associated value vector, where j is the index of all elements in the value vector set;

[0037] Finally, c is input into the fully connected layer to obtain the predicted label value;

[0038] 2) Train the global feature network GFPNet, which includes a one-dimensional convolutional neural network, positional encoding, a Transformer encoder, and fully connected layers;

[0039] The signal values ​​after feature extraction are used as input values ​​for GFPNet. GFPNet first uses a one-dimensional convolutional neural network with a kernel of 1 to change the number of channels in the input values, providing suitable input feature dimensions for subsequent positional encoding and the Transformer encoder; let the input sequence be... Where R represents the set of real numbers, N′ is the sequence length, and d input It is the input dimension; the convolution operation is represented as:

[0040] x′ conv [i] = w·x′[i] + b

[0041] In the formula, i is the i-th value in the input sequence x′, and x′ conv The output is the convolution output, w is the weight of the input sequence x′, and b is the bias.

[0042] Then x′ conv Position encoding is performed, which uses mathematical methods to encode positional information into time-series data, thereby giving GFPNet the ability to recognize the relative positional relationships of points in the input. The positional encoding PE is calculated as follows:

[0043]

[0044]

[0045] In the formula, d conv The convolutional dimension is the feature dimension; pos is the position index of the data in the input sequence; i′ is the dimension index in the positional encoding vector; and the positional encoding output x′ is the output x′. pos =x′ conv +PE.

[0046] The Transformer encoder is a deep learning model composed of a multi-layer self-attention mechanism and a feedforward neural network. Through the self-attention mechanism, the Transformer encoder can consider all inputs globally, allowing GFPNet to capture complex patterns in time series more flexibly. In GFPNet, the Transformer encoder is used to encode the position and then perform self-attention calculation on the output. The calculation result is then input into a fully connected layer to obtain another predicted label value.

[0047] 3) Construct an integrated network LFP-GFPNet that combines local and global features:

[0048] After training the local feature extraction network LFPNet and the global feature network GFPNet using the signal values ​​after feature extraction, the outputs of the fully connected layers of the local feature extraction network LFPNet and the global feature network GFPNet are connected with weights of 0.4:0.6 to obtain the final predicted label value, where the output weight of LFPNet is 0.4 and the output weight of GFPNet is 0.6; the trained LFP-GFPNet is then used as the main model.

[0049] Furthermore, the model fine-tuning module performs the following operations:

[0050] 1) Collect EEG signals from new subjects during driving, process the data through the data processing module, select 20% of the data as the fine-tuning training set based on the label values, and use the remaining data as the test set to carry out a cross-subject driving fatigue detection task.

[0051] 2) Maintain the original learning rate for the fully connected layers of the main model to ensure that they can learn the unique high-level features of new subjects; while for the rest of the main model, reduce the learning rate to one-tenth of the original, so that the rest of the main model can be fine-tuned without destroying the general features they have learned.

[0052] 3) Train the main model using the fine-tuned training set to obtain the optimal detection model for new subjects.

[0053] Furthermore, the cross-subject driving fatigue detection module performs the following operations:

[0054] 1) Select the test set divided by the model fine-tuning module, input the optimal detection model to perform cross-subject driving fatigue detection task, and obtain cross-subject driving fatigue detection results;

[0055] 2) Save the obtained optimal model parameters so that when the subject performs cross-subject driving fatigue detection tasks in the future, the saved parameters can be directly called to improve efficiency and accuracy.

[0056] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0057] 1. This invention constructs a Local Feature Extraction Network (LFPNet) to extract local features of EEG signals during driving. By accurately capturing local patterns and subtle changes in EEG signals during driving, LFPNet can more effectively identify and predict EEG activities related to driving fatigue, thereby providing accurate and timely assessment of driving fatigue.

[0058] 2. This invention constructs a global feature extraction network GFPNet to extract global features of EEG signals during driving. By comprehensively analyzing the overall patterns and trends of EEG signals, GFPNet can effectively reveal the driver's overall cognitive state and fatigue level.

[0059] 3. This invention integrates LFPNet and GFPNet into LFP-GFPNet, combining the advantages of both networks: LFPNet's efficiency in capturing local details of EEG signals and GFPNet's comprehensiveness in analyzing overall EEG activity patterns. Through this integration, LFP-GFPNet can more comprehensively and accurately assess driving fatigue, improving the network's accuracy in cross-subject driving fatigue detection.

[0060] 4. This invention only requires fine-tuning the trained LFP-GFPNet using a small amount of EEG data from the subject, which can further improve the network's generalization ability and the subject's detection performance during cross-subject driving fatigue. Attached Figure Description

[0061] Figure 1 This is a schematic diagram showing the relationship between the various modules of the system of the present invention.

[0062] Figure 2 This is a diagram illustrating the relationship between the data processing module and the model training module.

[0063] Figure 3 This is a diagram of the residual convolutional neural network structure in the local feature extraction network.

[0064] Figure 4 This is a flowchart of the fatigue detection process for cross-subject driving using the system of the present invention. Detailed Implementation

[0065] The present invention will be further described below with reference to specific embodiments.

[0066] This embodiment discloses a deep learning-based cross-subject driving fatigue detection system. It is a cross-subject driving fatigue detection system developed using Python and capable of running on Windows devices. The relationships between the system modules are as follows: Figure 1 As shown, it includes:

[0067] The data processing module is used to load the EEG signals of the test subjects during driving and perform feature processing on the EEG signals to obtain driving fatigue EEG data of uniform size and the same characteristics.

[0068] The model training module designed and built an integrated network combining local and global features, called LFP-GFPNet. This network consists of two parts: a local feature extraction network, LFPNet, which uses a residual convolutional neural network, BiLSTM layers, additive attention mechanisms, and fully connected layers to extract local features from driving fatigue EEG data; and a global feature extraction network, GFPNet, which uses a one-dimensional convolutional neural network, positional encoding, a Transformer encoder, and fully connected layers to extract global features from the driving fatigue EEG data. The trained LFP-GFPNet was then used as the main model for cross-subject driving fatigue detection tasks, trained using driving fatigue EEG data processed by the data processing module.

[0069] The model fine-tuning module uses EEG data from a portion of the new subject's driving process to fine-tune the main model, obtaining the optimal detection model for that subject. This improves the generalization ability of the main model in cross-subject driving fatigue detection tasks and enhances the cross-subject driving fatigue detection effect for new subjects.

[0070] The cross-subject driving fatigue detection module uses the optimal detection model to perform cross-driving fatigue detection tasks and saves the optimal model parameters.

[0071] The flowcharts for the data processing module and the model training module are as follows: Figure 2 As shown in the diagram, the data loading module reads multiple sets of EEG signals from the test subjects during their driving process. The EEG signals include signal values ​​and label values. The signal values ​​consist of channel values ​​from A channels, with each channel having a sampling frequency of F and a sampling time of t. The label values ​​reflect the degree of driving fatigue and are divided into three levels based on the degree of fatigue: 0 represents a conscious state, 1 represents a fatigued state, and 2 represents a drowsy state.

[0072] The data preprocessing module is used to perform bandpass filtering of the divided signal values ​​from 1 to 70 Hz to reduce noise interference and obtain preprocessed signal values.

[0073] The feature extraction module is used to extract DE features, or differential entropy features, from the preprocessed signal values. The specific steps are as follows:

[0074] The DE feature is obtained using the logarithmic spectral feature, and the formula is as follows:

[0075]

[0076] Among them, DE r σ r and P r represents the DE characteristics, variance, and spectral characteristics of the EEG signal in the r-band, respectively, with const being a constant.

[0077] For the spectral feature P, the short-time Fourier algorithm is used to calculate:

[0078]

[0079]

[0080] In the formula, j is the imaginary number; w[nm] is the window function of the short-time Fourier transform, where a Hamming window is selected; m represents the translation parameter of the window function w[nm]; n = 1, 2, ..., N is the number of sample points in the discrete-time index, and N is the total number of sample points; ω is the angular frequency; x[n] is the EEG time-series signal at time index n, and X(m,ω) is the result obtained by the short-time Fourier transform;

[0081] Based on the aforementioned method for DE feature extraction, we first extract features from the preprocessed signal values. Within the frequency range of 1Hz to 50Hz, we extract a differential entropy feature every 2Hz, with the last feature ranging from 49Hz to 50Hz. Next, we use a linear dynamical system to smooth the extracted features, and finally normalize the features to obtain the final signal values ​​after feature extraction.

[0082] Next, the signal values ​​after feature extraction are input into the local feature extraction network LFPNet and the global feature extraction network GFPNet, respectively, to train the two networks.

[0083] For LFPNet, the signal values ​​after feature extraction are used as input values. LFPNet first uses a residual convolutional neural network to extract deep local features. The residual convolutional neural network is as follows: Figure 3 As shown, it consists of two residual blocks and a pooling layer. Above the residual blocks are two 3×3 convolutional kernels with predetermined input and output channel numbers; below, a 1×1 convolutional kernel is used to adjust the number of input channels so that the number of output channels matches the number of output channels above the residual block. By summing the outputs above and below the residual blocks during forward propagation, the gradient vanishing problem can be effectively prevented, and the training speed of the residual convolutional neural network can be accelerated.

[0084] Furthermore, the pooling layer employs max pooling, which compresses features by removing redundant information, simplifying network complexity and reducing computational cost. Max pooling helps retain key features while reducing the dimensionality of the feature map, thus improving computational efficiency. This series of design decisions aims to optimize LFPNet's performance in extracting local features.

[0085] After the input passes through the residual convolutional neural network, a BiLSTM layer is introduced to effectively solve the long-term dependency problem in the neural network, thereby achieving effective memorization of long-term information.

[0086] Subsequently, an additive attention mechanism is employed to assign different weights to different input positions. The core idea of ​​the additive attention mechanism is that for the decoding of each output unit, a corresponding weight is assigned to each position in the input sequence, ensuring that LFPNet can focus its attention on the input position most relevant to the current output position. This design enables LFPNet to process information in the input sequence more flexibly and accurately, improving LFPNet's learning ability on long-distance dependencies. The specific calculation steps are as follows: First, for the output of each BiLSTM layer, a random query q and key k are assigned, and the additive attention score E is defined as:

[0087] E(q,k)=v T tanh(W q q+W k k)

[0088] Where v is the weight vector of the additive attention score E, v T W represents the transpose of v. q and W k These are the weight matrices for the query and the key, respectively. Next, we use the softmax function to calculate the attention weight α:

[0089]

[0090] Among them, e E(q,k) It is the exponent of E(q,k), and k′ is the index of all elements in the set of key k vectors.

[0091] We obtain the context vector c, which is a weighted average of all keys:

[0092] c = ∑ j α(q,k j V j

[0093] Where V j Is with key k j The associated value vector, where j is the index of all elements in the value vector set.

[0094] Finally, c is input into the fully connected layer to obtain the predicted label value.

[0095] For GFPNet,

[0096] The signal values ​​after feature extraction are used as input values ​​for GFPNet. GFPNet first uses a one-dimensional convolutional neural network with a kernel of 1 to change the number of channels in the input values, providing suitable input feature dimensions for subsequent positional encoding and the Transformer encoder; let the input sequence be... Where R represents the set of real numbers, N′ is the sequence length, and d inputIt is the input dimension; the convolution operation is represented as:

[0097] x′ conv [i] = w·x′[i] + b

[0098] In the formula, i is the i-th value in the input sequence x′, and x′ conv The output is the convolution output, w is the weight of the input sequence x′, and b is the bias.

[0099] Then x′ conv Position encoding is performed, which uses mathematical methods to encode positional information into time-series data, thereby giving GFPNet the ability to recognize the relative positional relationships of points in the input. The positional encoding PE is calculated as follows:

[0100]

[0101]

[0102] In the formula, d conv The convolutional dimension is the feature dimension; pos is the position index of the data in the input sequence; i′ is the dimension index in the positional encoding vector; and the positional encoding output x′ is the output x′. pos =x′ conv +PE.

[0103] The Transformer encoder is a deep learning model composed of a multi-layer self-attention mechanism and a feedforward neural network. Through the self-attention mechanism, the Transformer encoder can consider all inputs globally, allowing GFPNet to capture complex patterns in time series more flexibly. In GFPNet, the Transformer encoder is used to perform self-attention calculation on the output after position encoding, and the calculation result is input into a fully connected layer to obtain another predicted label value.

[0104] After training the local feature extraction network LFPNet and the global feature extraction network GFPNet using the extracted signal and label values, the outputs of the fully connected layers of LFPNet and GFPNet are concatenated with weights of 0.4:0.6 to obtain the final predicted label value. The weight of LFPNet is 0.4, and the weight of GFPNet is 0.6. The final predicted class G can be calculated using the following formula:

[0105] Y1 = Softmax(Model1(S))

[0106] Y2 = Softmax(Model2(S))

[0107] Y = 0.4 × Y1 + 0.6 × Y2

[0108] G = argmax(Y)

[0109] Where S is the signal value after feature extraction, Model1 and Model2 represent LFPNet and GFPNet respectively, Y1 and Y2 are the outputs of LFPNet and GFPNet respectively, which are probability vectors with three values, and Y is the weighted probability vector. G is the final predicted category obtained from the probability vector.

[0110] Fine-tuning process Figure 4 As shown, the trained LFP-GFPNet was used as the main model. EEG signals from new test subjects during driving were collected and processed using a data processing module. 20% of the data was selected as the training set for fine-tuning, and the remaining 80% was used as the test set for a cross-subject driving fatigue detection task.

[0111] The fully connected layers of the main model retain their original learning rate to ensure they learn the unique high-level features of new subjects. For the rest of the main model, the learning rate is reduced to one-tenth of its original value. This allows for fine-tuning of the remaining structures without destroying the general features they have previously learned.

[0112] The main model is trained using a fine-tuned training set to obtain the optimal detection model for the subject.

[0113] The cross-subject driving fatigue detection module performs the following operations:

[0114] The test set is input into the optimal detection model to perform a cross-subject driving fatigue detection task, and the cross-subject driving fatigue detection results are obtained.

[0115] Save the obtained optimal model parameters. This way, when the subject performs a cross-subject driving fatigue detection task in the future, these saved parameters can be directly called, improving efficiency and accuracy.

[0116] The above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Therefore, any changes made in accordance with the shape and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A deep learning-based cross-subject driving fatigue detection system, characterized in that, include: The data processing module is used to load the EEG signals of the test subjects during driving and to perform feature processing on the EEG signals to obtain driving fatigue EEG data of uniform size and the same characteristics. The model training module designed and built an ensemble network combining local and global features, called LFP-GFPNet. This ensemble network consists of two parts: a local feature extraction network, LFPNet, which uses a residual convolutional neural network, BiLSTM layers, additive attention mechanisms, and fully connected layers to extract local features from driving fatigue EEG data; and a global feature extraction network, GFPNet, which uses a one-dimensional convolutional neural network, positional encoding, a Transformer encoder, and fully connected layers to extract global features from the driving fatigue EEG data. The LFP-GFPNet is trained using driving fatigue EEG data processed by the data processing module, and serves as the main model for cross-subject driving fatigue detection tasks. Specifically, the local feature extraction network LFPNet is trained, which includes a residual convolutional neural network, BiLSTM layers, additive attention mechanisms, and fully connected layers. The extracted signal values ​​are used as input to LFPNet. First, a residual convolutional neural network (RCN) is used to extract deep local features. The core components of the RCN include residual blocks and pooling layers. Above the residual block are two 3×3 convolutional kernels with predetermined input and output channels. Below, a 1×1 convolutional kernel is used to adjust the number of input channels so that the number of output channels matches the number of output channels above the residual block. By adding the outputs above and below the residual block during forward propagation, the vanishing gradient problem can be effectively prevented, and the training speed of the RCN can be accelerated. The pooling layer uses max pooling, which compresses features by removing redundant information, simplifies network complexity, and reduces computational cost. Max pooling helps to retain key features while reducing the dimensionality of the feature map, thus improving computational efficiency. After the input of LFPNet passes through the RCN, a BiLSTM layer is introduced to effectively solve the long-term dependency problem in the neural network, thereby achieving effective memorization of long-term information. Subsequently, an additive attention mechanism is used to assign different weights to different input positions. The model fine-tuning module uses EEG data from a portion of the new subject's driving process to fine-tune the main model, obtaining the optimal detection model for that subject, improving the generalization ability of the main model in cross-subject driving fatigue detection tasks and improving the cross-subject driving fatigue detection effect for new subjects. The cross-subject driving fatigue detection module uses the optimal detection model to perform cross-driving fatigue detection tasks and saves the optimal model parameters.

2. The deep learning-based cross-subject driving fatigue detection system according to claim 1, characterized in that, The data processing module includes a data loading module, a data preprocessing module, and a feature extraction module, wherein: The data loading module reads multiple sets of EEG signals from the test subjects during their driving process from the local machine. The EEG signals include signal values ​​and label values. The signal values ​​are composed of channel values ​​from A channels, with each channel having a sampling frequency of F and a sampling time of t. The label values ​​reflect the degree of driving fatigue. Based on the different degrees of fatigue, the label values ​​are divided into three levels: 0 represents a conscious state, 1 represents a fatigued state, and 2 represents a drowsy state. The data preprocessing module is used to perform bandpass filtering of 1-70Hz on the divided signal values ​​to reduce noise interference and obtain preprocessed signal values. The feature extraction module is used to extract DE features, also known as differential entropy features, from the preprocessed signal values. The specific operation is as follows: The DE feature is obtained using the logarithmic spectral feature, and the formula is as follows: ； In the formula, , and These represent the brain signals at... DE characteristics, variance, and spectral characteristics within the frequency band. It is a constant; For spectral characteristics The short-time Fourier algorithm is used to calculate: ； ； Where, It is the symbol for imaginary numbers; It is a window function of the short-time Fourier transform, where a Hamming window is chosen; Representing window functions Translation parameters; It is the number of sample points in the discrete-time index. This represents the total number of sample points. It is angular frequency; It is a time index EEG timing signals at any given moment It is the result obtained from the short-time Lie transform; First, feature extraction is performed on the preprocessed signal values. A differential entropy feature is extracted every 2 Hz in the frequency range from 1 Hz to 50 Hz, with the last feature ranging from 49 Hz to 50 Hz. Next, a linear dynamic system is used to smooth the extracted features. Finally, the features are normalized to obtain the final signal values ​​after feature extraction.

3. The deep learning-based cross-subject driving fatigue detection system according to claim 2, characterized in that, The model training module also includes: 1) The core idea of ​​the additive attention mechanism is: for the decoding of each output unit, assign corresponding weights to each position in the input sequence, ensuring that LFPNet can focus its attention on the input position most related to the current output position; this design enables LFPNet to process information in the input sequence more flexibly and accurately, improving LFPNet's learning ability on long-distance dependencies; the specific calculation steps are: first, for the output of each BiLSTM layer, assign a random query... s and keys Define additive attention score for: ； Where, It is an additive attention score The weight vector, express transpose, and These are the weight matrices for the query and the key, respectively; next, the softmax function is used to calculate the attention weights. : ； In the formula, yes The index, Is a key The indices of all elements in the vector set; Get the context vector It is a weighted average of all bonds: ； In the formula, Is and key Related value vectors, It is the index of all elements in the value vector set; Finally, The input is fed into a fully connected layer to obtain the predicted label value; 2) Train the global feature network GFPNet, which includes a one-dimensional convolutional neural network, positional encoding, a Transformer encoder, and fully connected layers; The signal values ​​after feature extraction are used as input values ​​for GFPNet. GFPNet first uses a one-dimensional convolutional neural network with a kernel of 1 to change the number of channels in the input values, providing suitable input feature dimensions for subsequent positional encoding and the Transformer encoder; let the input sequence be... ,in Represents the set of real numbers. It is the sequence length. It is the input dimension; the convolution operation is represented as: ； Where, For the input sequence The first in One value, For convolution output, For the input sequence The weight, For bias; Subsequently Position encoding is performed, which uses mathematical methods to encode positional information into time-series data, thereby giving GFPNet the ability to recognize the relative positional relationships of points in the input; position encoding Calculate as follows: ； ； Where, It is the feature dimension after convolution; It is the position index of the data in the input sequence; It is the dimension index in the position-encoded vector; the output after position encoding. ; The Transformer encoder is a deep learning model composed of a multi-layer self-attention mechanism and a feedforward neural network. Through the self-attention mechanism, the Transformer encoder can consider all inputs globally, allowing GFPNet to capture complex patterns in time series more flexibly. In GFPNet, the Transformer encoder is used to encode the position and then perform self-attention calculation on the output. The calculation result is then input into a fully connected layer to obtain another predicted label value. 3) Construct an integrated network LFP-GFPNet that combines local and global features: After training the local feature extraction network LFPNet and the global feature network GFPNet using the signal values ​​after feature extraction, the outputs of the fully connected layers of the local feature extraction network LFPNet and the global feature network GFPNet are connected with weights of 0.4:0.6 to obtain the final predicted label value, where the output weight of LFPNet is 0.4 and the output weight of GFPNet is 0.6; the trained LFP-GFPNet is then used as the main model.

4. The deep learning-based cross-subject driving fatigue detection system according to claim 3, characterized in that, The model fine-tuning module performs the following operations: 1) Collect EEG signals from new subjects during their driving process, process the data through the data processing module, select 20% of the data as the fine-tuning training set based on the label values, and use the remaining data as the test set to carry out a cross-subject driving fatigue detection task. 2) Maintain the original learning rate for the fully connected layers of the main model to ensure that they can learn the unique high-level features of new subjects; For the rest of the main model's structure, the learning rate is reduced to one-tenth of the original. This allows for fine-tuning of the rest of the main model's structure without destroying the general features they originally learned. 3) Train the main model using the fine-tuned training set to obtain the optimal detection model for new subjects.

5. The deep learning-based cross-subject driving fatigue detection system according to claim 4, characterized in that, The cross-subject driving fatigue detection module performs the following operations: 1) Select the test set divided by the model fine-tuning module, input the optimal detection model to perform cross-subject driving fatigue detection task, and obtain cross-subject driving fatigue detection results; 2) Save the obtained optimal model parameters so that the saved parameters can be directly called when the subject performs cross-subject driving fatigue detection tasks in the future, thereby improving efficiency and accuracy.

Images