A method for detecting and analyzing driver fatigue

Abstract

This invention discloses a method for detecting and analyzing driver fatigue, belonging to the field of driver monitoring technology. The method includes real-time monitoring of the driver's multimodal physiological signals and testing and defining driver fatigue to form an original fatigue state dataset; denoising the multimodal physiological signals in the original fatigue state dataset using a denoising method based on ICEEMDAN and wavelet thresholding; establishing a driver fatigue state recognition model based on a deep belief network; optimizing the parameters of the driver fatigue state recognition model based on the Grey Wolf algorithm to establish an optimal driver fatigue state recognition model; and inputting the denoised multimodal physiological signals into the optimal driver fatigue state recognition model for driver fatigue detection and analysis. This invention solves the problems of pseudo-modalities and aliasing that may occur during signal denoising and improves the efficiency and accuracy of the model in recognizing driver fatigue.

Description

Technical Field

[0001] This invention relates to the field of driving monitoring technology, and in particular to a method for detecting and analyzing driving fatigue. Background Technology

[0002] Fatigue driving is one of the main causes of traffic accidents, accounting for 43% of major traffic accidents and seriously endangering people's safety. Therefore, accurately identifying driver fatigue is of great practical significance for ensuring road traffic safety.

[0003] Currently, methods for detecting fatigued driving typically include behavioral observation, vehicle control performance evaluation, and physiological indicator monitoring. Behavioral observation mainly uses sensors such as monitors to collect information such as the driver's facial expressions and body posture. Vehicle control performance evaluation analyzes indicators such as the driver's reaction time and braking response based on onboard sensors and computer models. Both methods have achieved corresponding technological results and breakthroughs. Physiological signals are not affected by the driver's subjective consciousness and are considered the most accurate fatigue detection method currently available. The journal article "Distinguishing Driver Fatigue State Based on Biomechanics and Cervical and Lumbar EMG" suggests that cervical and lumbar electromyography (EMG) signals can serve as key information for determining whether a tested driver is fatigued. The journal article "Study on Fatigue Driving State Based on EEG Signals" verifies the feasibility of a functional connectivity-based method for assessing mental fatigue by analyzing the driver's EEG signals. The raw physiological signals collected by sensors often contain a large amount of multi-source noise, which usually requires denoising processing. The journal article "A Denoising Method for Surface Electromyography Signals Based on EMD Autocorrelation" uses the Empirical Mode Decomposition (EMD) algorithm to denoise physiological signals, achieving certain results. The dissertation "Research on Fatigue Driving Based on ECG Signals" improved the accuracy of fatigue driving detection methods by performing wavelet denoising on collected ECG data samples. Machine learning has been widely applied in classification and recognition fields. The journal article "Research on EEG Signal Feature Selection Algorithm and Support Vector Machine Model in Fatigue Driving Recognition" uses Support Vector Machine (SVM) to identify physiological signals of fatigue driving; however, traditional machine learning algorithms rely too heavily on manual feature extraction and struggle to handle long data sets. The journal articles "Design of a Fatigue Driving Detection Headband Based on EEG Signals" and "Fatigue Driving Detection Based on Spatiotemporal EEG Features and Parallel Neural Networks" used Convolutional Neural Networks (CNNs) for training on multimodal physiological signal recognition; however, CNNs tend to handle two-dimensional image data more readily. The dissertation "Alertness Analysis Based on EEG Signals in Simulated Driving Environment" verified the superiority of Deep Belief Networks (DBNs) in physiological signal feature extraction. However, comparative analysis of the measurement effects of various physiological signals and their combined applications are rarely discussed in published papers, and no consistent conclusions have been reached. Traditional physiological signal denoising algorithms often produce problems such as pseudo-modalities and aliasing, affecting the quality of signal denoising. When performing classification and recognition, deep neural networks need to set various hyperparameters. How to scientifically and reasonably set hyperparameters to improve the efficiency and accuracy of the network is also an urgent problem to be solved. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide a method for detecting and analyzing driver fatigue. It solves the problem of pseudo-modalities that may occur during signal decomposition by using a denoising method that combines ICEEMDAN and wavelet thresholding. It establishes a deep learning model based on DBN network that can accurately determine the fatigue state of motor vehicle drivers. The gray wolf optimization algorithm is used to optimize the parameters of DBN, which can improve the training efficiency and accuracy of the model.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] A method for detecting and analyzing driver fatigue includes the following steps:

[0007] S1. Real-time monitoring of the driver's multimodal physiological signals, and testing and defining the driver's fatigue to form an original fatigue state dataset;

[0008] S2. A denoising method based on the fusion of ICEEMDAN and wavelet thresholding is used to denoise the multimodal physiological signals in the original fatigue state dataset to obtain the denoised multimodal physiological signals.

[0009] S3. Establish a driver fatigue state recognition model based on deep belief network;

[0010] S4. Optimize the parameters of the driver fatigue state recognition model based on the gray wolf algorithm and establish the optimal driver fatigue state recognition model.

[0011] S5. Input the denoised multimodal physiological signal into the optimal driver fatigue state recognition model to perform driver fatigue detection and analysis.

[0012] A further improvement of the technical solution of the present invention is that: in S1, the multimodal physiological signals include electroencephalogram (EEG) signals and electromyogram (EMG) signals. The physiological signals are monitored and recorded in real time by a biofeedback instrument during the experiment. The driver fatigue test and definition method includes a subjective questionnaire survey combined with a literature review.

[0013] A further improvement to the technical solution of the present invention is that S2 specifically includes the following steps:

[0014] S21. Use ICEEMDAN to decompose the original multimodal physiological signal into several parts, and set the ratio of the standard deviation of ICEEMDAN noise to information, the average number of iterations, and the maximum number of iterations.

[0015] S22. Based on the Spearman autocorrelation coefficient method, retain the IMF components whose correlation coefficient with the original multimodal physiological signal is greater than the threshold;

[0016] S23. Select the Symlet wavelet as the wavelet function and perform three-level wavelet decomposition to denoise the retained IMF components.

[0017] S24. Reconstruct the IMF components after wavelet denoising to obtain the denoised multimodal physiological signal.

[0018] A further improvement to the technical solution of the present invention is that S3 specifically includes the following steps:

[0019] S31. Establish a deep belief network;

[0020] Deep belief networks consist of multiple restricted Boltzmann machines, each of which contains a visible layer and a hidden layer. The visible layer receives the input data and generates the corresponding hidden layer feature representation.

[0021] v = (v1, v2, ..., v n (h1, h2, ..., h) represents the vector of the current state of the visible layer neurons. m Let be a vector representing the current state of a hidden neuron. The energy function of a restricted Boltzmann machine is defined as:

[0022]

[0023] Among them, a i b i These represent the biases of visible and hidden cells in an RBM, respectively; ω ij For node v in the visible layer i and node h in the hidden layer i The weight values ​​between; θ = {w, a, b} is the set of bias and weight value parameters;

[0024] At this point, the joint probability P(v,h;θ) of the nodes in the two layers of the deep belief network is calculated according to equation (9):

[0025] P(v,h;θ)=1 / Z(θ)exp(-E(v,h|θ)) (9)

[0026] In the formula: Z(θ) is the partition function, representing the sum of all possible values ​​that the neuron can take;

[0027] Z(θ)=∑ v,h exp(-E(v,h|θ)) (10)

[0028] The explicit conditional probability P(v|h;θ):

[0029] P(v|h;θ)=P(v,h;θ) / P(h;θ)=Π i P(v i |h;θ) (11)

[0030] The hidden conditional probability P(h|v;θ) is:

[0031] P(h|v;θ)=P(v,h;θ) / P(h;θ)=Π j P(v j |v;θ) (12)

[0032] Neurons at the same node are independent of each other, and the probability P(v) of a visible node being activated is... i =1|h;θ) is:

[0033]

[0034] The probability of a hidden node being activated is:

[0035]

[0036] Since the samples are independent of each other, we use the likelihood function to find suitable parameters; the likelihood function L(θ) is expressed as:

[0037] L(θ)=Π v L(θ|v)=Π v P(v) (15)

[0038]

[0039] In equation (16): E(v,h) represents the energy function. express The expectation under probability P(h|v) express The expectation under probability P(v|h);

[0040] All parameter update standards are as follows:

[0041]

[0042] In the formula, ε is the learning rate, and Δa i Δb i and Δω ij a i b i and ω ij The difference between the updated value and the original value;

[0043] The hidden layer of each Restricted Boltzmann Machine (RBM) is used as the input to train the next RBM. After all RBMs are trained, they are used to initialize a multi-layer neural network. After backpropagation, the weights of the neural network parameters are fine-tuned. Finally, the softmax function is used to find the label most likely to correspond to each sample to achieve the classification effect.

[0044] S32. Extract features from the denoised fatigue state dataset to form a feature dataset;

[0045] First, the time domain, frequency domain, and nonlinear features of the denoised fatigue state dataset are extracted. The time domain features include mean, peak-to-peak value, mean absolute value, variance, standard deviation, kurtosis, skewness, root mean square, waveform factor, peak factor, impulse factor, root square amplitude, and margin factor. The frequency domain features include average frequency, frequency variance, centroid frequency, root mean square frequency, and frequency standard deviation. The nonlinear features include energy entropy, information entropy, complexity, and multi-scale entropy. A total of 22 labeled feature vectors are used as the dataset.

[0046] S33. Divide the feature dataset into a training set and a test set in an 8:2 ratio, input them into a deep belief network for training, and obtain a driver fatigue state recognition model.

[0047] A further improvement to the technical solution of the present invention is that step S4 specifically includes the following steps:

[0048] S41. Randomly initialize the three parameters to be optimized: the number of DBN layers, the pre-learning rate, and the backward adjustment learning rate, with the objective function being to obtain the highest accuracy of model training.

[0049] S42. Use the Grey Wolf optimization algorithm to find better parameters from the initial values ​​according to the objective function, so that the parameters to be optimized are continuously updated toward the optimal solution;

[0050] Assuming the gray wolf population size is N, and the search space is d-dimensional, with each dimension corresponding to a parameter to be optimized, therefore d = 3, the position of the i-th gray wolf in the d-dimensional space is represented as... i = 1, 2, ..., N; The gray wolf pack gradually approaches and surrounds its prey using equation (18):

[0051]

[0052] Where t is the current iteration number, For the location of the prey, To enclose the step size, A and C are coefficients, defined as:

[0053] A = 2a·r1-a (19)

[0054] C = 2·r² (20)

[0055] Where r1 and r2 are random numbers between [0,1], and α is called the distance control parameter, which decreases linearly from 2 as the number of iterations increases:

[0056] α=2-2t / t max (twenty one)

[0057] In the formula, tmax This represents the maximum number of iterations.

[0058] Equation (22) defines the step size and direction of an individual in the wolf pack ω moving toward α, β and γ respectively. The gray wolf optimization algorithm is used to optimize the three parameters of DBN layer number, pre-learning rate and reverse adjustment learning rate. The mathematical model represented by each wolf is three-dimensional.

[0059] Equation (23) defines the final position of ω:

[0060]

[0061] In equation (22) Let be the updated position parameters for α, β, and γ, respectively. That is the final position of ω;

[0062] In each iteration of the initial parameters, the three optimal solutions obtained with the highest accuracy as the objective are saved, and the other wolves ω are forced to update their positions according to equation (23), repeating t. max Through the above steps, the wolf pack continuously approaches the target; ultimately, the position parameter value of the alpha wolf α is the optimal solution for the number of DBN layers, pre-learning rate, and back-adjusted learning rate of the established model.

[0063] The technological advancements achieved by this invention due to the adoption of the above technical solutions are as follows:

[0064] 1. This invention proposes a physiological signal denoising method combining ICEEMDAN and WT. The ICEEMDAN algorithm can divide the signal into multiple IMFs through refined multi-scale decomposition and remove redundant components. WT determines a threshold based on the variance of each sub-band, filtering out low-energy high-frequency noise while preserving a large amount of useful information. This invention combines the two methods to remove not only high-frequency noise signals but also low-frequency disturbance signals, achieving excellent denoising results.

[0065] 2. This invention manually extracts time-domain, frequency-domain, and nonlinear features from denoised multimodal physiological signals. A dataset is then built using these manually extracted features and input into a deep feature extraction database (DBN). This reduces data complexity and improves the real-time performance and robustness of the subsequent fatigue recognition model.

[0066] 3. This invention establishes the ICEEMDAN-WT-DBN fatigue driving recognition model, which utilizes...

[0067] ICEEMDAN-WT denoises the original signal and innovatively utilizes the Grey Wolf Optimization Algorithm to optimize the DBN parameters, effectively improving the DBN network performance. Compared with the DBN, WT-DBN, ICEEMDAN-WT-BP, SVM, and KNN fatigue recognition models established in this invention, the ICEEMDAN-WT-DBN model proposed in this invention achieves the highest recognition accuracy of 93.7%. Attached Figure Description

[0068] Figure 1 This is a schematic diagram of the confined Boltzmann machine in this invention;

[0069] Figure 2 This is a simplified diagram of the Deep Belief Network (DBN) structure in this invention;

[0070] Figure 3 This is a schematic diagram of the gray wolf social hierarchy in this invention;

[0071] Figure 4 This is a schematic diagram of the electrode attachment position in this invention;

[0072] Figure 5 This is a schematic diagram of the original physiological signals in this invention;

[0073] Figure 6 This is an exploded diagram of ICEEMDAN in this invention;

[0074] Figure 7 This is a schematic diagram of the frequency domain distribution of the IMF in this invention;

[0075] Figure 8 This is a noise reduction effect diagram of ICEEMDAN-WT in this invention;

[0076] Figure 9 This is a diagram showing the wavelet threshold denoising effect of this invention;

[0077] Figure 10 This is a complete training flowchart of the driver fatigue recognition model in this invention;

[0078] Figure 11 This invention demonstrates the impact of different DBN layer numbers on the training process settings, with a pre-learning rate of 0.007 and a back-adjustment learning rate of 2.0; (a) shows the training process and results of a 3-layer Boltzmann machine; (b) shows the training process and results of a 2-layer Boltzmann machine; (c) shows the training process and results of a 4-layer Boltzmann machine; and (d) shows the training process and results of a 5-layer Boltzmann machine.

[0079] Figure 12This invention illustrates the impact of different pre-learning rates of the DBN on the training process. The Boltzmann machine has 3 layers and a back-adjusted learning rate of 2.0. (a) Training process and results with a pre-learning rate of 0.007; (b) Training process and results with a pre-learning rate of 0.003; (c) Training process and results with a pre-learning rate of 0.005; (d) Training process and results with a pre-learning rate of 0.009.

[0080] Figure 13 This invention describes the impact of different backward adjustment learning rates of DBN on the training process. The Boltzmann machine has 3 layers and a pre-learning rate of 0.007. (a) Training process and results with a backward adjustment learning rate of 2; (b) Training process and results with a backward adjustment learning rate of 1.5; (c) Training process and results with a backward adjustment learning rate of 2.5; (d) Training process and results with a backward adjustment learning rate of 3.0.

[0081] Figure 14 This is a schematic diagram of the DBN training process in this invention. Detailed Implementation

[0082] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments:

[0083] A method for detecting and analyzing driver fatigue includes the following steps:

[0084] S1. Real-time monitoring of the driver's multimodal physiological signals, and testing and defining the driver's fatigue to form an original fatigue state dataset;

[0085] In S1, multimodal physiological signals include electroencephalogram (EEG) signals and electromyogram (EMG) signals. The physiological signals are monitored and recorded in real time by a biofeedback device (Biofeedback 2000) during the experiment. The driver fatigue test and definition method includes a subjective questionnaire survey combined with a literature review.

[0086] S2. A denoising method based on the fusion of ICEEMDAN and wavelet thresholding is used to denoise the multimodal physiological signals in the original fatigue state dataset.

[0087] A novel signal denoising method based on the fusion of ICEEMDAN and wavelet thresholding is designed to denoise multimodal physiological signals in the original fatigue state dataset. This method removes not only low-frequency disturbance noise but also some high-frequency interference. Based on this, a denoised fatigue state dataset is generated, addressing the pseudo-modal problem that may arise during signal decomposition. The specific steps include:

[0088] The ICEEMDAN algorithm is used to denoise signals, specifically including:

[0089] Define operator E j() is used to find the j-th IMF component of an EMD decomposition of a multimodal physiological signal;

[0090] The operator M() is defined as the local mean of a multimodal physiological signal.

[0091] Let the original signal be x[n]. Apply EMD to iteratively decompose the original sequence signal x[n] I times. First, calculate the first-order residual r1 and the first-order intrinsic mode function IMF1, and then calculate the k-order residual r1 in sequence. k and the k-th order intrinsic mode function (IMF) k :

[0092]

[0093] IMF1 = x - r1 (2)

[0094]

[0095] IMF k =r k-1 -r k (4)

[0096] In the formula, ω (i) The i-th group of Gaussian white noise added refers to the IMF component of the original noise signal added in each round of IMF calculation; ε j This refers to the coefficient multiplied when adding a noise component; this coefficient represents the ratio of the signal-to-noise ratio of the added noise to the standard deviation of that noise component; the number of white noise groups added and ε. j These are all input parameters for the ICEEMDAN algorithm;

[0097] The final residual R and the original sequence signal x are calculated as follows:

[0098] Denoising signals using wavelet thresholding specifically includes:

[0099] The wavelet function is selected to have good adaptability to non-stationary signals, good information compression, and good preservation of original information.

[0100] Based on the statistical characteristics of multimodal physiological signals, a criterion for wavelet threshold denoising is selected to determine the threshold; the expression for the wavelet threshold denoising criterion is as follows:

[0101]

[0102] In the formula, ω represents the wavelet transform coefficients after denoising. j,k λ represents the wavelet transform coefficients before denoising; sign is the sign function, and λ is the set threshold.

[0103] This invention uses a denoising method that combines ICEEMDAN and wavelet thresholding to denoise multimodal physiological signals. A novel signal denoising algorithm is designed to effectively remove low-frequency interference signals and irregular sharp noise. Specifically, the algorithm includes the following steps:

[0104] S21. Use ICEEMDAN to decompose the original multimodal physiological signal into several parts, and set the ratio of the standard deviation of ICEEMDAN noise to information, the average number of iterations, and the maximum number of iterations.

[0105] S22. Based on the Spearman autocorrelation coefficient method, retain the IMF components whose correlation coefficient with the original multimodal physiological signal is greater than the threshold;

[0106] S23. Select the Symlet wavelet as the wavelet function and perform three-level wavelet decomposition to denoise the retained IMF components.

[0107] S24. Reconstruct the IMF components after wavelet denoising to obtain the denoised multimodal physiological signal. S3. Establish a driver fatigue state recognition model based on a deep belief network, specifically including the following steps;

[0108] S31. Establish a deep belief network:

[0109] Deep belief networks consist of multiple restricted Boltzmann machines (RBMs). Each RBM contains one explicit layer and one hidden layer. The explicit layer receives the input data and generates the corresponding hidden layer feature representation. The RBM structure is as follows: Figure 1 As shown;

[0110] v = (v1, v2, ..., v n (h1, h2, ..., h) represents the vector of the current state of the visible layer neurons. m () represents the vector of the current state of a hidden layer neuron. The energy function of the RBM is defined as:

[0111]

[0112] Among them, a i b i These represent the biases of visible and hidden cells in an RBM, respectively; ω ij For node v in the visible layer i and node h in the hidden layer i The weight values ​​between; θ = {w, a, b} is the set of bias and weight value parameters;

[0113] At this point, the joint probability P(v,h;θ) of the nodes in the two layers of the deep belief network is calculated according to equation (9):

[0114] P(v,h;θ)=1 / Z(θ)exp(-E(v,h|θ)) (9)

[0115] In the formula: Z(θ) is the partition function, representing the sum of all possible values ​​that the neuron can take;

[0116] Z(θ)=∑ v,h exp(-E(v,h|θ)) (10)

[0117] The explicit conditional probability P(v|h;θ):

[0118] P(v|h;θ)=P(v,h;θ) / P(h;θ)=Π i P(v i |h;θ) (11)

[0119] The hidden conditional probability P(h|v;θ) is:

[0120] P(h|v;θ)=P(v,h;θ) / P(h;θ)=Π j P(v j |v;θ) (12)

[0121] Neurons at the same node are independent of each other, and the probability P(v) of a visible node being activated is... i =1|h;θ) is:

[0122]

[0123] The probability of a hidden node being activated is:

[0124]

[0125] Because the samples are independent of each other, we can find suitable parameters by maximizing the likelihood function; the likelihood function L(θ) can be expressed as:

[0126] L(θ)=Π v L(θ|v)=Π v P(v) (15)

[0127]

[0128] In equation (16): E(v,h) represents the energy function. express The expectation under probability P(h|v) express The expectation under probability P(v|h);

[0129] All parameter update standards are as follows:

[0130]

[0131] In the formula, ε is the learning rate, and Δa i Δb i and Δω ij a i b i and ω ij The difference between the updated value and the original value.

[0132] The hidden layers of each RBM are used as input for training the next RBM. This process continues until all RBMs are trained. They are then used to initialize a multi-layer neural network. After backpropagation, the network parameters and weights are fine-tuned. Finally, a softmax function is used to find the most likely label for each sample to achieve classification. The resulting DBN model is as follows: Figure 2 As shown.

[0133] S32. Extract features from the denoised fatigue state dataset to form a feature dataset;

[0134] To ensure higher accuracy and real-time performance of the fatigue recognition model and reduce data complexity, the time-domain, frequency-domain, and nonlinear features of the denoised fatigue state dataset are first extracted. The time-domain features include mean, peak-to-peak value, mean absolute value, variance, standard deviation, kurtosis, skewness, root mean square, waveform factor, peak factor, impulse factor, root square amplitude, and margin factor. The frequency-domain features include average frequency, frequency variance, centroid frequency, root mean square frequency, and frequency standard deviation. The nonlinear features include energy entropy, information entropy, complexity, and multi-scale entropy. A total of 22 labeled feature vectors are used as the dataset.

[0135] S33. Divide the feature dataset into a training set and a test set in an 8:2 ratio, input them into a deep belief network for training, and obtain a driver fatigue state recognition model.

[0136] S4. Optimize the parameters of the driver fatigue state recognition model based on the gray wolf algorithm and establish the optimal driver fatigue state recognition model.

[0137] The Grey Wolf Optimizer (GWO) is a swarm intelligence algorithm based on the social behavior of grey wolves in nature. It is used to solve parameter optimization problems, inspired by the population structure and predation behavior of grey wolves. In GWO, a certain number of grey wolves are first randomly initialized. These wolves are sorted according to their fitness and their positions are updated using a strategy of "searching for the target, following the target, and hunting the target" according to certain rules. The grey wolf population hierarchy pyramid is shown below. Figure 3 As shown. The principle is:

[0138] In the pyramid, the first level, α, represents the leader of the pack. The second level, β, is second only to α and acts as a bridge, feeding back the performance of other members to α. The third level, δ, follows the decisions of α and β, and so on. In the mathematical model, the alpha wolf α is the optimal solution for the current model, β is second only to α, δ is second only to β, and so on. Their guidance of other wolves towards the goal is equivalent to finding better solutions for the model parameters. The remaining wolves ω are defined as candidate solutions, updating their positions around α, β, and δ.

[0139] DBN, as a deep neural network model built using multiple layers of unsupervised learning algorithms, contains many parameters. Setting appropriate parameters can enable the network to achieve better performance.

[0140] This invention innovatively utilizes the Grey Wolf algorithm to optimize parameters such as the number of DBN layers, pre-learning rate, and reverse adjustment learning rate, specifically including the following steps:

[0141] S41. Randomly initialize the three parameters to be optimized: the number of DBN layers, the pre-learning rate, and the backward adjustment learning rate, with the objective function being to obtain the highest accuracy of model training.

[0142] S42. Use the Grey Wolf optimization algorithm to find better parameters from the initial values ​​according to the objective function, so that the parameters to be optimized are continuously updated toward the optimal solution.

[0143] Assuming the gray wolf population size is N, and the search space is d-dimensional, with each dimension corresponding to a parameter to be optimized, therefore d = 3 in this paper; the initial positions of the gray wolves are randomly generated; the position of the i-th gray wolf in the d-dimensional space is represented as... i = 1, 2, ..., N; The gray wolf pack gradually approaches and surrounds its prey using equation (18):

[0144]

[0145] Where t is the current iteration number, For the location of the prey, To enclose the step size, A and C are coefficients, defined as:

[0146] A = 2a·r1-a (19)

[0147] C = 2·r² (20)

[0148] Where r1 and r2 are random numbers between [0,1], and α is called the distance control parameter, which decreases linearly from 2 as the number of iterations increases:

[0149] α=2-2t / t max (twenty one)

[0150] In the formula, t maxThis represents the maximum number of iterations.

[0151] Equation (22) defines the step size and direction of an individual in the wolf pack ω moving towards α, β, and γ, respectively. This invention uses a gray wolf optimization algorithm to optimize the three parameters, so the mathematical model represented by each wolf is three-dimensional. Equation (23) defines the final position of ω:

[0152]

[0153] In equation (22) Let be the updated position parameters for α, β, and γ, respectively. That is the final position of ω.

[0154] The wolf pack is capable of identifying the initial location of its prey (the optimal solution). However, in an abstract search space, it is difficult to pinpoint the precise location of the prey. Therefore, in each iteration of the initial parameters (as in equation (22)), the three optimal solutions (α, β, and γ) obtained with the highest accuracy as the objective are preserved, and other wolves ω are forced to update their positions according to equation (23). Repeat t max Through the above steps, the wolf pack continuously approaches the target; ultimately, the position parameter value of the alpha wolf α is the optimal solution for the number of DBN layers, pre-learning rate, and back-adjusted learning rate of the established model.

[0155] S5. Input the denoised multimodal physiological signal into the optimal driver fatigue state recognition model to perform driver fatigue detection and analysis.

[0156] Example

[0157] This experiment required 10 male drivers to drive continuously for 120 minutes on the Shenyang-Dandong section of the Danfu Expressway, maintaining a speed below 100 km / h. Real-time monitoring of the drivers' multimodal physiological signals (EEG and EMG signals) was conducted. The physiological signal acquisition locations were as follows: Figure 4 As shown. Physiological signals were recorded by a portable wearable device—the Biofeedback 2000, consisting of two sensor modules worn on the subject's body with a sampling frequency of 200Hz. Data reception was achieved via a wireless body area network, and the collected data was directly sent to the processing system. Simultaneously, during the driving process, an assistant helped the driver complete the SOFI-25 (Swedish Occupational Fatigue Inventory-25) subjective questionnaire every 15 minutes. This questionnaire reflects the driver's five sensations: lack of energy, exhaustion, physical discomfort, lack of motivation, and drowsiness. A score of 0 indicates almost no of these five sensations, while a score of 10 indicates strong sensations. This questionnaire represents the subject's subjective judgment of their current level of fatigue based on their own feelings, thus helping to determine whether the driver has reached a state of fatigue.

[0158] Using ICEEMDAN Figure 5 The original physiological signal was decomposed, and several IMF components were obtained, such as... Figure 6 As shown.

[0159] Performing a Fast Fourier Transform on these components yields their frequency domain distributions, such as... Figure 7 As shown.

[0160] A fusion algorithm combining ICEEMDAN and wavelet thresholding was used to denoise multimodal physiological signals. First, ICEEMDAN was used to decompose the original multimodal physiological signal into several parts, with the standard deviation ratio of noise to information set to 0.16, the average number of iterations set to 50, and the maximum number of iterations set to 200. IMF components with a correlation coefficient greater than 0.3 with the original signal were retained using the Spearman autocorrelation coefficient method. Symlet wavelet was selected as the wavelet function, and three-level wavelet decomposition was performed on the retained IMF components for denoising. Finally, the wavelet-denoised IMF components were reconstructed to obtain the final denoised signal. Figure 8 As shown.

[0161] To demonstrate the superiority of the denoising method proposed in this invention, this invention compares the results of denoising using only wavelet denoising, as shown below. Figure 9 As shown, taking electromyography (EMG) signals as an example, it can be seen that the waveform denoising effect of wavelet thresholding alone is not significant, and the signal fluctuation after denoising is still relatively large. However, the waveform after ICEEMDAN-WT denoising is significantly smoother. This is because ICEEMDAN decomposition removes some low-frequency noise components through correlation coefficients, and wavelet thresholding removes some high-frequency noise from the signal. The fusion of the two achieves a very good denoising effect.

[0162] The results of the SOFI-25 subjective questionnaire are shown in Table 1: As driving time increased, the subjects' subjective questionnaire scores showed an upward trend, indicating that the subjects' fatigue levels were continuously increasing. From 0-30 minutes, the subjects' scores were close to 0, indicating that the drivers had virtually no fatigue at this time; from 90-120 minutes, the average scores were greater than 6, indicating significant fatigue at this time. Therefore, based on a combination of the SOFI-25 subjective questionnaire survey and literature review, the driving experiment period of 0-30 minutes can be defined as the "awake state" (classified as state 1), and 90-120 minutes as the "fatigued state" (classified as state 2).

[0163] Table 1. Mean scores of subjective questionnaires given by participants at different times

[0164]

[0165] To ensure higher accuracy and real-time performance of the fatigue recognition model and reduce data complexity, this invention first extracts 22 labeled feature vectors from the driver's original physiological signals in the time domain (including variance, root mean square, etc.), frequency domain (including average power spectral density, frequency center, etc.), and nonlinear features (including energy entropy, information entropy, multi-scale entropy, etc.) as a dataset. Then, the extracted labeled feature vectors are input into a deep belief network (DBN) to further extract deeper features representing the original information, thereby achieving the goal of identifying the driver's fatigue state.

[0166] The established dataset is divided into training and test sets in an 8:2 ratio and randomly fed into a Deep Belief Network (DBN) for training. The complete training flowchart for the driver fatigue recognition model is shown below. Figure 10 .

[0167] This invention utilizes the Grey Wolf Optimization Algorithm to optimize three key parameters: the number of DBN layers, the pre-learning rate, and the back-adjustment learning rate. The globally optimal solution is a three-layer Boltzmann machine with a pre-learning rate of 0.007 and a back-adjustment learning rate of 2.0. To demonstrate the effectiveness of the Grey Wolf Optimization Algorithm in optimizing these parameters, this invention establishes models with different parameters to compare the training process and results. While maintaining the same pre-learning rate and back-adjustment learning rate, DBN models consisting of 2, 4, and 5-layer RBMs are established, and the training results are compared. Figure 11 As shown; while maintaining the same number of RBM layers and the same backward adjustment learning rate, DBN models with pre-learning rates of 0.003, 0.005, and 0.009 were established respectively. The training results were compared with those of DBN models with the following results: Figure 12 As shown; while keeping the number of RBM layers and the pre-adjusted learning rate the same, DBN models with back-adjusted learning rates of 1.5, 2.5, and 3.0 were established respectively. The training results were compared with those of... Figure 13 As shown.

[0168] Based on the above results, it can be concluded that the DBN parameters obtained by the Grey Wolf optimization algorithm used in this invention give the model higher accuracy and robustness compared to other parameters. Specific DBN parameters are shown in Table 2.

[0169] Table 2 Main DBN Parameters

[0170]

[0171] State 1 corresponds to a non-fatigue state, while state 2 corresponds to a fatigue state. Therefore, the fatigue recognition model used in this invention achieves a comprehensive recognition accuracy of 96.67%, and a 100% accuracy rate for identifying fatigue states, demonstrating that the model proposed in this invention has a relatively accurate fatigue state recognition effect.

[0172] To further verify the reliability of the model proposed in this invention, physiological signals collected from nine other subjects were sequentially input into the model for training. The results are as follows: Figure 14 .

[0173] It is evident that the model performed well on the training data of 10 subjects, achieving an average accuracy of 93.2% on the test set, with an average accuracy of 98.8% for fatigue state identification. This demonstrates that the model exhibits good accuracy, universality, and robustness, accurately identifying the driver's state, particularly with a near 100% accuracy rate in identifying fatigue state.

[0174] To further verify the accuracy of the model proposed in this invention, other driver fatigue state recognition models such as ICEEMDAN-BP and ICEEMDAN-DBN were established and compared with the ICEEMDAN-WT-DBN model proposed in this invention. The results are shown in Table 3.

[0175] Table 3 Comparison of recognition accuracy of six models

[0176]

[0177] A comparison of the accuracy results of six fatigue state recognition models shows that the ICEEMDAN-WT-DBN model proposed in this invention has the highest recognition accuracy; (Undenoised raw data input)

[0178] The accuracy of training in DBN is lower than that after denoising, indicating that noise significantly affects training accuracy. WT-DBN, which only uses wavelet thresholding to denoise multimodal physiological signals, has a slightly lower accuracy than the ICEEMDAN-WT-DBN model, suggesting that the WT-DBN model is less effective at eliminating noise.

[0179] The ICEEMDAN-WT-DBN model is superior; however, the ICEEMDAN-WT-BP model has a classification accuracy of only 82.2%, and its convergence speed is slow, making it prone to getting trapped in local optima. Traditional machine learning classification algorithms are weak at extracting subtle features from data, with SVM and KNN achieving accuracy below 80%.

[0180] Previous assessments of driver fatigue primarily relied on observations of vehicle conditions and driver facial expressions, methods with inherent limitations. Furthermore, comparative analyses of the effectiveness of multiple physiological signal measurements, and their combined applications, have been limited and inconsistently concluded in published research. Building upon previous studies, this research utilizes multimodal physiological signal analysis to determine driver fatigue.

[0181] Previous research on signal denoising has largely focused on wavelet thresholding, but wavelet thresholding struggles to remove some low-frequency noise. This invention proposes a combination of the ICEEMDAN algorithm and wavelet thresholding, which effectively reduces noise in signals and improves signal reconstruction quality. The ICEEMDAN algorithm uses refined multi-scale decomposition to divide the signal into many relatively smooth and coarse parts, removing redundant components. Wavelet thresholding, on the other hand, determines a threshold based on the variance of each sub-band, filtering out low-energy high-frequency noise while retaining useful information. Experimental results show that this method can not only filter sharp noise signals but also remove disturbances caused by the subjects' own activities. Previous classification of labeled datasets has primarily relied on traditional machine learning algorithms such as Support Vector Machines (SVM) and K-Nearest Neighbors (KNN). These algorithms excessively depend on manual feature extraction and are ineffective for multimodal physiological signals with unclear initial features. This invention extracts multi-domain features from electroencephalogram (EEG) signals and denoised electromyography (EMG) signals, and uses this dataset as input to a deep neural network (DBN). By fully utilizing the DBN's ability to extract deep features, an ICEEMDAN-WT-DBN fatigue driving recognition model is constructed, reducing the computational complexity of the algorithm and improving the efficiency of model training and prediction. To find suitable model parameters, this invention employs the Grey Wolf optimization algorithm to optimize key parameters of the DBN, avoiding the need for manually setting deep neural network parameters and further improving the accuracy and robustness of model training.

[0182] In summary, this invention first collects EEG and EMG signals from drivers during actual driving, and proposes a denoising method based on ICEEMDAN combined with wavelet thresholding to solve problems such as pseudo-modalities that may occur during signal decomposition. Finally, a deep learning model that can accurately determine the fatigue state of motor vehicle drivers is established based on a DBN network, and the gray wolf optimization algorithm is innovatively used to optimize the parameters of the DBN, thereby improving the training efficiency and accuracy of the fatigue driving recognition model.

Claims

1. A method for detecting and analyzing driver fatigue, characterized in that: Includes the following steps: S1. Real-time monitoring of the driver's multimodal physiological signals, and testing and defining the driver's fatigue to form an original fatigue state dataset; S2. A denoising method based on the fusion of ICEEMDAN and wavelet thresholding is used to denoise the multimodal physiological signals in the original fatigue state dataset to obtain the denoised multimodal physiological signals. S3. Establish a driver fatigue state recognition model based on deep belief network; S4. Optimize the parameters of the driver fatigue state recognition model based on the gray wolf algorithm and establish the optimal driver fatigue state recognition model. S5. Input the denoised multimodal physiological signal into the optimal driver fatigue state recognition model to perform driver fatigue detection and analysis.

2. The driving fatigue detection and analysis method according to claim 1, characterized in that: In S1, multimodal physiological signals include electroencephalogram (EEG) signals and electromyogram (EMG) signals. These physiological signals are monitored and recorded in real time by a biofeedback device during the experiment. The driver fatigue test and definition method includes a combination of subjective questionnaire surveys and literature review.

3. The driving fatigue detection and analysis method according to claim 1, characterized in that: S2 specifically includes the following steps: S21. Use ICEEMDAN to decompose the original multimodal physiological signal into several parts, and set the ratio of the standard deviation of ICEEMDAN noise to information, the average number of iterations, and the maximum number of iterations. S22. Based on the Spearman autocorrelation coefficient method, retain the IMF components whose correlation coefficient with the original multimodal physiological signal is greater than the threshold; S23. Select the Symlet wavelet as the wavelet function and perform three-level wavelet decomposition to denoise the retained IMF components. S24. Reconstruct the IMF components after wavelet denoising to obtain the denoised multimodal physiological signal.

4. The driving fatigue detection and analysis method according to claim 1, characterized in that: S3 specifically includes the following steps: S31. Establish a deep belief network; A deep belief network consists of multiple restricted Boltzmann machines. Each restricted Boltzmann machine contains a visible layer and a hidden layer. The visible layer receives the input data and generates the corresponding hidden layer feature representation. v = (v1, v2, ..., v n (h1, h2, ..., h) represents the vector of the current state of the visible layer neurons. m Let be a vector representing the current state of a hidden neuron. The energy function of a restricted Boltzmann machine is defined as: Among them, a i b i These represent the biases of visible and hidden cells in an RBM, respectively; ω ij For node vi in ​​the visible layer and node h in the hidden layer i The weight values ​​between; θ = {w, a, b} is the set of bias and weight value parameters; At this point, the joint probability P(v,h;θ) of the nodes in the two layers of the deep belief network is calculated according to equation (9): P(v,h;θ)=1 / Z(θ)exp(-E(v,h|θ)) (9) In the formula: Z(θ) is the partition function, representing the sum of all possible values ​​that the neuron can take; Z(θ)=∑ v,h exp(-E(v,h|θ)) (10) The explicit conditional probability P(v|h;θ): P(v|h;θ)=P(v,h;θ) / P(h;θ)=Π i P(v i |h;θ) (11) The hidden conditional probability P(h|v;θ) is: P(h|v;θ)=P(v,h;θ) / P(h;θ)=Π j P(v j |v;θ) (12) Neurons at the same node are independent of each other, and the probability P(v) of a visible node being activated is... i =1|h;θ) is: The probability of a hidden node being activated is: Since the samples are independent of each other, we use the likelihood function to find suitable parameters; the likelihood function L(θ) is expressed as: L(θ)=Π v L(θ|v)=Π v P(v) (15) In equation (16): E(v,h) represents the energy function. express The expectation under probability P(h|v) express The expectation under probability P(v|h); All parameter update standards are as follows: In the formula, ε is the learning rate, and Δa i Δb i and Δω ij a i b i and ω ij The difference between the updated value and the original value; The hidden layer of each Restricted Boltzmann Machine (RBM) is used as the input to train the next RBM. After all RBMs are trained, they are used to initialize a multi-layer neural network. After backpropagation, the weights of the neural network parameters are fine-tuned. Finally, the softmax function is used to find the label most likely to correspond to each sample to achieve the classification effect. S32. Extract features from the denoised fatigue state dataset to form a feature dataset; First, the time domain, frequency domain, and nonlinear features of the denoised fatigue state dataset are extracted. The time domain features include mean, peak-to-peak value, mean absolute value, variance, standard deviation, kurtosis, skewness, root mean square, waveform factor, peak factor, impulse factor, root square amplitude, and margin factor. The frequency domain features include average frequency, frequency variance, centroid frequency, root mean square frequency, and frequency standard deviation. The nonlinear features include energy entropy, information entropy, complexity, and multi-scale entropy. A total of 22 labeled feature vectors are used as the dataset. S33. Divide the feature dataset into a training set and a test set in an 8:2 ratio, input them into a deep belief network for training, and obtain a driver fatigue state recognition model.

5. The driving fatigue detection and analysis method according to claim 1, characterized in that: S4 specifically includes the following steps: S41. Randomly initialize the three parameters to be optimized: the number of DBN layers, the pre-learning rate, and the backward adjustment learning rate, with the objective function being to obtain the highest accuracy of model training. S42. Use the Grey Wolf optimization algorithm to find better parameters from the initial values ​​according to the objective function, so that the parameters to be optimized are continuously updated toward the optimal solution; Assume the gray wolf population size is N, the search space is d-dimensional, and each dimension corresponds to a parameter to be optimized, therefore d = 3. The initial positions of the gray wolves are randomly generated, and the position of the i-th gray wolf in the d-dimensional space is represented as... i = 1, 2, ..., N; The gray wolf pack gradually approaches and surrounds its prey using equation (18): Where t is the current iteration number, For the location of the prey, To enclose the step size, A and C are coefficients, defined as: A = 2a·r1-a (19) C＝2·r2 (20) Where r1 and r2 are random numbers between [0,1], and α is called the distance control parameter, which decreases linearly from 2 as the number of iterations increases: α=2-2t / t max (21) In the formula, t max This represents the maximum number of iterations. Equation (22) defines the step size and direction of an individual in the wolf pack ω moving toward α, β and γ respectively. The gray wolf optimization algorithm is used to optimize the three parameters of DBN layer number, pre-learning rate and reverse adjustment learning rate. The mathematical model represented by each wolf is three-dimensional. Equation (23) defines the final position of ω: In equation (22) Let be the updated position parameters for α, β, and γ, respectively. That is the final position of ω; In each iteration of the initial parameters, the three optimal solutions obtained with the highest accuracy as the objective are saved, and the other wolves ω are forced to update their positions according to equation (23), repeating t. max Through the above steps, the wolf pack continuously approaches the target; ultimately, the position parameter value of the alpha wolf α is the optimal solution for the number of DBN layers, pre-learning rate, and back-adjusted learning rate of the established model.

Images