Driver fatigue identification method
By combining ECG and time point methods, using electrocardiogram features and neural networks to judge driving status, the problem of inaccurate driver fatigue recognition in the prior art is solved, and more accurate and real-time fatigue monitoring is achieved.
Patent Information
- Application Number
- CN202411131685.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-08-19
AI Technical Summary
The existing driver fatigue recognition methods are difficult to detect synchronously due to slow image recognition speed and fast ECG monitoring speed, and incomplete image recognition, which can easily lead to misjudgment and inaccurate fatigue recognition.
Combining ECG and time points, by calculating the R-wave time interval, mean square root of difference and low frequency/high frequency ratio of the electrocardiogram, the frequency domain characteristics are extracted using Fourier transform, and the optimal weight matrix is iteratively calculated through neural networks to judge driving status. Avoid facial recognition lag.
Real-time capture of psychological state changes is achieved, the accuracy of driver fatigue recognition is improved, the lag and uncertainty of image recognition is avoided, and more comprehensive fatigue monitoring is provided.
Smart Images

Figure CN119058712A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of Internet of Things, and in particular relates to a method for identifying driver fatigue. Background Art
[0002] The existing method for identifying driver fatigue mainly combines ECG and facial recognition. The disadvantage of this method is that the image recognition is slow and there will be delays. The ECG monitoring speed is fast, so image recognition cannot be detected synchronously with ECG. Moreover, given the incompleteness of image recognition, sometimes only the driver's ECG can be monitored alone, and the driver's facial image cannot be monitored. For example, when the driver's head is not facing the camera, this situation will result in only a single ECG signal. The lack of a facial image can easily lead to misjudgment of the driver's fatigue, resulting in inaccurate recognition of the driver's fatigue. Summary of the invention
[0003] In order to achieve the above-mentioned purpose, a method for identifying driver fatigue of the present invention can combine ECG and time point to avoid the situation where facial recognition cannot be performed or facial recognition is delayed.
[0004] A method for identifying driver fatigue of the present invention comprises the following steps:
[0005] S1: taking time period T as an interval, continuously monitoring the electrocardiogram of a driver during driving, obtaining the electrocardiogram of several intervals of the driver, and preprocessing the electrocardiogram of several intervals; specifically comprising the following steps:
[0006] S1.1: Calculate the time intervals between adjacent R waves of an electrocardiogram in a certain interval, and combine them with the start time of the electrocardiogram in this interval to form the result sequence of the electrocardiogram in this interval, which is:
[0007] [tm,RR (1,tm) ,RR (2,tm) ,……,RR (Q,tm) , ..., RR (i,tm) ]; where tm is the start time of the mth interval ECG; RR (Q,tm) represents the Qth time interval in the ECG within the mth interval;
[0008] S1.2: Calculate the root mean square error of the electrocardiogram in this interval. The formula is:
[0009]
[0010] S1.3: Perform Fourier transform FFT on the result sequence in S1.1 to extract the frequency domain characteristics of HRV. The formula is:
[0011] Wherein, LFrange and HFrange represent the frequency range of low frequency 0.04-0.15HZ and high frequency 0.15-0.4HZ in ECG respectively; P(f1) and P(f2) represent the power spectral density of low frequency and high frequency in ECG respectively;
[0012] S2: Set the weight matrix of all drivers in different driving periods;
[0013] W 1 =[α 1 ,α 2 ,……,α p ], where α p is the fatigue coefficient;
[0014] S3: Input the driving start time, root mean square error and LF / HF in the result sequence into the neural network, combine the weight matrix of the driver in different driving periods, and iteratively calculate the optimal weight matrix;
[0015] S4: Substitute the optimal weight matrix into S3 to calculate the current driving state probability and judge the current driving state.
[0016] Furthermore, step S3 includes the following steps:
[0017] S3.1: Obtain a large number of samples, each sample including the time point, root mean square error and LF / HF of the result sequence of different drivers; divide the large number of samples into training set, validation set and test set in proportion;
[0018] S3.2: Set the number of neurons in the input layer and the hidden layer; and calculate the output of the hidden layer, the formula is:
[0019] h d =f(W k ·x k +b d ) (1)
[0020] Where f is the activation function Relu, Wk is the weight matrix of the kth feature, k∈[1,3], k is an integer; x k is the input feature vector; b d is the bias term of the randomly generated dth hidden layer neuron, d∈[1,32], d is an integer; h d The output of the dth hidden layer neuron;
[0021] S3.3: Calculate the probability of drowsy driving at the output layer;
[0022] β z =σ(W h ·h d +b h )(2)
[0023] σ is the Sigmoid function; W h is the weight from the hidden layer to the output layer, which is a known quantity; b h is the bias of the output layer;
[0024] S3.4: Calculate the loss function;
[0025]
[0026] Among them, y z Indicates the true label of the zth sample, which is 0 or 1; β z represents the probability of the zth sample prediction; S is the total number of samples;
[0027] S3.5: Iterate weight matrix;
[0028]
[0029] W k,new is the updated kth weight matrix; W k is the original weight before updating; η is the learning rate; is the gradient of the kth weight matrix;
[0030] S3.6: Set the number of iterations, loop S3.2-S3.5, and obtain the optimal weight matrix, denoted as W k,best .
[0031] Further, in step S4, W in S3 is k,best Substitute into equation (1) (2) to solve the driving state probability β of the current sample 当前 ; Use the evaluation formula to evaluate the driving state, the evaluation formula is:
[0032]
[0033] Among them, θ=0.5.
[0034] Furthermore, the number of iterations in S3.6 is set to 100.
[0035] Further, in step S2, at W 1 Set the time period when you are prone to fatigue. The formula is:
[0036]
[0037] Beneficial effects:
[0038] 1. In ECG feature extraction, the root mean square deviation (RMSSD) and low frequency / high frequency ratio (LF / HF) are selected as model input features because they can effectively reflect the activity state of the autonomic nervous system. RMSSD measures heart rate variability and reflects the sensitivity of the heart to changes in the autonomic nervous system; the LF / HF ratio reveals the balance of sympathetic and parasympathetic nerves, which is important for assessing fatigue and stress. These two features are simple and intuitive, easy to calculate, and widely recognized in heart rate variability analysis.
[0039] 2. Combining ECG and time, it can capture changes in psychological state in real time. Compared with the lag and uncertainty of image recognition, moment analysis can perfectly avoid the above situation. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 This is a flow chart of the whole method. DETAILED DESCRIPTION
[0041] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0042] A method for identifying driver fatigue comprises the following steps:
[0043] S1: Taking time period T as an interval, continuously monitoring the electrocardiogram of a driver during driving, obtaining the electrocardiogram of several intervals of the driver, and preprocessing the electrocardiogram of several intervals.
[0044] The specific steps include:
[0045] S1.1: Starting from the departure time of a certain driver as the starting time, and continuously recording the driver's electrocardiogram for a time period T = 1min until the driver's departure is completed, obtaining the electrocardiogram within each time period T and the time point corresponding to the electrocardiogram within the time period T.
[0046] Specifically, assume that the driver records N intervals of electrocardiograms (i.e., generates N intervals of ECG) during one driving process, and the start time of each interval of the electrocardiogram is t1, t2, ..., tN. t1, t2, ..., tN are real time points, i.e., the current Beijing time point. Because it is a continuous monitoring, the time difference between two adjacent time points is 1 minute.
[0047] S1.2: Use a Butterworth bandpass filter to filter the ECG of each interval and convert it into a digital signal, and input the digital signal ECG into the Pan-Tompkins algorithm;
[0048] S1.3: The Pan-Tompkins algorithm detects the time interval between adjacent R waves in each interval ECG, recorded as RR; and the time points corresponding to the electrocardiogram in the interval and the time interval between adjacent R waves in the electrocardiogram are combined into a result sequence.
[0049] Specifically, assume that the start time corresponding to the mth interval ECG is tm, m∈[1,N];
[0050] Assuming that there are i+1 R waves in the mth interval ECG, there are i time intervals, and the resulting sequence is:
[0051] [tm,RR (1,tm) ,RR (2,tm) ,……,RR (Q,tm) , ..., RR (i,tm) ]; tm is the start time of the mth interval ECG; RR (Q,tm) represents the Qth time interval in the ECG within the mth interval.
[0052] S1.4: Perform time domain feature analysis on the electrocardiogram in each interval and calculate the root mean square error. The formula is:
[0053]
[0054] S1.5: Perform Fourier transform FFT on the result sequence in S1.3 to extract the frequency domain features of HRV (heart rate variability). The formula is:
[0055]
[0056] Among them, LFrange and HFrange represent the frequency range of low frequency 0.04-0.15HZ and high frequency 0.15-0.4HZ in ECG respectively; P(f1) and P(f2) represent the power spectral density of low frequency and high frequency in ECG respectively.
[0057] S2: Set the weight matrix of all drivers in different driving periods.
[0058] Based on the driving time of buses from 6:00 to 21:00, the fatigue coefficient α of all drivers in each driving time period is randomly generated in half-hour intervals. p, p is the number of intervals from 6:00 to 21:00. In this embodiment, the driving time from 6:00 to 21:00 is 15 hours, with half an hour as the interval, there are 30 intervals in total, that is, p∈[1,30], p is an integer; where α 1 = 6:00 to 6:30; α 1 =6:30 to 7:00 time period; and so on.
[0059] Then, the weight matrix of the driver in different driving periods is:
[0060] W 1 =[α 1 ,α 2 ,……,α p ]
[0061] Among them, α 1 , α 2 , α 6 and α 30 Because it is a time period that is prone to fatigue, the random number 0-1 is multiplied by 2 to match the actual situation; the other fatigue coefficients are random numbers between 0 and 1. That is:
[0062]
[0063] S3: Neural network training and recognition.
[0064] The following steps are involved:
[0065] S3.1: Obtain the time points, root mean square error and LF / HF of the result sequences of a large number of different drivers in S1, hereinafter referred to as samples; divide the training set, validation set and test set into 6:2:2.
[0066] S3.2: Set the number of neurons in the input layer.
[0067] The number of neurons in the input layer is set to 3, which are used to receive the driving start time, root mean square error, and LF / HF of the result sequence in the training set.
[0068] S3.3: Set the number of neurons in the hidden layer.
[0069] Set the number of neurons in the hidden layer to 32 and calculate the output of the hidden layer. The formula is:
[0070] h d =f(W k ·x k +b d ) (1)
[0071] Among them, f is the activation function Relu, W k is the weight matrix of the kth feature, k∈[1,3], k is an integer, where W1 is the weight of a driver in different driving periods;
[0072] W 2 is the weight matrix of the heart rate signal root mean square error to the hidden layer, W 2 The initial matrix of W is generated randomly and then updated iteratively; 3 is the weight matrix of LF / HF ratio for hidden layer, W 3 The initial matrix is generated randomly and then updated iteratively.
[0073] x k is the input feature vector, where x 1 is the starting time, x 2 is the root mean square error, x 3 is the LF / HF ratio;
[0074] b d is the bias term of the randomly generated dth hidden layer neuron, d∈[1,32], d is an integer; h d The output of the dth hidden layer neuron.
[0075] S3.4: Calculate the drowsy driving probability at the output layer.
[0076] β z =σ(W h ·h d +b h ) (2)
[0077] σ is the Sigmoid function; W h is the weight from the hidden layer to the output layer, which is a known quantity; b h is the bias of the output layer.
[0078] S3.5: Calculate the loss function and gradient.
[0079]
[0080] Among them, y z Indicates the true label of the zth sample, which is 0 or 1 (0 means non-fatigue, 1 means fatigue). z represents the probability of the zth sample prediction; S is the total number of samples.
[0081] S3.6: Iterate the weight matrix.
[0082]
[0083] W k,new is the kth weight matrix after update; W k is the original weight before updating; η is the learning rate; is the gradient of the kth weight matrix.
[0084] S3.7: Set the number of iterations and loop S3.3-S3.6.
[0085] In this embodiment, the number of iterations is 100. The optimal weight matrix after iteration is obtained, which is recorded as W k,best .
[0086] S4: Determine whether the driver is tired.
[0087] Use the optimal weight matrix W after iteration in S3.7 k,best , substitute into equation (1) (2) to solve the driving state probability β of the current sample 当前 ,
[0088]
[0089] Among them, θ=0.5.
[0090] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.
Claims
1. A method for identifying driver fatigue, characterized in that: The following steps are involved: S1: taking time period T as an interval, continuously monitoring the electrocardiogram of a driver during driving, obtaining the electrocardiogram of the driver in several intervals, and preprocessing the electrocardiograms in several intervals; specifically comprising the following steps: S1.1: Calculate the time interval between adjacent R waves of an interval electrocardiogram, and combine it with the start time of the interval electrocardiogram to form the result sequence of the interval electrocardiogram: [tm,RR (1,tm) ,RR (2,tm) ,……,RR (Q,tm) , ..., RR (i,tm) ]; where tm is the start time of the mth interval ECG; RR (Q,tm) represents the Qth time interval in the ECG within the mth interval; S1.2: Calculate the root mean square error of the electrocardiogram in this interval. The formula is: S1.3: Perform Fourier transform FFT on the result sequence in S1.1 to extract the frequency domain characteristics of HRV. The formula is: Wherein, LFrange and HFrange represent the frequency range of low frequency 0.04-0.15HZ and high frequency 0.15-0.4HZ in ECG respectively; P(f1) and P(f2) represent the power spectral density of low frequency and high frequency in ECG respectively; S2: Set the weight matrix of all drivers in different driving periods; W1=[α1,α2,……,α p ], where α p is the fatigue coefficient; S3: Input the driving start time, root mean square error and LF / HF in the result sequence into the neural network, combine the weight matrix of the driver in different driving periods, and iteratively calculate the optimal weight matrix; S4: Substitute the optimal weight matrix into S3 to calculate the current driving state probability and judge the current driving state.
2. A method for identifying driver fatigue according to claim 1, characterized in that: Step S3 includes the following steps: S3.1: Obtain a large number of samples, each sample including the time point, root mean square error and LF / HF of the result sequence of different drivers; divide the large number of samples into training set, validation set and test set in proportion; S3.2: Set the number of neurons in the input layer and the hidden layer; and calculate the output of the hidden layer, the formula is: h d =f(W k ·x k +b d ) (1) Among them, f is the activation function Relu, W k is the weight matrix of the kth feature, k∈[1,3], k is an integer; x k is the input feature vector; b d is the bias term of the randomly generated dth hidden layer neuron, d∈[1,32], d is an integer; h d The output of the dth hidden layer neuron; S3.3: Calculate the probability of drowsy driving at the output layer; b z =σ(W h ·h d +b h )(2) σ is the Sigmoid function; W h is the weight from the hidden layer to the output layer, which is a known quantity; b h is the bias of the output layer; S3.4: Calculate the loss function; Among them, y z Indicates the true label of the zth sample, which is 0 or 1; β z represents the probability of the zth sample prediction; S is the total number of samples; S3.5: Iterate weight matrix; W k,new is the updated kth weight matrix; W k is the original weight before updating; η is the learning rate; is the gradient of the kth weight matrix; S3.6: Set the number of iterations, loop S3.2-S3.5, and obtain the optimal weight matrix, denoted as W k,best .
3. A method for identifying driver fatigue according to claim 1, characterized in that: In step S4, W in S3 is k,best Substitute into equation (1) (2) to solve the driving state probability β of the current sample 当前 ; Use the evaluation formula to evaluate the driving state, the evaluation formula is: Among them, θ=0.
5.
4. A method for identifying driver fatigue according to claim 1, characterized in that: In step S3.6, the number of iterations in S3.6 is set to 100.
5. A method for identifying driver fatigue according to claim 1, characterized in that: In step S2, the time period when one is prone to fatigue is set in W1, and the formula is:
Citation Information
Patent Citations
Driving fatigue judgment method based on driver electrocardiogram RR interval
CN107149480A
Fatigue recognition early warning method and system based on multi-dimensional electrocardiosignal analysis
CN118415640A
State estimation apparatus, information processor, and state estimation system
JP2018202130A
System of prevention using pilot's drowsiness detection and cumulative analysis
KR102108515B1
Drowsiness estimation device and drowsiness estimation program
WO2016104498A1
