A driver fatigue recognition method

Abstract

The application discloses a driver fatigue recognition method and belongs to the technical field of Internet of Things.The method comprises the following steps: S1, monitoring electrocardiograms of a driver during driving in a time period T; S2, calculating time intervals of adjacent R waves in the electrocardiograms in the time period, combining a starting moment of the electrocardiograms in the time period, and forming a result sequence of the electrocardiograms in the time period; S3, calculating a root mean square deviation of the electrocardiograms in the time period; S4, setting a weight matrix of all drivers in different driving time periods; S5, iteratively calculating an optimal weight matrix; and S6, calculating a current driving state probability.The method avoids a traditional face image recognition mode and avoids a lag caused by face recognition.

Description

Technical Field

[0001] This invention belongs to the field of Internet of Things (IoT) technology, specifically relating to a method for driver fatigue recognition. Background Technology

[0002] Existing methods for identifying driver fatigue mainly combine ECG and facial recognition. The drawbacks of this approach are that image recognition is slow and prone to delays, while ECG monitoring is fast. Therefore, image recognition cannot detect ECG simultaneously. Furthermore, due to the incompleteness of image recognition—sometimes only the driver's ECG can be detected, without capturing the driver's face (e.g., when the driver's head is not directly facing the camera)—only a single ECG signal is detected, and the lack of a facial image can easily lead to misjudgments of driver fatigue, resulting in inaccurate driver fatigue detection. Summary of the Invention

[0003] To achieve the above objectives, the present invention provides a driver fatigue recognition method that combines ECG and time points to avoid situations where facial recognition is unavailable or delayed.

[0004] The present invention provides a method for identifying driver fatigue, comprising the following steps:

[0005] S1: Continuously monitor the electrocardiogram (ECG) of a driver during the driving process, using time interval T as the interval, to obtain several intervals of ECG data for the driver, and then preprocess these interval ECG data; specifically, this includes the following steps:

[0006] S1.1: Calculate the time interval between adjacent R waves in a certain interval of the electrocardiogram (ECG), and combine it with the start time of the ECG in that interval to form the result sequence of the ECG in that interval, as follows:

[0007] [tm,RR (1,tm) ,RR (2,tm) ,……,RR (Q,tm) , ..., RR (i,tm) ]; where tm is the start time of the m-th interval ECG; RR (Q,tm) This represents the Q-th time interval in the ECG within the m-th interval;

[0008] S1.2: Calculate the root mean square error of the electrocardiogram within this interval using the following formula:

[0009]

[0010] S1.3: Perform a Fourier transform (FFT) on the result sequence from S1.1 to extract the frequency domain features of the HRV. The formula is as follows:

[0011] Wherein, LFrange and HFrange represent the frequency ranges of the ECG in the low-frequency range of 0.04-0.15 Hz and the high-frequency range of 0.15-0.4 Hz, respectively; P(f1) and P(f2) represent the power spectral density of the ECG in the low-frequency and high-frequency ranges, respectively.

[0012] S2: Set the weight matrix for all drivers at different driving times;

[0013] W1=[α1,α2,……,α p ], where α p Fatigue coefficient;

[0014] S3: Input the driving start time, root mean square error, and LF / HF from the result sequence into the neural network, and combine them with the driver's weight matrix at different driving time periods to iteratively calculate the optimal weight matrix;

[0015] S4: Substitute the optimal weight matrix into S3 to calculate the probability of the current driving state and evaluate the current driving state.

[0016] Furthermore, step S3 includes the following steps:

[0017] S3.1: Obtain a large number of samples, each of which includes the time point, root mean square error, and LF / HF of different driver outcome sequences; divide the large number of samples into training set, validation set, and test set according to the proportions;

[0018] S3.2: Set the number of neurons in the input layer and hidden layer; and calculate the output of the hidden layer using the following formula:

[0019] h d =f(W k ·x k +b d (1)

[0020] Where f is the ReLU activation function, Wk is the weight matrix of the k-th feature, k∈[1,3], and k is an integer; x k b is the input feature vector; d h represents the bias term of the randomly generated d-th hidden layer neuron, where d ∈ [1, 32] and d is an integer; d The output of the d-th hidden layer neuron;

[0021] S3.3: Calculate the probability of fatigued driving in the output layer;

[0022] β z =σ(W h ·h d +b h (2)

[0023] σ is the Sigmoid function; W h These are the weights from the hidden layer to the output layer, and they are known quantities; b h It is the bias of the output layer;

[0024] S3.4: Calculate the loss function;

[0025]

[0026] Among them, y z β represents the true label of the z-th sample, which is either 0 or 1; z Let S represent the probability predicted for the z-th sample; S is the total number of samples.

[0027] S3.5: Iterative weight matrix;

[0028]

[0029] W k,new W is the updated k-th weight matrix; k The original weights are the weights before the update; η is the learning rate. The gradient of the k-th weight matrix;

[0030] S3.6: Set the number of iterations, loop through S3.2-S3.5, and obtain the optimal weight matrix, denoted as W. k,best .

[0031] Furthermore, in step S4, W in S3 is... k,best Substituting into equations (1) and (2), we can solve for the driving state probability β of the current sample. 当前 The driving condition is evaluated using an evaluation formula, which is as follows:

[0032]

[0033] Where θ = 0.5.

[0034] Furthermore, the number of iterations in S3.6 is set to 100.

[0035] Furthermore, in step S2, a time period of easy fatigue is set in W1, using the following formula:

[0036]

[0037] Beneficial effects:

[0038] 1. In ECG feature extraction, the root mean square difference (RMSSD) and low-frequency / high-frequency ratio (LF / HF) were chosen as model input features because they effectively reflect the activity state of the autonomic nervous system. RMSSD measures heart rate variability, reflecting the heart's sensitivity to changes in the autonomic nervous system; the LF / HF ratio reveals the balance between the sympathetic and parasympathetic nervous systems, which is important for assessing fatigue and stress levels. These two features are simple, intuitive, easy to calculate, and widely accepted in heart rate variability analysis.

[0039] 2. By combining ECG and time, it can capture changes in psychological state in real time. Compared with the lag and uncertainty of image recognition, time analysis can perfectly avoid the above situations. Attached Figure Description

[0040] Figure 1 This is the overall flowchart of this method. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] A method for identifying driver fatigue includes the following steps:

[0043] S1: Using time interval T as the interval, continuously monitor the electrocardiogram (ECG) of a driver during the driving process, obtain the ECG of the driver in several intervals, and preprocess the ECG of several intervals.

[0044] Specifically, the following steps are included:

[0045] S1.1: Starting from the departure time of a driver's current train trip, the driver's electrocardiogram (ECG) is continuously recorded over a time interval T = 1 minute until the end of the current train trip. The ECG data for each time interval T and the corresponding time point of the ECG data for that time interval T are obtained.

[0046] Specifically, assume that during a single driving session, the driver records N intervals of electrocardiogram (ECG) data (i.e., generates N intervals of ECG). The start times for each interval are t1, t2, ..., tN, respectively. t1, t2, ..., tN are the actual time points, i.e., the current Beijing time. Because the monitoring is continuous, the time difference between two adjacent time points is 1 minute.

[0047] S1.2: Use a Butterworth bandpass filter to filter the ECG of each interval and convert it into a digital signal. Then input the ECG of the digital signal into the Pan-Tompkins algorithm.

[0048] S1.3: The Pan-Tompkins algorithm detects the time interval between adjacent R waves in each ECG interval, denoted as RR; and combines the time point corresponding to the ECG interval and the time interval between adjacent R waves in the ECG to form the result sequence.

[0049] Specifically, assume that the start time of the ECG corresponding to the m-th interval is tm, where m∈[1,N];

[0050] Assuming there are i+1 R waves in the m-th ECG interval, and i time intervals, the resulting sequence is as follows:

[0051] [tm,RR (1,tm) ,RR (2,tm) ,……,RR (Q,tm) , ..., RR (i,tm) ]; where tm is the start time of the m-th interval ECG; RR (Q,tm) This represents the Q-th time interval in the ECG within the m-th interval.

[0052] S1.4: Perform time-domain characteristic analysis on the electrocardiogram for each interval and calculate the root mean square error. The formula is:

[0053]

[0054] S1.5: Perform a Fourier Transform (FFT) on the result sequence from S1.3 to extract the frequency domain features of HRV (Heart Rate Variability). The formula is:

[0055]

[0056] Wherein, LFrange and HFrange represent the frequency ranges of the ECG in the low-frequency range of 0.04-0.15 Hz and the high-frequency range of 0.15-0.4 Hz, respectively; P(f1) and P(f2) represent the power spectral density of the ECG in the low-frequency and high-frequency ranges, respectively.

[0057] S2: Set the weight matrix for all drivers at different driving times.

[0058] Since the bus driving hours are from 6:00 to 21:00, the fatigue coefficient α of all drivers is randomly generated for each driving time period in half-hour intervals. pp is the number of intervals from 6 o'clock to 21 o'clock. In this embodiment, the driving time from 6 o'clock to 21 o'clock is 15 hours, with half-hour intervals, and there are a total of 30 intervals, i.e., p∈[1,30], where p is an integer; where α1=6 o'clock to 6:30 o'clock time period; α1=6:30 to 7 o'clock time period; and so on.

[0059] Therefore, the weight matrix for drivers at different driving times is:

[0060] W1=[α1,α2,……,α p ]

[0061] Among them, α1, α2, α6 and α 30 Because this is a period of high fatigue, the random number 0-1 is multiplied by 2 to reflect the actual situation; other fatigue coefficients are random numbers between 0 and 1. That is:

[0062]

[0063] S3: Neural Network Training and Recognition.

[0064] Includes the following steps:

[0065] S3.1: Obtain the time points, root mean square error, and LF / HF of the result sequences of a large number of different drivers in S1, hereinafter referred to as samples; divide the training set, validation set, and test set in a 6:2:2 ratio.

[0066] S3.2: Set the number of neurons in the input layer.

[0067] The number of neurons in the input layer is set to 3, which are used to receive the driving start time, root mean square error, and LF / HF of the result sequence in the training set.

[0068] S3.3: Set the number of neurons in the hidden layer.

[0069] The number of neurons in the hidden layer is set to 32. The output of the hidden layer is calculated using the following formula:

[0070] h d =f(W k ·x k +b d (1)

[0071] Where f is the activation function ReLU, W k Let W1 be the weight matrix of the k-th feature, k∈[1,3], where k is an integer, and W1 is the weight of a driver in different driving periods.

[0072] W2 is the weight matrix of the root mean square error of the heart rate signal to the hidden layer. The initial matrix of W2 is randomly generated and subsequently updated through iteration. W3 is the weight matrix of the LF / HF ratio to the hidden layer. The initial matrix of W3 is randomly generated and subsequently updated through iteration.

[0073] x k The input feature vector is given by x1, where x2 is the start time, x3 is the root mean square error, and x4 is the LF / HF ratio.

[0074] b d h represents the bias term of the randomly generated d-th hidden layer neuron, where d ∈ [1, 32] and d is an integer; d The output of the d-th hidden layer neuron.

[0075] S3.4: Calculate the probability of fatigued driving in the output layer.

[0076] β z =σ(W h ·h d +b h (2)

[0077] σ is the Sigmoid function; W h These are the weights from the hidden layer to the output layer, and they are known quantities; b h It is the bias of the output layer.

[0078] S3.5: Calculate the loss function and gradient.

[0079]

[0080] Among them, y z β represents the true label of the z-th sample, which is either 0 or 1 (0 indicates no fatigue, 1 indicates fatigue). z Let S represent the probability predicted for the z-th sample; S is the total number of samples.

[0081] S3.6: Iterative weight matrix.

[0082]

[0083] W k,new W is the updated k-th weight matrix; k The original weights are the weights before the update; η is the learning rate. Let be the gradient of the k-th weight matrix.

[0084] S3.7: Set the number of iterations, looping through S3.3-S3.6.

[0085] In this embodiment, the number of iterations is 100. The optimal weight matrix after iteration is obtained, denoted as W. k,best .

[0086] S4: Determine if the driver is fatigued.

[0087] The optimal weight matrix W obtained after iteration in S3.7 k,best Substituting these values ​​into equations (1) and (2), we can solve for the driving state probability β of the current sample. 当前 ,

[0088]

[0089] Where θ = 0.5.

[0090] Based on the above-described preferred embodiments of the present invention, and through the foregoing description, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A method for identifying driver fatigue, characterized in that, Includes the following steps: S1: Continuously monitor the electrocardiogram (ECG) of a driver during the driving process, using time interval T as the interval, to obtain several intervals of ECG data for the driver, and then preprocess these interval ECG data; specifically, this includes the following steps: S1.1: Calculate the time interval between adjacent R waves in a certain interval of the electrocardiogram (ECG), and combine it with the start time of the ECG in that interval to form the result sequence of the ECG in that interval, as follows: [tm,RR (1,tm) ,RR (2,tm) ,……,RR (Q,tm) , ..., RR (i,tm) ]; where tm is the start time of the m-th interval ECG; RR (Q,tm) This represents the Q-th time interval in the ECG within the m-th interval; S1.2: Calculate the root mean square error of the electrocardiogram within this interval using the following formula: S1.3: Perform a Fourier transform (FFT) on the result sequence from S1.1 to extract the frequency domain features of the HRV. The formula is as follows: Wherein, LF range and HF range represent the frequency ranges of ECG in the low and high frequencies (0.04-0.15Hz and 0.15-0.4Hz, respectively); P(f1) and P(f2) represent the power spectral density of ECG in the low and high frequencies, respectively. S2: Set the weight matrix for all drivers at different driving times; W1=[α1,α2,……,α p ], where α p Fatigue coefficient; S3: Input the driving start time, root mean square error, and LF / HF from the result sequence into the neural network, and combine them with the driver's weight matrix at different driving time periods to iteratively calculate the optimal weight matrix; S4: Substitute the optimal weight matrix into S3 to calculate the probability of the current driving state and evaluate the current driving state; Step S3 includes the following steps: S3.1: Obtain a large number of samples, each of which includes the time point, root mean square error, and LF / HF of different driver outcome sequences; divide the large number of samples into training set, validation set, and test set according to the proportions; S3.2: Set the number of neurons in the input layer and hidden layer; and calculate the output of the hidden layer using the following formula: h d =f(W k ·x k +b d )(1) Where f is the activation function ReLU, W k Let x be the weight matrix for the k-th feature, where k ∈ [1, 3] and k is an integer; k b is the input feature vector; d h is the bias term for the randomly generated d-th hidden layer neuron, where d∈[1,32] and d is an integer; d The output of the d-th hidden layer neuron; S3.3: Calculate the probability of fatigued driving in the output layer; b z =σ(W h ·h d +b h ) (2) σ is the Sigmoid function; W h These are the weights from the hidden layer to the output layer, and they are known quantities; b h It is the bias of the output layer; S3.4: Calculate the loss function; Among them, y z β represents the true label of the z-th sample, which is either 0 or 1; z Let S represent the probability predicted for the z-th sample; S is the total number of samples. S3.5: Iterative weight matrix; W k,new W is the updated k-th weight matrix; k The original weights are the weights before the update; η is the learning rate. The gradient of the k-th weight matrix; S3.6: Set the number of iterations, loop through S3.2-S3.5, and obtain the optimal weight matrix, denoted as W. k,best ; In step S4, W in S3 is... k,best Substituting into equations (1) and (2), we can solve for the driving state probability β of the current sample. 当前 The driving condition is evaluated using an evaluation formula, which is as follows: Where θ = 0.

5.

2. The driver fatigue recognition method according to claim 1, characterized in that, In step S3.6, the number of iterations in S3.6 is set to 100.

3. The driver fatigue recognition method according to claim 1, characterized in that, In step S2, the time period when fatigue is easily experienced is set in W1, using the following formula:

Images