A driver fatigue detection method based on YOLO neural network
By using a driver fatigue detection method based on YOLO neural networks, a model was built using a training dataset and the YOLOv5 algorithm, which enabled accurate identification and real-time early warning of driver fatigue, thereby reducing the occurrence of traffic accidents.
Patent Information
- Application Number
- CN202111665628.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-31
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-12-31
AI Technical Summary
Existing driver fatigue detection technologies lack precise criteria, resulting in low accuracy and a high risk of detection errors.
The driver fatigue detection method based on YOLO neural network is developed by creating a training dataset of blinking images, using Roboflow to label the data, establishing a YOLOv5 algorithm training model, and connecting it to a driver's cab monitoring camera for real-time monitoring and early warning.
This improves the accuracy of driver fatigue detection, which can effectively reduce the occurrence of traffic accidents.
Smart Images

Figure CN114359879B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of computer vision, and in particular relates to a driver fatigue detection method based on a YOLO neural network. Background Art
[0002] Traffic accidents have been recognized as the number one public hazard that endangers human life safety today. According to traffic accident statistics, 80%-90% of traffic accidents are caused by human factors. Driver fatigue driving will double the possibility of traffic accidents. About 48% of traffic accidents are caused by driver fatigue driving. Studies have shown that traffic accidents usually occur within a few seconds. If a warning can be given 2 seconds in advance, 92% of traffic accidents can be reduced. If a warning is given 0.5 seconds in advance, 73% of traffic accidents can be avoided.
[0003] The existing detection technology does not have a very accurate standard for fatigue driving, and the detection technology used is relatively conservative. The final result is judged only through simple face recognition. Not only is the recognition method inaccurate, but it is also easy to cause detection errors. Summary of the invention
[0004] In view of the above problems, the present invention provides a driver fatigue detection method based on YOLO neural network, comprising the following steps:
[0005] Create a training dataset of blinking images;
[0006] Use roboflow to annotate the blinking pictures in the dataset;
[0007] Based on the data set, a training model is established using the YOLOv5 algorithm;
[0008] Input the annotated blinking images into the training model to train the model;
[0009] After the training model is mature, a discriminant model is established based on the training model;
[0010] The discrimination model is connected to the surveillance camera in the cab for real-time monitoring and early warning.
[0011] Preferably, the data set includes a training set, a validation set and a test set;
[0012] The training model includes a YOLO neural network.
[0013] Preferably, N×n original blink data DEO are obtained from the blink picture training data set, where N is the number of samples, n is the number of blink frequency level categories, and a sliding window with a step size of h+δ is used along the time dimension of the original blink data to obtain L blink data DE with a size of TE×E, where h represents a fixed step size, δ represents a random disturbance, TE=FE×t, TE represents the number of frames of blink data, FE represents the sampling rate of the eye tracker, t represents the acquisition time, and E represents the dimensional size of the blink data;
[0014] Obtain N×n original blink data DIO from the blink image training data set, and extract L corresponding blink data DI of size TI×I according to the timestamp of the acquired blink data DE, where TI=FI×t, TI represents the number of frames of blink data, FI represents the number of frames per second FPS, t represents the acquisition time, and I represents the dimension size of the blink data;
[0015] Obtain N×n original depth data DPO from the blink image training data set, and extract corresponding L depth data DP of size TI×P according to the blink data DI obtained in the blink image training data set, where P represents the dimension size of the depth data;
[0016] Obtain the blink frequency level label corresponding to the blink data DE from the blink picture training dataset;
[0017] Obtain 3L / 5 groups of training blink data DEtrain, training blink data DItrain and training depth data DPtrain from the blink data DE, blink data DI and depth data DP respectively, form the training blink data DEtrain, training blink data DItrain and training depth data DPtrain into a training set, and form the training set and the corresponding blink frequency level label into a training sample;
[0018] Obtain 1L / 5 groups of verification blink data DEval, verification blink data DIval and verification depth data DPval from the blink data DE, blink data DI and depth data DP respectively, form the verification blink data DEval, verification blink data DIval and verification depth data DPval into a verification set, and form a verification sample set with the verification set and the corresponding blink frequency level label;
[0019] 1L / 5 groups of test blink data DEtest, test blink data DItest and test depth data DPtest are obtained from the blink data DE, the blink data DI and the depth data DP respectively, and the test blink data DEtest, the test blink data DItest and the test depth data DPtest are combined into a test set.
[0020] Preferably, the ratio of the number of images in the training set, validation set and test set is 6:2:2.
[0021] Preferably, the training model includes a training algorithm, and the training algorithm includes the following steps:
[0022] First, resize the blinking images in the input training set to 448x448;
[0023] Then the separated blinking images are fed into the CNN network;
[0024] Finally, the network prediction results are processed to obtain the detection target;
[0025] The validation set is then fed into the training model to verify the target accuracy;
[0026] Finally, the images in the test set are used for testing.
[0027] Preferably, the discrimination model includes: a primary judgment module and a confirmation judgment module; wherein the primary judgment module is used to preliminarily judge the driver's fatigue level, and then transfer to the confirmation judgment module for final confirmation.
[0028] Preferably, it further comprises a detection system, wherein the detection system is connected to the discrimination model, and the detection system is connected to an alarm system.
[0029] The beneficial effects of the present invention are:
[0030] In the present invention, a data set is prepared based on the YOLOv5 algorithm; a training model is established based on the data set, and a discrimination model is established based on the training model, and blinking pictures in the data set are annotated using roboflow; the pictures are input into the training model to train the model; after the training model is mature, a discrimination model is established based on the training model; the discrimination model is connected to the monitoring camera in the cab for real-time monitoring and early warning. The model has a good recognition effect on driver fatigue and effectively reduces the occurrence of accidents.
[0031] Other features and advantages of the present invention will be described in the following description, and part of them will become obvious from the description, or be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0033] Figure 1 Flow chart of the detection method of the present invention. DETAILED DESCRIPTION
[0034] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0035] In an embodiment of the present invention, a driver fatigue detection method based on a YOLO neural network is proposed, comprising the following steps:
[0036] Create a training dataset of blinking images;
[0037] Use roboflow to annotate the blinking pictures in the dataset;
[0038] Based on the data set, a training model is established using the YOLOv5 algorithm;
[0039] Input the annotated blinking images into the training model to train the model;
[0040] After the training model is mature, a discriminant model is established based on the training model;
[0041] The discrimination model is connected to the surveillance camera in the cab for real-time monitoring and early warning.
[0042] Wherein, the data set includes a training set, a validation set and a test set;
[0043] The training model includes a YOLO neural network.
[0044] Obtain N×n original blink data DEO from the blink image training data set, where N is the number of samples, n is the number of blink frequency level categories, and use a sliding window with a step size of h+δ along the time dimension of the original blink data to obtain L blink data DE with a size of TE×E, where h represents a fixed step size, δ represents random perturbation, TE=FE×t, TE represents the number of frames of blink data, FE represents the sampling rate of the eye tracker, t represents the acquisition time, and E represents the dimensional size of the blink data;
[0045] Obtain N×n original blink data DIO from the blink image training data set, and extract L corresponding blink data DI of size TI×I according to the timestamp of the acquired blink data DE, where TI=FI×t, TI represents the number of frames of blink data, FI represents the number of frames per second FPS, t represents the acquisition time, and I represents the dimension size of the blink data;
[0046] Obtain N×n original depth data DPO from the blink image training data set, and extract corresponding L depth data DP of size TI×P according to the blink data DI obtained in the blink image training data set, where P represents the dimension size of the depth data;
[0047] Obtain the blink frequency level label corresponding to the blink data DE from the blink picture training dataset;
[0048] Obtain 3L / 5 groups of training blink data DEtrain, training blink data DItrain and training depth data DPtrain from the blink data DE, blink data DI and depth data DP respectively, form the training blink data DEtrain, training blink data DItrain and training depth data DPtrain into a training set, and form the training set and the corresponding blink frequency level label into a training sample;
[0049] Obtain 1L / 5 groups of verification blink data DEval, verification blink data DIval and verification depth data DPval from the blink data DE, blink data DI and depth data DP respectively, form the verification blink data DEval, verification blink data DIval and verification depth data DPval into a verification set, and form a verification sample set with the verification set and the corresponding blink frequency level label;
[0050] 1L / 5 groups of test blink data DEtest, test blink data DItest and test depth data DPtest are obtained from the blink data DE, the blink data DI and the depth data DP respectively, and the test blink data DEtest, the test blink data DItest and the test depth data DPtest are combined into a test set.
[0051] The ratio of the number of images in the training set, validation set, and test set is 6:2:2. The main purpose of dividing the set into several sets is to improve the accuracy of the model.
[0052] The training model includes a training algorithm, which includes the following steps:
[0053] First, resize the blinking images in the input training set to 448x448;
[0054] Then the separated blinking images are fed into the CNN network;
[0055] Finally, the network prediction results are processed to obtain the detection target;
[0056] The validation set is then fed into the training model to verify the target accuracy;
[0057] Finally, the images in the test set are used for testing.
[0058] The discrimination model includes: a primary judgment module and a confirmation judgment module; wherein the primary judgment module is used to preliminarily judge the driver's fatigue level, and then transfer to the confirmation judgment module for final confirmation.
[0059] It also includes a detection system, which is connected to the discrimination model and connected to an alarm system.
[0060] The camera inside the car actually captures the driver's eyes and obtains the driver's blinking frequency and eye closing time. For example, if the blinking frequency decreases, the eye closing time increases, and the discrimination model can determine whether the driver is tired, and then the alarm system will sound an alarm.
[0061] Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent substitutions for some of the technical features therein; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A driver fatigue detection method based on YOLO neural network, characterized in that: The steps include: Creating a blink image training data set, the data set comprising a training set, a validation set, and a test set; Obtain N×n original blink data DEO from the blink image training data set, where N is the number of samples, n is the number of blink frequency level categories, and use a sliding window with a step size of h+δ along the time dimension of the original blink data to obtain L blink data DE with a size of TE×E, where h represents a fixed step size, δ represents random perturbation, TE=FE×t, TE represents the number of frames of blink data, FE represents the sampling rate of the eye tracker, t represents the acquisition time, and E represents the dimensional size of the blink data; Obtain N×n original blink data DIO from the blink image training data set, and extract L corresponding blink data DI of size TI×I according to the timestamp of the acquired blink data DE, where TI=FI×t, TI represents the number of frames of blink data, FI represents the number of frames per second FPS, t represents the acquisition time, and I represents the dimension size of the blink data; Obtain N×n original depth data DPO from the blink image training data set, and extract corresponding L depth data DP of size TI×P according to the blink data DI obtained in the blink image training data set, where P represents the dimension size of the depth data; Obtain the blink frequency level label corresponding to the blink data DE from the blink picture training dataset; Obtain 3L / 5 groups of training blink data DEtrain, training blink data DItrain and training depth data DPtrain from the blink data DE, blink data DI and depth data DP respectively, form the training blink data DEtrain, training blink data DItrain and training depth data DPtrain into a training set, and form the training set and the corresponding blink frequency level label into a training sample; Obtain 1L / 5 groups of verification blink data DEval, verification blink data DIval and verification depth data DPval from the blink data DE, blink data DI and depth data DP respectively, form the verification blink data DEval, verification blink data DIval and verification depth data DPval into a verification set, and form a verification sample set with the verification set and the corresponding blink frequency level label; Obtain 1L / 5 groups of test blink data DEtest, test blink data DItest and test depth data DPtest from the blink data DE, the blink data DI and the depth data DP respectively, and form the test blink data DEtest, the test blink data DItest and the test depth data DPtest into a test set; Use roboflow to annotate the blinking pictures in the dataset; Based on the data set, a training model is established using the YOLOv5 algorithm; Input the annotated blinking images into the training model to train the model; After the training model is mature, a discriminant model is established based on the training model; The discrimination model is connected to the surveillance camera in the cab for real-time monitoring and early warning.
2. A driver fatigue detection method based on YOLO neural network according to claim 1, characterized in that: The training model includes a training algorithm, which includes the following steps: First, resize the blinking images in the input training set to 448x448; Then the separated blinking images are fed into the CNN network; Finally, the network prediction results are processed to obtain the detection target; The validation set is then fed into the training model to verify the target accuracy; Finally, the images in the test set are used for testing.
3. A driver fatigue detection method based on YOLO neural network according to claim 2, characterized in that: The discrimination model includes: a primary judgment module and a confirmation judgment module; wherein the primary judgment module is used to preliminarily judge the driver's fatigue level, and then transfer to the confirmation judgment module for final confirmation.
4. The driver fatigue detection method based on YOLO neural network according to claim 1 is characterized in that: It also includes a detection system, which is connected to the discrimination model and connected to an alarm system.
