Fatigue driving detection system based on image recognition

Abstract

The invention discloses a fatigue driving detection system based on image recognition and belongs to the technical field of fatigue driving. The invention aims to solve the problem that the existing fatigue driving technology is difficult to apply due to large operand. A camera is arranged in an automobile and used for collecting a face image of a driver and sending the face image to a face recognition module, and the face recognition module extracts a face area in the face image and sends the face area to a fatigue calculation and judgment module; the fatigue calculation and judgment module is used for extracting and calculating the percentage of eye closing time in unit time and the percentage of mouth opening time in unit time by utilizing the face region image, and acquiring the fatigue of the current driver by utilizing the percentage of eye closing time in unit time and the percentage of mouth opening time in unit time; and the fatigue calculation and judgment module is loaded to a controller in an automobile cabin. The method is mainly used for fatigue driving detection.

Description

Technical Field

[0001] The invention belongs to the technical field of fatigue driving. Background Art

[0002] With the development of intelligent technology, consumers have an increasing demand for intelligent car cockpits. Among them, the popularization of fatigue driving detection technology is imminent. Therefore, there are many studies on this technology at home and abroad. The current mainstream solution is fatigue driving detection technology based on the YOLO (You Only Look Once) series of models. As an advanced real-time object detection algorithm, YOLO has been used in many fields due to its high efficiency and accuracy. Through real-time monitoring and analysis of the driver's facial features, such as blinking frequency, yawning times, and head posture, the YOLO model can accurately determine whether the driver is in a state of fatigue.

[0003] However, due to the cost of vehicle electronics, some domestically produced mid-range cockpit chips are insufficiently equipped with AI computing power. For example, the Xinchi X9M cockpit chip has difficulty running the YOLO model smoothly for inference on the terminal side, making it difficult to apply fatigue driving detection technology in practice. Summary of the invention

[0004] The present invention aims to solve the problem that the existing fatigue driving technology has large computational complexity and thus is difficult to apply, and now provides a fatigue driving detection method based on image recognition.

[0005] The fatigue driving detection system based on image recognition of the present invention comprises: a camera, a face recognition module and a fatigue degree calculation and judgment module;

[0006] The camera is arranged in the car, and is used to collect the driver's facial image, and send it to the face recognition module, the face recognition module extracts the face area in the face image and sends it to the fatigue calculation and judgment module, the fatigue calculation and judgment module calculates the percentage of eye closure time per unit time and the percentage of mouth opening time per unit time by using the face area image extraction, and obtains the current driver's fatigue by using the percentage of eye closure time per unit time and the percentage of mouth opening time per unit time;

[0007] The fatigue calculation and judgment module is loaded into a controller in the car cabin.

[0008] Furthermore, in the present invention, the calculation formula for the percentage of eye closure time per unit time is:

[0009]

[0010] Among them, P PERCLOSThe parameter represents the percentage of eye closure duration in unit time, n represents the total number of video frames in unit time, and f i Indicates the number of frames with closed eyes in the video per unit time.

[0011] Furthermore, in the present invention, the calculation formula for the percentage of mouth opening time per unit time is:

[0012]

[0013] Among them, P PMOT Indicates the percentage of mouth opening time per unit time, P i Indicates the number of frames with open mouth in the video per unit time.

[0014] The present invention separates image recognition from fatigue calculation, effectively improving the calculation efficiency and the utilization rate of the automobile main control. At the same time, the face recognition module is implemented by a neural network, which can quickly extract the facial features of the driver to be identified. The fatigue calculation and judgment module then calculates the ratio of the frames with closed eyes and open mouth in the video stream per unit time to the total number of frames per unit time to estimate the driver's fatigue. Since image recognition is implemented by a neural network, the driver's fatigue estimation is embedded in the automobile main control CPU, which effectively improves the efficiency of fatigue estimation and can effectively improve the efficiency of the application. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a block diagram of the system principles of the present invention. DETAILED DESCRIPTION

[0016] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0017] Specific implementation method 1: refer to Figure 1 Specifically describing this embodiment, the fatigue driving detection system based on image recognition described in this embodiment includes: a camera, a face recognition module and a fatigue degree calculation and judgment module;

[0018] The camera is arranged in the car, and is used to collect the driver's facial image, and send it to the face recognition module, the face recognition module extracts the face area in the face image and sends it to the fatigue calculation and judgment module, the fatigue calculation and judgment module calculates the percentage of eye closure time per unit time and the percentage of mouth opening time per unit time by using the face area image extraction, and obtains the current driver's fatigue by using the percentage of eye closure time per unit time and the percentage of mouth opening time per unit time;

[0019] The fatigue calculation and judgment module is loaded into a controller in the car cabin.

[0020] Furthermore, in the present invention, the calculation formula for the percentage of eye closure time per unit time is:

[0021]

[0022] The PERCLOS parameter indicates the percentage of eye closure duration in a unit time, n indicates the total number of video frames in a unit time, and f i Indicates the number of frames with closed eyes in the video per unit time.

[0023] Furthermore, in the present invention, the calculation formula for the percentage of mouth opening time per unit time is:

[0024]

[0025] Among them, PMOT represents the percentage of mouth opening time per unit time, P i Indicates the number of frames with open mouth in the video per unit time.

[0026] The present invention uses the percentage of the number of frames with open mouth in unit time to the total frames of the video stream in unit time and the percentage of the total frames with closed eyes in unit time to the total frames of the video stream in unit time to judge the fatigue degree.

[0027] The present invention adopts the YOLO model to detect human faces and combines it with DLIB to extract feature points, and applies the PERCLOS fatigue algorithm to calculate the driver's fatigue level. This can reduce the computing pressure of the NPU, reasonably utilize the remaining CPU (car main controller) computing power, and smoothly run the fatigue detection algorithm.

[0028] The main idea of ​​the solution adopted by the present invention is to reduce the amount of model calculations and combine the DLIB and PERCLOS algorithms to detect the driver's status. Dlib is a modern C++ toolkit that contains machine learning algorithms and tools and is widely used in image processing and computer vision. Its powerful face detection and feature point positioning functions make it one of the preferred tools in the field of face recognition. DLIB can quickly detect 68 feature points in the image and return the coordinate values ​​of the corresponding feature points.

[0029] The present invention uses the PERCLOS (percentage of eye closure time per unit time) parameter and the PMOT (percentage of mouth open time per unit time) parameter. PERCLOS is a physical quantity proposed by the Carnegie Mellon Institute to measure fatigue. The Federal Highway Administration of the United States conducted a correlation analysis study on nine fatigue parameters such as PERCLOS, blinking frequency, and yawning in a driving situation, and concluded that the PERCLOS parameter has the highest correlation with the fatigue driving state.

[0030] The PERCLOS parameter indicates the percentage of eye closure duration in a unit time, and is expressed as the ratio of eye closure frames to the total number of video frames n in a unit time. The calculation formula is as follows:

[0031]

[0032] According to current research data, the P80 standard, which means that the eyelid covers 80% of the pupil area and is judged as closed eyes, is the most accurate for detecting fatigue status.

[0033] The calculation formula of the PMOT parameter is shown in the following formula, which is used to express the percentage of mouth opening time in unit time, and is expressed by the ratio of the number of mouth opening frames to the total number of video frames m in unit time.

[0034]

[0035] Using PMOT parameters alone may sometimes cause false positives of yawning or talking. When actually performing fatigue detection, the threshold can be determined in combination with the blinking frequency.

[0036] The YOLO model is only responsible for detecting the position of the face, which greatly reduces the amount of computation of the GPU and NPU. Then Dlib extracts the six feature points of the eyes, and finally calculates the fatigue state through the perclos algorithm. Among them, DLIB and perclos algorithms only require the CPU to participate in the calculation. After actual measurement, this scheme increases the frame rate from 8 to 30, meets the real-time requirements, and the detection accuracy is consistent with the mainstream YOLO scheme. The present invention decomposes part of the detection tasks of the YOLO model (neural network) into the DLIB and perclos algorithm modules, reduces the amount of computation of the NPU, and reasonably utilizes the remaining CPU computing power. Reasonable modular decomposition of fatigue driving detection technology optimizes the use of computing power of the entire chip, so that the fatigue driving detection scheme can run smoothly on the mid-end domestic chip Core Chi X9M. The present invention not only saves NPU computing power, increases the overall solution modules, increases the complexity of the software architecture, but also has limited reference value on chips with sufficient NPU computing power.

[0037] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that the features described in conjunction with a single embodiment may be used in other described embodiments.

Claims

1. The fatigue driving detection system based on image recognition is characterized by: include: Camera, face recognition module and fatigue calculation and judgment module; The camera is arranged in the car, and is used to collect the driver's facial image, and send it to the face recognition module, the face recognition module extracts the face area in the face image and sends it to the fatigue calculation and judgment module, the fatigue calculation and judgment module calculates the percentage of eye closure time per unit time and the percentage of mouth opening time per unit time by using the face area image extraction, and obtains the current driver's fatigue by using the percentage of eye closure time per unit time and the percentage of mouth opening time per unit time; The fatigue calculation and judgment module is loaded into a controller in the car cabin.

2. The fatigue driving detection system based on image recognition according to claim 1 is characterized in that: The formula for calculating the percentage of eye closure time per unit time is: Among them, P PERCLOS The parameter represents the percentage of eye closure duration in unit time, n represents the total number of video frames in unit time, and f i Indicates the number of frames with closed eyes in the video per unit time.

3. The image recognition-based fatigue driving detection system according to claim 2 is characterized in that: The formula for calculating the percentage of mouth opening time per unit time is: Among them, P PMOT Indicates the percentage of mouth opening time per unit time, P i Indicates the number of frames with open mouth in the video per unit time.

Images