Fatigue driving detection method, equipment, medium and device based on face registration points
Through the fatigue driving detection method based on the face registration point, the binocular registration point and pitch angle data in the face image data are used to accurately judge the driver's fatigue status, and the detection results in the prior art are affected by driving habits. Big problem.
Patent Information
- Application Number
- CN202010430741.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-05-20
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2040-05-20
AI Technical Summary
The existing fatigue driving detection methods are greatly affected by the driver's driving habits, resulting in inaccurate detection results.
The fatigue driving detection method based on the face registration point is adopted. By collecting and processing the driver's face image data, the coordinates of the binocular registration point are extracted, the eye height and pixel width are calculated, and the pitch angle data is combined to determine whether the driver is in a fatigue state. Avoid the impact of driving habits on detection.
It improves the accuracy of fatigue driving detection, reduces misjudgment, and provides more accurate driver fatigue status judgments.
Smart Images

Figure CN111753659B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of fatigue driving detection, and in particular to a fatigue driving detection method, equipment, medium and device based on face registration points. Background Art
[0002] With the improvement of living standards, cars have become a means of transportation for families instead of luxury goods. As a result, there are also many potential safety hazards caused by cars. In the process of driving a car, the driver is prone to decreased attention or even sleepiness after driving for a long time, which may lead to traffic accidents. Therefore, it is essential to detect whether the driver is driving fatigued during driving, which can effectively reduce the occurrence of automobile safety accidents.
[0003] At present, domestic and foreign auto parts manufacturers have made some detection and prevention measures for fatigue driving. The most commonly used fatigue detection method is driver driving behavior analysis, that is, by recording and analyzing the driver's behavioral characteristics such as turning the steering wheel and stepping on the brakes, to determine whether the driver is fatigued. However, the existing fatigue driving detection method is greatly affected by the driver's driving habits, resulting in inaccurate fatigue driving detection results. Summary of the invention
[0004] In order to overcome the shortcomings of the prior art, one of the objectives of the present invention is to provide a fatigue driving detection method based on face registration points, which can solve the problem that the existing fatigue driving detection method is greatly affected by the driver's driving habits, resulting in inaccurate fatigue driving detection results.
[0005] The second object of the present invention is to provide an electronic device that can solve the problem that the existing fatigue driving detection method is greatly affected by the driver's driving habits, resulting in inaccurate fatigue driving detection results.
[0006] The third object of the present invention is to provide a computer-readable storage medium, which can solve the problem that the existing fatigue driving detection method is greatly affected by the driver's driving habits, resulting in inaccurate fatigue driving detection results.
[0007] The fourth object of the present invention is to provide a fatigue driving detection device based on face registration points, which can solve the problem that the existing fatigue driving detection method is greatly affected by the driver's driving habits, resulting in inaccurate fatigue driving detection results.
[0008] One of the purposes of the present invention is achieved by the following technical solution:
[0009] The fatigue driving detection method based on face registration points includes the following steps:
[0010] Image acquisition, collecting facial image data of the driver to be detected;
[0011] Face recognition, inputting the face image data into a preset face recognition model for computational processing to obtain recognized face image data containing face registration point data;
[0012] Registration point data extraction, extracting face registration point data from the identified face image data, wherein the face registration point data contains binocular registration point coordinates;
[0013] Fatigue driving judgment: judging whether the person to be detected is in a fatigue state according to the coordinates of the binocular registration points.
[0014] Furthermore, the binocular registration point coordinates include upper eyelid registration point coordinates, lower eyelid registration point coordinates, and eye corner registration point coordinates, and the fatigue driving determination includes:
[0015] Calculating the eye height, calculating the eye height of both eyes according to the upper eyelid registration point coordinates and the lower eyelid registration point coordinates, wherein the eye height is the distance from the upper eyelid to the lower eyelid;
[0016] Calculating the eye pixel width, and calculating the eye pixel width corresponding to both eyes according to the eye corner registration point coordinates;
[0017] Eye-closing state determination, calculating the ratio of the eye height to the eye pixel width of both eyes respectively, when the ratios corresponding to both eyes are less than a preset threshold, the driver to be detected is in an eye-closing state;
[0018] Fatigue determination: when the time that the driver to be detected is in the eyes-closed state exceeds a preset fatigue eyes-closed time threshold, the driver to be detected is in a fatigue state.
[0019] Furthermore, the fatigue driving judgment also includes: when the time that the driver to be detected is in a closed-eye state does not exceed a preset fatigue closed-eye time threshold and exceeds a preset normal blinking duration, the driver to be detected is in a fatigue blinking state; when the number of times the driver to be detected is in a fatigue blinking state exceeds a preset fatigue blinking number threshold, the driver to be detected is in a fatigue state.
[0020] Furthermore, the recognized facial image data also contains eye-eye pitch angle data, and the fatigue driving judgment also includes: extracting the eye-eye pitch angle data from the recognized facial image data, and determining whether the driver to be detected is in a fatigue state based on the eye-eye pitch angle data.
[0021] Furthermore, determining whether the driver to be detected is in a fatigue state based on the binocular pitch angle data is specifically as follows: obtaining the head-lowering angle of the driver to be detected based on the binocular pitch angle data; when the head-lowering angle is greater than a preset head-lowering angle threshold and the head-lowering time exceeds a preset head-lowering time threshold, the driver to be detected is in a fatigue state, and the head-lowering time is the total time that the head-lowering angle is greater than the preset head-lowering angle threshold.
[0022] Furthermore, the face recognition also includes image preprocessing before preprocessing the face image data, and the preprocessing includes cropping the face image data and extracting color channels.
[0023] The second object of the present invention is achieved by adopting the following technical solution:
[0024] An electronic device, comprising: a processor;
[0025] A memory; and a program, wherein the program is stored in the memory and is configured to be executed by a processor, and the program includes a method for executing the fatigue driving detection method based on face registration points of the present application.
[0026] The third object of the present invention is achieved by adopting the following technical solutions:
[0027] A computer-readable storage medium stores a computer program, wherein the computer program is executed by a processor to implement the fatigue driving detection method based on face registration points of the present application.
[0028] The fourth object of the present invention is achieved by the following technical solutions:
[0029] The fatigue driving detection device based on face registration points includes an image acquisition module, a processor, and a signal processing module. The image acquisition module is connected to the processor, the processor is connected to the signal processing module, and a preset face recognition model is built in the processor.
[0030] The image acquisition module acquires facial image data of a driver to be detected, and the processor inputs the facial image data into a preset facial recognition model for operation processing to obtain recognized facial image data containing facial registration point data; the processor extracts facial registration point data from the recognized facial image data, and the facial registration point data contains binocular registration point coordinates; the signal processing module determines whether the person to be detected is in a fatigue state based on the binocular registration point coordinates to obtain driving status information, and the signal processing module sends the driving status information to a CAN bus in the form of a CAN message.
[0031] Furthermore, the image acquisition module includes a common image sensor and an infrared image sensor, and the signal processing module sends the driving status information in the form of a CAN message to the CAN bus according to a preset sending interval.
[0032] Compared with the prior art, the beneficial effect of the present invention is that the fatigue driving detection method based on face registration points in the present application includes the following steps: collecting face image data of the driver to be detected; inputting the face image data into a preset face recognition model for calculation and processing to obtain recognized face image data containing face registration point data; extracting face registration point data from the recognized face image data, wherein the face registration point data contains binocular registration point coordinates; judging whether the person to be detected is in a fatigue state according to the binocular registration point coordinates. By calculating and processing the collected face image data to obtain binocular registration point coordinates, and detecting whether the driver to be detected is in a fatigue state according to the binocular registration point coordinates, the influence of driving habits on the detection is avoided, and the detection result of fatigue driving is more accurate.
[0033] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention in conjunction with the accompanying drawings. The specific implementation of the present invention is given in detail by the following embodiments and their accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0035] Figure 1 It is a flow chart of the fatigue driving detection method based on face registration points of the present invention;
[0036] Figure 2 It is a schematic diagram of the distribution of coordinate points corresponding to face registration point data in the fatigue driving detection method based on face registration points of the present invention. DETAILED DESCRIPTION
[0037] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that, under the premise of no conflict, the various embodiments or technical features described below can be arbitrarily combined to form a new embodiment.
[0038] like Figure 1 The fatigue driving detection method based on face registration points shown in the figure comprises the following steps:
[0039] Image acquisition, collecting facial image data of the driver to be detected. In this embodiment, image acquisition uses an independent image acquisition module, that is, it includes a common image sensor and an infrared image sensor to collect facial images of the driver to be detected. The infrared image sensor includes an infrared fill light and an infrared camera. When collecting, the common image sensor is used for collection. The image acquisition module determines whether the light is qualified. If it is not qualified, the infrared fill light is started, and then the infrared camera is switched to collect to obtain the facial image data of the driver to be detected.
[0040] Image preprocessing, preprocessing the face image data, wherein the preprocessing includes cropping the face image data and extracting the color channels.
[0041] Face recognition: input the pre-processed face image data into a preset face recognition model for computational processing to obtain recognized face image data containing face registration point data. The recognized face image data also contains eye-to-eye pitch angle data.
[0042] Registration point data extraction, extracting face registration point data from the identified face image data, wherein the face registration point data includes eye registration point coordinates. In this embodiment, the face registration point data is data including registration point coordinates of various parts of the face, specifically including registration point coordinates of the face contour area, registration point coordinates of the left and right eyebrow areas, registration point coordinates of the eye area, registration point coordinates of the nose area, and registration point coordinates of the mouth area, such as Figure 2 As shown, the binocular registration point coordinates include 16 coordinate points, each eye includes 8 coordinate points, and the binocular registration point coordinates include the upper eyelid registration point coordinates, the lower eyelid registration point coordinates, and the eye corner registration point coordinates. In this embodiment, the binocular registration point coordinates are the registration point coordinates of the two eyes, that is, the registration point coordinates of each eye include the upper eyelid registration point coordinates, the lower eyelid registration point coordinates, and the eye corner registration point coordinates. Figure 2 The coordinate points corresponding to the coordinates of the corresponding registration points in the left eye area are: 0, 1, 2, 3, 4, 5, 6, 7. The coordinate points corresponding to the coordinates of the upper eyelid registration point in the left eye area are 5, 6, and 7. The coordinate points corresponding to the coordinates of the lower eyelid registration point in the left eye area are 1, 2, and 3. The coordinate points corresponding to the coordinates of the corner of the eye registration point in the left eye area are 0 and 4. The corner of the eye registration point includes the left corner of the eye registration point coordinates and the right corner of the eye registration point coordinates. Here 0 is the coordinate point corresponding to the left corner of the eye registration point coordinates, and 4 is the coordinate point corresponding to the right corner of the eye registration point coordinates.
[0043] Fatigue driving judgment, judging whether the person to be detected is in a fatigue state according to the coordinates of the binocular registration points. Specifically:
[0044] Calculate the eye height, and calculate the eye height of both eyes according to the coordinates of the upper eyelid registration point and the coordinates of the lower eyelid registration point, wherein the eye height is the distance from the upper eyelid to the lower eyelid. In this embodiment, the eye height of both eyes is calculated by this step. Specifically, as shown in formula (1):
[0045]
[0046] Where h is the eye height; w is the weight value; i is the coordinate point corresponding to the registration point coordinates; is the horizontal coordinate value of the coordinate point i corresponding to the coordinate of the upper eyelid registration point; is the horizontal coordinate value of the coordinate point i corresponding to the coordinate of the lower eyelid registration point; is the ordinate value of the coordinate point i corresponding to the coordinates of the upper eyelid registration point; is the ordinate value of the coordinate point i corresponding to the coordinate of the lower eyelid registration point; in this embodiment, the weight corresponding to each registration point coordinate is pre-set and is a known value.
[0047] Calculate the eye pixel width, and calculate the eye pixel width corresponding to both eyes according to the eye corner registration point coordinates. In this embodiment, the eye pixel widths of both eyes are calculated through this step. Specifically, as shown in formula (2):
[0048]
[0049] Where l is the pixel width of the eye; x right x is the horizontal coordinate value of the coordinate point corresponding to the right eye corner registration point coordinate; left y is the horizontal coordinate value of the coordinate point corresponding to the left eye corner registration point; right y is the ordinate value of the coordinate point corresponding to the right eye corner registration point coordinate; left The ordinate value of the coordinate point corresponding to the left eye corner registration point coordinates.
[0050] Eye-closing state determination, respectively calculate the ratio of the eye height of both eyes to the eye pixel width, when the ratios corresponding to both eyes are less than a preset threshold, the driver to be detected is in an eye-closing state. In this embodiment, first calculate the ratio of the eye height of both eyes to the eye pixel width, let the ratio be P, in the calculation process, because the denominator cannot be zero, therefore, Wherein, P is the ratio, h is the eye height, and l is the eye pixel width. In this embodiment, when the ratios corresponding to both eyes are less than the preset threshold, the driver to be detected is in a closed-eye state, and when the ratio corresponding to any one eye is less than the preset threshold, the driver to be detected is in an open-eye state.
[0051] Fatigue driving judgment: when the time when the driver to be detected is in the closed eye state does not exceed the preset fatigue closed eye time threshold and exceeds the preset normal blinking duration, the driver to be detected is in the fatigue blinking state; when the number of times the driver to be detected is in the fatigue blinking state exceeds the preset fatigue blinking number threshold, the driver to be detected is in a fatigue state. The preset fatigue closed eye time threshold in this embodiment is generally 15 seconds; the preset fatigue blinking number threshold is three times; and the preset normal blinking duration is 5 seconds.
[0052] In the present embodiment, the fatigue driving judgment also includes judging by the eye pitch angle, specifically: extracting the eye pitch angle data from the identified facial image data, and judging whether the driver to be detected is in a fatigue state according to the eye pitch angle data. The head-lowering angle of the driver to be detected is obtained according to the eye pitch angle data. When the head-lowering angle is greater than a preset head-lowering angle threshold and the head-lowering time exceeds the preset head-lowering time threshold, the driver to be detected is in a fatigue state, and the head-lowering time is the total time value when the head-lowering angle is greater than the preset head-lowering angle threshold. The preset head-lowering time threshold generally used in the present embodiment is 30 degrees, and the preset head-lowering time threshold is 15 seconds.
[0053] In an embodiment of the present invention, there is provided an electronic device, comprising: a processor;
[0054] A memory; and a program, wherein the program is stored in the memory and is configured to be executed by a processor, and the program includes a method for executing the fatigue driving detection method based on face registration points of the present application.
[0055] In an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and the computer program is executed by a processor to implement the fatigue driving detection method based on face registration points of the present application.
[0056] In an embodiment of the present invention, a fatigue driving detection device based on face registration points is provided, comprising an image acquisition module, a processor, and a signal processing module, wherein the image acquisition module is connected to the processor, the processor is connected to the signal processing module, and a preset face recognition model is built in the processor;
[0057] The image acquisition module acquires facial image data of the driver to be detected, and the processor inputs the facial image data into a preset face recognition model for calculation and processing to obtain recognized facial image data containing facial registration point data; the processor extracts facial registration point data from the recognized facial image data, and the facial registration point data contains binocular registration point coordinates; the signal processing module determines whether the person to be detected is in a fatigue state according to the binocular registration point coordinates, and obtains driving state information, and the signal processing module sends the driving state information to the CAN bus in the form of a CAN message according to a preset sending interval. In this embodiment, the image acquisition module includes an ordinary image sensor and an infrared image sensor to acquire facial images of the driver to be detected, and the infrared image sensor includes an infrared fill light and an infrared camera. When acquiring, an ordinary image sensor is used for acquisition, and the image acquisition module determines whether the light is qualified. If it is not qualified, the infrared fill light is started, and then the infrared camera is switched to acquire, and the facial image data of the driver to be detected is obtained.
[0058] The fatigue driving detection method based on face registration points in the present application includes the following steps: collecting face image data of the driver to be detected; inputting the face image data into a preset face recognition model for computational processing to obtain recognized face image data containing face registration point data; extracting face registration point data from the recognized face image data, wherein the face registration point data contains binocular registration point coordinates; judging whether the person to be detected is in a fatigue state according to the binocular registration point coordinates. By performing computational processing on the collected face image data to obtain binocular registration point coordinates, and detecting whether the driver to be detected is in a fatigue state according to the binocular registration point coordinates, the influence of driving habits on the detection is avoided, and the detection result of fatigue driving is more accurate.
[0059] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in any form. Any ordinary technician in the industry can smoothly implement the present invention as shown in the drawings and above. However, any equivalent changes, modifications and evolutions made by technicians familiar with the profession without departing from the scope of the technical solution of the present invention using the technical content disclosed above are all equivalent embodiments of the present invention. At the same time, any equivalent changes, modifications and evolutions made to the above embodiments based on the essential technology of the present invention are still within the protection scope of the technical solution of the present invention.
Claims
1. Fatigue driving detection method based on face registration points, Features: The following steps are involved: Image acquisition, collecting facial image data of the driver to be detected. Image acquisition uses an independent image acquisition module, including an ordinary image sensor and an infrared image sensor to collect facial images of the driver to be detected. The infrared image sensor includes an infrared fill light and an infrared camera. At the beginning of acquisition, an ordinary image sensor is used for acquisition. The image acquisition module determines whether the light is qualified. If it is unqualified, the infrared fill light is started, and then the infrared camera is switched to acquire the facial image data of the driver to be detected; face recognition, the facial image data is input into a preset face recognition model for calculation and processing, and the recognized facial image data containing face registration point data is obtained; Registration point data extraction, extracting face registration point data from the identified face image data, wherein the face registration point data contains binocular registration point coordinates; Fatigue driving judgment, judging whether the person to be detected is in a fatigue state according to the coordinates of the binocular registration points; The binocular registration point coordinates include upper eyelid registration point coordinates, lower eyelid registration point coordinates, and eye corner registration point coordinates. The fatigue driving judgment includes: calculating the eye height, and calculating the eye height of both eyes according to the upper eyelid registration point coordinates and the lower eyelid registration point coordinates. The eye height is the distance from the upper eyelid to the lower eyelid. The formula for calculating the eye height is: Where h is the eye height; w is the weight value; i is the coordinate point corresponding to the registration point coordinates; is the horizontal coordinate value of the coordinate point i corresponding to the coordinate of the upper eyelid registration point; is the horizontal coordinate value of the coordinate point i corresponding to the coordinate of the lower eyelid registration point; is the ordinate value of the coordinate point i corresponding to the coordinates of the upper eyelid registration point; is the ordinate value of the coordinate point i corresponding to the coordinate of the lower eyelid registration point; Calculate the eye pixel width. Calculate the eye pixel width corresponding to both eyes according to the eye corner registration point coordinates. The formula for calculating the eye width is: Where l is the pixel width of the eye; x right x is the horizontal coordinate value of the coordinate point corresponding to the right eye corner registration point coordinate; left y is the horizontal coordinate value of the coordinate point corresponding to the left eye corner registration point; right y is the ordinate value of the coordinate point corresponding to the right eye corner registration point coordinate; left The ordinate value of the coordinate point corresponding to the left eye corner registration point coordinate; Eyes closed state determination, respectively calculating the ratio of the eye height to the eye pixel width of both eyes, when the ratios corresponding to both eyes are less than the preset threshold value, the driver to be detected is in eyes closed state; fatigue determination, when the time when the driver to be detected is in eyes closed state exceeds the preset fatigue eyes closed time threshold, the driver to be detected is in fatigue state; the fatigue driving determination also includes: when the time when the driver to be detected is in eyes closed state does not exceed the preset fatigue eyes closed time threshold and exceeds the preset normal blinking time, the driver to be detected is in fatigue blinking state, when the number of times the driver to be detected is in fatigue blinking state exceeds the preset fatigue blinking number threshold, the driver to be detected is in fatigue state.
2. The method for detecting fatigue driving based on face registration points according to claim 1, Features: The face recognition also includes image preprocessing, which preprocesses the face image data. The preprocessing includes cropping the face image data and extracting color channels.
3. An electronic device, Features include: processor; Memory; And a program, wherein the program is stored in the memory and configured to be executed by a processor, and the program includes instructions for executing the method according to any one of claims 1 to 2.
4. A computer-readable storage medium having a computer program stored thereon, Features: The computer program is executed by a processor to perform the method according to any one of claims 1 to 2.
5. Fatigue driving detection device based on face registration points, Features: It includes an image acquisition module, a processor, and a signal processing module, wherein the image acquisition module is connected to the processor, and the processor is connected to the signal processing module, and a preset face recognition model is built in the processor; the image acquisition module acquires face image data of a driver to be detected, and the processor inputs the face image data into the preset face recognition model for operation processing to obtain recognized face image data containing face registration point data; the processor extracts face registration point data from the recognized face image data, and the face registration point data contains binocular registration point coordinates; the signal processing module determines whether the person to be detected is in a fatigue state according to the binocular registration point coordinates to obtain driving state information, and the signal processing module sends the driving state information to a CAN bus in the form of a CAN message; The fatigue driving detection method of the signal processing module based on the coordinates of the binocular registration points in the face registration points is the method described in any one of claims 1-2.
6. The fatigue driving detection device based on face registration points as claimed in claim 5, Features: The image acquisition module includes a common image sensor and an infrared image sensor, and the signal processing module sends the driving status information to the CAN bus in the form of a CAN message according to a preset sending interval.
Citation Information
Patent Citations
Human face eye region positioning method for fatigue driving detection
CN104361332A
Method for detecting fatigue driving and driving in abnormal posture under multiple postures
CN104688251A
Driving fatigue detection and early warning system and method based on vision
CN108791299A
Driver fatigue detection method and system based on computer vision
CN109934199A
Fatigue driving detection method and system based on facial feature fusion
CN110532887A
