Visual fatigue testing and analyzing device for driver

Abstract

The invention discloses a driver visual fatigue test analysis device, which comprises a visual detector used for carrying out real-time video image data acquisition on a driver and sending the acquired real-time video image data to a controller; the controller is used for detecting and capturing eye features of the driver in the real-time video image data, performing fatigue detection based on machine vision according to the collected eye features in the real-time video image data, and sending an alarm instruction to the alarm to give an alarm to the driver; the millimeter wave detector is used for sending millimeter wave monitoring to obtain real-time heart rate data of a driver and sending the real-time heart rate data to the controller, and the controller detects whether fatigue driving exists in the driver or not according to the real-time heart rate data and sends an alarm instruction to the alarm to give an alarm to the driver; the driver fatigue test analysis device is used for solving the technical problem that an existing driver fatigue test analysis device is prone to being affected by various factors, so that fatigue detection analysis of the device is prone to failure.

Description

Technical Field

[0001] The present application relates to the technical field of driver fatigue testing, and in particular to a driver visual fatigue testing and analysis device. Background Art

[0002] Driving fatigue is the main cause of traffic accidents. The casualties and property losses caused by this accident worldwide each year are difficult to estimate. Huge amounts of manpower and financial resources have been invested both at home and abroad to study the monitoring of fatigue driving.

[0003] However, the current devices for detecting driver fatigue have the following three main shortcomings: first, if the driver wears wide-rimmed glasses, the camera of the fatigue detection device based on machine vision can easily fail to illuminate the eyes, resulting in fatigue detection failure; second, when in an environment with strong external light sources, the image of the camera of the fatigue detection device based on machine vision is easily overexposed, resulting in fatigue detection failure; third, due to individual differences in drivers, small eyes, etc., fatigue detection failure will occur. Therefore, a driver visual fatigue test and analysis device is urgently needed to solve the above problems. Summary of the invention

[0004] An embodiment of the present application provides a driver visual fatigue test and analysis device to solve the technical problem that the existing driver fatigue test and analysis device is easily affected by various factors such as the driver wearing glasses, excessive external light sources and small eyes of the driver, which makes the driver fatigue test and analysis device prone to fatigue detection and analysis failure.

[0005] In view of this, the present application provides a driver visual fatigue test and analysis device, including a visual detector, a millimeter wave detector, a controller and an alarm, wherein the millimeter wave detector, the visual detector and the alarm are all electrically connected to the controller; the visual detector is used to collect real-time video image data of the driver and send the collected real-time video image data to the controller; the controller is used to detect and capture the eye features of the driver in the real-time video image data, and determine whether the eye features are complete. If the eye features are complete, fatigue based on machine vision is performed according to the eye features in the collected real-time video image data. Detection, detecting whether the driver is driving fatigued. If the driver is found to be driving fatigued, an alarm command is issued to the alarm, and the alarm receives the alarm command and alarms the driver; if the eye feature is incomplete, the controller issues a command to control the millimeter wave detector to start working; the millimeter wave detector is used to emit millimeter waves to monitor and obtain the real-time heart rate data of the driver, and send the real-time heart rate data to the controller. The controller detects whether the driver is driving fatigued through the real-time heart rate data. If the driver is driving fatigued, an alarm command is issued to the alarm, and the alarm receives the alarm command and alarms the driver.

[0006] Optionally, the visual detector includes an optical camera module with a near-infrared light emitter, the optical camera module is used to support image and video acquisition during the day and at night, the optical camera module is installed at the position of the car dashboard, and the optical camera module is tilted upward to perform real-time video image acquisition of the driver's eye features.

[0007] Optionally, the millimeter wave detector includes a millimeter wave radar installed in an A-pillar or a seat back of the automobile.

[0008] Optionally, when the controller performs fatigue detection based on machine vision according to the eye features in the collected real-time video image data, the following steps are included: S401, first perform face detection and obtain the driver's facial image; S402, then perform face key point detection on the facial image to obtain the eye features in the facial image; S403, then obtain the real-time pupil closure percentage by collecting the maximum value of the pupil size in the eye features; S404, finally determine whether the driver is driving fatigued by calculating whether the real-time pupil closure percentage of the driver in unit time exceeds a preset threshold of the pupil closure percentage time.

[0009] Optionally, the controller detects whether the driver is driving fatigued through the real-time heart rate data, including the following steps: S501, obtaining heart rate variability power spectrum data by sorting the real-time heart rate data; S502, filtering and calculating the heart rate variability power spectrum data to obtain the low-frequency mean, the high-frequency mean and the ratio of the low-frequency mean to the high-frequency mean; S503, judging that the driver is driving fatigued based on the increase of the low-frequency mean, the decrease of the high-frequency mean and the increase of the ratio of the low-frequency mean to the high-frequency mean reaching the set threshold range.

[0010] Optionally, the real-time video image data includes lip video image data, and the controller calculates the yawning frequency of the driver based on the lip video image data, and compares the yawning frequency with a set yawning frequency threshold. When the yawning frequency exceeds the yawning threshold, it is determined that the driver is driving fatigued, and an alarm command is sent to the alarm. The alarm receives the alarm command and alarms the driver.

[0011] Optionally, the controller is provided with a memory, a processor, and a program stored in the memory and executable on the processor.

[0012] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0013] The present application provides a driver visual fatigue testing and analysis device. Compared with the prior art, the present invention collects real-time video image data of the driver through the visual detector, and sends the collected real-time video image data to the controller; the controller detects and captures the eye features of the driver in the real-time video image data, and determines whether the eye features are complete. If the eye features are complete, fatigue detection based on machine vision is performed according to the eye features in the collected real-time video image data to detect whether the driver is driving fatigued. If the driver is found to be driving fatigued, an alarm instruction is issued to the alarm, and the alarm receives the alarm instruction and alarms the driver; if the eye features are incomplete , the controller sends out an instruction to control the millimeter wave detector to start working; the millimeter wave detector sends out millimeter waves to monitor and obtain the real-time heart rate data of the driver, and sends the real-time heart rate data to the controller, the controller detects whether the driver is driving fatigued through the real-time heart rate data, and if so, sends an alarm instruction to the alarm, the alarm receives the alarm instruction and alarms the driver, thereby realizing that the driver visual fatigue test and analysis device has a stronger ability to overcome various unfavorable conditions such as the driver wearing glasses, the external light source is too strong, and the driver has small eyes. The accuracy of fatigue driving detection is further improved, and it is not affected by weather, suitable for all-weather applications, and suitable for all drivers. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly express the technical solutions of the embodiments of the present application, the drawings required for describing the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0015] Figure 1 This is a working block diagram of a driver visual fatigue testing and analysis device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0016] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0017] For easier understanding, see Figure 1An embodiment of a driver visual fatigue testing and analysis device provided by the present application includes a visual detector, a millimeter wave detector, a controller and an alarm, wherein the millimeter wave detector, the visual detector and the alarm are all electrically connected to the controller; the visual detector is used to collect real-time video image data of the driver and send the collected real-time video image data to the controller; the controller is used to detect and capture the driver's eye features in the real-time video image data, that is, select the driver's eye features in the video image data, and determine whether the eye features are complete. If the eye features are complete, fatigue detection based on machine vision is performed according to the eye features in the collected real-time video image data to detect whether the driver is fatigued driving. If the driver is found to be If the driver is driving fatigued, an alarm command is sent to the alarm, and the alarm receives the alarm command and warns the driver; if the eye feature is incomplete, the controller sends a command to control the millimeter wave detector to start working; the millimeter wave detector is used to send millimeter waves to monitor and obtain the driver's real-time heart rate data, and send the real-time heart rate data to the controller. The controller detects whether the driver is driving fatigued through the real-time heart rate data. If so, an alarm command is sent to the alarm, and the alarm receives the alarm command and warns the driver; through the above settings, when the driver drives the vehicle, the device automatically starts running, the visual detector collects real-time video image data of the driver, and sends the collected real-time video image data to the controller The controller receives the real-time video image data of the driver, detects and captures the eye feature data of the driver in the real-time video image data, and determines whether the eye features are complete, that is, the eye features are not blocked by glasses or other obstructions, etc., and the visual detector can fully detect the driver's eyes. If the eye features are complete, fatigue detection based on machine vision is performed according to the eye features in the collected real-time video image data to detect whether the driver is fatigued. If the driver is found to be fatigued, an alarm command is sent to the alarm. After receiving the alarm command, the alarm reminds the driver to avoid accidents caused by fatigue driving; if the driver wears glasses, the external light source is too strong, or the driver has small eyes, etc., If the eye features collected by the visual detector are incomplete, the controller sends a command to control the millimeter wave detector to start working. The millimeter wave detector sends out millimeter waves to monitor and obtain the driver's real-time heart rate data, and sends the real-time heart rate data to the controller. The controller detects whether the driver is driving fatigued through the real-time heart rate data. If so, an alarm command is sent to the alarm. The alarm receives the alarm command and alerts the driver, thereby realizing that the driver's visual fatigue test and analysis device has a stronger ability to overcome various unfavorable conditions such as the driver wearing glasses, the external light source is too strong, and the driver has small eyes. The accuracy of fatigue driving detection is further improved, and it is not affected by weather, suitable for all-weather applications, and suitable for all drivers.

[0018] As a further solution of the present invention, the visual detector includes an optical camera module with a near-infrared light emitter, and the optical camera module is used to support image and video acquisition during the day and at night, that is, the optical camera module can acquire video images of the driver's eye features during the day and at night. The optical camera module is installed at the position of the car dashboard, and the optical camera module is tilted upward to facilitate real-time video image acquisition of the driver's eye features, thereby improving the accuracy of the driver's visual fatigue test and analysis device in detecting driver fatigue.

[0019] Furthermore, the millimeter wave detector includes a millimeter wave radar, which is installed in the A-pillar or seat back of the car to facilitate the emission of millimeter waves to collect real-time heart rate data of the driver, further improving the accuracy of the driver's fatigue detection by the driver's visual fatigue test and analysis device.

[0020] Furthermore, when the controller performs fatigue detection based on machine vision according to the eye features in the collected real-time video image data, the controller includes the following steps: S401, first perform face detection through a visual detector and obtain the driver's facial image; S402, then perform facial key point detection and extraction on the facial image to obtain the eye features in the facial image; S403, then collect the maximum value of the pupil size in the eye features, and calculate the real-time pupil closure percentage based on the maximum value; S404, finally determine whether the driver is driving fatigued by calculating the time when the driver's real-time pupil closure percentage per unit time exceeds the threshold of the preset pupil closure percentage, thereby improving the accuracy of the driver's fatigue driving detection by the driver's visual fatigue test and analysis device.

[0021] Specifically, when the driver is in a cloudy scene, the light intensity is appropriate, and the driver is not wearing glasses, the optical camera module in the visual detector is used to collect video images of the driver, detect whether the driver's eye features exist in the video image data, and detect whether the eye features are complete. When the driver's eye features are detected and the eye features are complete, fatigue detection based on machine vision is performed according to the eye features in the collected video image data. The maximum value of the pupil size in the eye features is collected, and the real-time pupil closure percentage is calculated based on the maximum value. The time when the real-time pupil closure percentage of the driver per unit time exceeds the threshold of the preset pupil closure percentage time. When the real-time pupil closure percentage per unit time exceeds the threshold of the preset pupil closure percentage time, the driver is judged to be fatigued driving, and an alarm is issued to the driver, thereby improving the accuracy of the driver's visual fatigue testing and analysis device in detecting fatigue driving of the driver.

[0022] Furthermore, the controller detects whether the driver is driving fatigued through real-time heart rate data, including the following steps: S501, sorting the driver's real-time heart rate data through a millimeter wave detector to obtain heart rate variability power spectrum data; S502, filtering and calculating the heart rate variability power spectrum data to obtain the low-frequency mean, the high-frequency mean, and the ratio of the low-frequency mean to the high-frequency mean; S503, judging that the driver is driving fatigued based on the increase in the low-frequency mean, the decrease in the high-frequency mean, and the increase in the ratio of the low-frequency mean to the high-frequency mean, all of which reach the set threshold range.

[0023] Specifically, when the driver wears glasses, the frames of the glasses will block the eyes, and if the driver is too tall or too short, the optical camera module will not be able to collect the complete features of the glasses. In addition, the light intensity is strong and shines into the cab towards the driver. The strong light source brings a large amount of infrared rays. Excessive infrared light energy causes the collected image to be overexposed, and the optical camera module cannot collect the driver's eye features. In this case, the millimeter-wave radar that controls the millimeter-wave detector is automatically started to emit millimeter waves to detect the driver's heart rate, and the heart rate data is used to detect whether the driver is fatigued. The heart rate variability power spectrum data is obtained through the heart rate data and filtered to obtain the low-frequency mean, high-frequency mean, and the ratio of the low-frequency mean to the high-frequency mean. If the low-frequency mean rises, the high-frequency mean decreases, and the ratio of the low-frequency mean to the high-frequency mean increases to reach the set threshold range, it is determined that the driver is fatigued, and an alarm is issued to the driver, thereby improving the accuracy and applicability of the driver's visual fatigue testing and analysis device for detecting driver fatigue driving.

[0024] Furthermore, the real-time video image data includes lip video image data. The controller calculates the driver's yawning frequency based on the real-time lip video image data, and compares the real-time yawning frequency with the set yawning frequency threshold. When the real-time yawning frequency exceeds the yawning threshold, it is determined that the driver is driving fatigued, and an alarm command is sent to the alarm. The alarm receives the alarm command and alarms the driver, thereby further improving the accuracy of the driver visual fatigue test and analysis device in detecting driver fatigue driving.

[0025] Furthermore, the controller is provided with a memory, a processor, and a program stored in the memory and executable on the processor, which are used to improve the controller's performance in storing and processing data and ensure the normal operation of the driver's visual fatigue testing and analysis device.

[0026] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A driver visual fatigue test and analysis device, characterized in that: It includes a visual detector, a millimeter wave detector, a controller and an alarm, wherein the millimeter wave detector, the visual detector and the alarm are all electrically connected to the controller; The visual detector is used to collect real-time video image data of the driver and send the collected real-time video image data to the controller; The controller is used to detect and capture the eye features of the driver in the real-time video image data, and determine whether the eye features are complete. If the eye features are complete, fatigue detection based on machine vision is performed according to the eye features in the real-time video image data collected to detect whether the driver is driving fatigued. If it is found that the driver is driving fatigued, an alarm instruction is sent to the alarm device, and the alarm device receives the alarm instruction and alarms the driver; If the eye feature is incomplete, the controller sends an instruction to control the millimeter wave detector to start working; The millimeter wave detector is used to emit millimeter waves to monitor and obtain the real-time heart rate data of the driver, and send the real-time heart rate data to the controller. The controller detects whether the driver is driving fatigued through the real-time heart rate data. If so, an alarm instruction is sent to the alarm. The alarm receives the alarm instruction and alarms the driver.

2. The driver fatigue monitoring device based on machine vision according to claim 1, characterized in that: The visual detector includes an optical camera module with a near-infrared light emitter, and the optical camera module is used to support image and video acquisition during the day and at night. The optical camera module is installed at the position of the car dashboard, and the optical camera module is tilted upward to perform real-time video image acquisition of the driver's eye features.

3. The driver fatigue monitoring device based on machine vision according to claim 1 is characterized in that: The millimeter wave detector includes a millimeter wave radar installed in an A-pillar or a seat back of the automobile.

4. The driver fatigue monitoring device based on machine vision according to claim 1 is characterized in that: When the controller performs fatigue detection based on machine vision according to the eye features in the collected real-time video image data, the controller includes the following steps: S401, performing face detection and obtaining a facial image of the driver; S402, performing facial key point detection on the facial image to obtain eye features in the facial image; S403, obtaining a real-time pupil closure percentage by collecting the maximum value of the pupil size in the eye features; S404: Determine whether the driver is driving fatigued by calculating whether the real-time pupil closure percentage of the driver in a unit time exceeds a preset threshold of the pupil closure percentage.

5. The driver fatigue monitoring device based on machine vision according to claim 1, characterized in that: The controller detects whether the driver is driving fatigued by using the real-time heart rate data, including the following steps: S501, obtaining heart rate variability power spectrum data by sorting the real-time heart rate data; S502, filtering and calculating the heart rate variability power spectrum data to obtain a low-frequency mean, a high-frequency mean, and a ratio of the low-frequency mean to the high-frequency mean; S503: If the low-frequency mean value increases, the high-frequency mean value decreases, and the ratio of the low-frequency mean value to the high-frequency mean value increases and all reach a set threshold range, it is determined that the driver is driving fatigued.

6. The driver fatigue monitoring device based on machine vision according to claim 1 is characterized in that: The real-time video image data includes lip video image data. The controller calculates the yawning frequency of the driver based on the lip video image data, and compares the yawning frequency with a set yawning frequency threshold. When the yawning frequency exceeds the yawning threshold, it is determined that the driver is driving fatigued, and an alarm command is sent to the alarm. The alarm receives the alarm command and alarms the driver.

7. The driver fatigue monitoring device based on machine vision according to claim 1 is characterized in that: The controller is provided with a memory, a processor, and a program stored in the memory and executable on the processor.

Images