Driver fatigue detection method and system

Abstract

The present invention provides a driver fatigue detection method and system, which respectively set a camera detection mode, a sensor monitoring mode and a joint monitoring mode to perform real-time monitoring of safe driving. When a user wears a smart wearable device, the smart wearable device is connected to the vehicle computer, and human feature data and portrait feature data are monitored simultaneously, so as to timely detect whether the user's driving status is fatigue driving and issue an alarm in time, thereby avoiding safety accidents caused by fatigue driving caused by involuntary or inertial operation during long-term driving.

Description

Technical Field

[0001] The present invention relates to the technical field of safe driving, and in particular to a driver fatigue detection method and system. Background Art

[0002] The number of cars is rapidly increasing, and traffic accidents are also on the rise. Driving fatigue accounts for approximately 40% of these accidents. Driving fatigue refers to the psychological and physiological dysfunction that occurs after prolonged driving, resulting in symptoms such as blurred vision, slow reaction times, clumsy movements, back pain, and decreased driving ability. Driving fatigue has become a significant factor in traffic accidents, posing a serious threat to life and property. Therefore, rapid, real-time, and effective detection of driving fatigue and issuing early warning signals to drivers can effectively reduce the probability of traffic accidents.

[0003] In the detection of driver fatigue, the main detection method is based on the driver's facial image features, such as facial expressions, changes in eye behavior, mouth status, etc.; among them, eye features are important features reflecting the fatigue state. After the driver enters the fatigue state, his blinking frequency decreases, and the time of closing eyes is significantly increased compared with the normal state. The time of opening eyes decreases accordingly, and the degree of eye openness also decreases to a certain extent. If the driver enters a deep fatigue state, the driver's eyes may be in a serious situation of being closed for a long time. Therefore, facial image features, especially eye features, can well reflect the driver's status. However, due to the increased differences in drivers' faces and the fact that this method is easily affected by light, the detection results are inaccurate, and misjudgment and missed judgments are prone to occur. Summary of the Invention

[0004] In response to the above technical problems, the present invention proposes a driver fatigue detection method and system, which aims to solve the technical problems in the existing technology of low driver fatigue detection accuracy and easy misjudgment.

[0005] The present invention provides a driver fatigue detection method, comprising the following steps:

[0006] S1: The user gets on the car; determine whether the user is wearing a smart wearable device. If so, go to S2; otherwise, start the camera detection mode;

[0007] S2: Determine whether the camera is authorized to record. If so, start the joint monitoring mode; otherwise, start the sensor monitoring mode.

[0008] Preferably, the joint monitoring mode is to fuse the monitoring results of the camera detection mode and the sensor monitoring mode.

[0009] The camera detection mode includes: monitoring the situation in the cockpit through the camera terminal, storing the captured video images in the image database, and analyzing the portrait feature data of the images in the image database.

[0010] The portrait feature data at least includes a facial image, pupil image data and a behavior video.

[0011] The user is judged to be fatigued driving according to the analysis result of the portrait feature data. If so, a fatigue alarm is issued to the user through the vehicle-mounted central control system. If not, the user continues driving.

[0012] The sensor monitoring mode includes: when a user wears a smart wearable device and gets in the car, the smart wearable device automatically or manually connects to the car computer and starts to monitor human body characteristic data in real time, and judges whether fatigue driving is in progress based on the configuration parameters of the fatigue driving reminder function. If so, a fatigue alarm is issued to the user through the on-board central control system; if not, the user continues driving.

[0013] The human body characteristic data at least includes blood pressure, heart rate and blood oxygen index.

[0014] The smart wearable device communicates with the vehicle computer via the Bluetooth protocol.

[0015] The fusion process includes:

[0016] If the camera detection mode and the sensor monitoring mode both detect fatigue driving, a fatigue alarm is issued to the user through the vehicle's central control system;

[0017] If the monitoring results of the camera detection mode and the sensor monitoring mode are both non-fatigue driving, the status quo will be maintained, monitoring will continue, and the driving time will be accumulated. When the driving time reaches the fatigue reminder time preset by the user, a fatigue alarm will be actively issued to the user through the on-board central control system.

[0018] The fusion process further includes:

[0019] If the monitoring results of the camera detection mode and the sensor monitoring mode are inconsistent, the portrait feature data and the body feature data are compared respectively to determine whether the feature difference between any two data is within the preset range. If so, it is determined that the data source is abnormal and the user is prompted to perform data initialization processing; if not, it is determined that the data has mutated and abnormal data needs to be eliminated.

[0020] As another preferred embodiment, the present invention further provides a driver fatigue detection system, comprising at least:

[0021] An acquisition module, used to acquire the driver's portrait feature data and body feature data within a predetermined time period;

[0022] a first fatigue determination module, configured to determine whether the driver is driving fatigued based on an analysis result of the driver's portrait feature data;

[0023] a second fatigue determination module, configured to determine whether the driver is fatigued based on the analysis result of the human body characteristic data;

[0024] a third fatigue determination module, configured to determine whether the driver is fatigued based on a combined analysis result of the portrait feature data and the body feature data;

[0025] The fatigue driving determination module is used to determine the current driver's status based on the first fatigue determination module, the second fatigue determination module, or the third fatigue determination module. If it is fatigue driving, a fatigue alarm is issued to the user through the vehicle's central control system; if not, the driver continues driving.

[0026] In summary, the present invention provides a driver fatigue detection method and system, which respectively set a camera detection mode, a sensor monitoring mode and a joint monitoring mode to perform real-time monitoring of safe driving. When the user wears a smart wearable device, the smart wearable device is connected to the vehicle computer, and human feature data and portrait feature data are monitored at the same time, so as to timely detect whether the user's driving status is fatigue driving, and issue an alarm in time, thereby avoiding safety accidents caused by fatigue driving due to involuntary or inertial operation when the user is driving for a long time. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a schematic diagram of a driver fatigue detection method according to the present invention.

[0028] Figure 2 This is a fatigue detection flow chart of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0030] like Figure 1 As shown, the present invention provides a driver fatigue detection method, comprising the following steps:

[0031] S1: After the user gets in the car, the built-in application of the vehicle's central control system is used to connect to the smart wearable device. It is also determined whether the user is wearing the smart wearable device. If so, the process goes to S2; otherwise, the camera detection mode is activated.

[0032] S2: Determine whether the camera is authorized to record. If so, start the joint monitoring mode; otherwise, start the sensor monitoring mode.

[0033] Preferably, the joint monitoring mode is to fuse the monitoring results of the camera detection mode and the sensor monitoring mode.

[0034] The camera detection mode includes: monitoring the situation in the cockpit through the camera terminal, storing the captured video images in the image database, and analyzing the portrait feature data of the images in the image database.

[0035] The portrait feature data at least includes a facial image, pupil image data and a behavior video.

[0036] The user is judged to be fatigued driving according to the analysis result of the portrait feature data. If so, a fatigue alarm is issued to the user through the vehicle-mounted central control system. If not, the user continues driving.

[0037] During driving, the user's fatigue status is determined by comparing normal facial images, pupil image data, and behavioral videos with real-time data. Because fatigued drivers are prone to intermittent head-down movements, this embodiment optionally uses facial images to capture head movements and determines the number of times the driver lowers their head within a predetermined timeframe. This number of head-down movements is then used to determine driver fatigue.

[0038] Furthermore, eye characteristics are crucial for reflecting fatigue. When a driver enters a fatigued state, their blink frequency decreases, and the amount of time they spend closed increases significantly compared to a normal state. This decreases the amount of time they spend open, and the degree of openness also decreases to a certain extent. In a state of deep fatigue, the driver's eyes may remain closed for extended periods, leading to serious consequences. Identifying the driver's pupil condition can provide a good indicator of their state.

[0039] The sensor monitoring mode includes: when a user wears a smart wearable device and gets in the car, the smart wearable device automatically or manually connects to the car computer and starts to monitor human body characteristic data in real time, and judges whether fatigue driving is in progress based on the configuration parameters of the fatigue driving reminder function. If so, a fatigue alarm is issued to the user through the on-board central control system; if not, the user continues driving.

[0040] The human body characteristic data at least includes blood pressure, heart rate and blood oxygen index.

[0041] The smart wearable device communicates with the vehicle computer via the Bluetooth protocol. The smart wearable device is preferably a smart headband, but is not limited thereto.

[0042] The fusion process includes:

[0043] If the camera detection mode and the sensor monitoring mode both detect fatigue driving, a fatigue alarm is issued to the user through the vehicle's central control system;

[0044] If the monitoring results of the camera detection mode and the sensor monitoring mode are both non-fatigue driving, the status quo will be maintained, monitoring will continue, and the driving time will be accumulated. When the driving time reaches the fatigue reminder time preset by the user, a fatigue alarm will be actively issued to the user through the on-board central control system.

[0045] The fusion process further includes:

[0046] If the monitoring results of the camera detection mode and the sensor monitoring mode are inconsistent, the portrait feature data and the body feature data are compared respectively to determine whether the feature difference between any two data is within the preset range. If so, it is determined that the data source is abnormal and the user is prompted to perform data initialization processing; if not, it is determined that the data has mutated and abnormal data needs to be eliminated.

[0047] As another preferred embodiment, the present invention further provides a driver fatigue detection system, comprising at least:

[0048] An acquisition module, used to acquire the driver's portrait feature data and body feature data within a predetermined time period;

[0049] a first fatigue determination module, configured to determine whether the driver is driving fatigued based on an analysis result of the driver's portrait feature data;

[0050] a second fatigue determination module, configured to determine whether the driver is fatigued based on the analysis result of the human body characteristic data;

[0051] a third fatigue determination module, configured to determine whether the driver is fatigued based on a combined analysis result of the portrait feature data and the body feature data;

[0052] The fatigue driving determination module is used to determine the current driver's status based on the first fatigue determination module, the second fatigue determination module, or the third fatigue determination module. If it is fatigue driving, a fatigue alarm is issued to the user through the vehicle's central control system; if not, the driver continues driving.

[0053] In a preferred embodiment of the present application, the driver fatigue detection method is to realize real-time monitoring of fatigue driving through joint monitoring of the vehicle terminal and the wearable device. In order to realize the method described in the present invention, the corresponding system described in the present invention, preferably, the system also includes a processor, a memory, an input device, and an output device.

[0054] The processor, memory, input device, and output device are coupled via a connector, which may include various interfaces, transmission lines, or buses, etc., which are not limited in the embodiments of this application. It should be understood that in various embodiments of this application, coupling refers to mutual connection in a specific manner, including direct connection or indirect connection through other devices, such as connection through various interfaces, transmission lines, buses, etc.

[0055] The processor may be one or more graphics processing units (GPUs). If the processor is a GPU, the GPU may be a single-core GPU or a multi-core GPU. Alternatively, the processor may be a processor group consisting of multiple GPUs, with the multiple processors coupled to each other via one or more buses. Alternatively, the processor may be other types of processors, etc., which are not limited in the embodiments of the present application.

[0056] The memory may be used to store computer program instructions and various computer program codes, including the program code for executing the solution of the present application. Optionally, the memory includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD ROM), and is used for related instructions and data.

[0057] The input device is used to input data and / or signals, and the output device is used to output data and / or signals. The input device and the output device can be independent devices or an integrated device.

[0058] It can be understood that in the embodiments of the present application, the memory can be used not only to store relevant instructions, but also to store relevant data. For example, the memory can be used to store data obtained through an input device, or the memory can also be used to store comparison results obtained by a processor, etc. The embodiments of the present application do not limit the specific data stored in the memory.

[0059] It can be understood that in actual applications, the driver fatigue detection device can also include other necessary components, including but not limited to any number of input / output devices, processors, memories, etc., and all driver fatigue detection devices that can implement the embodiments of this application are within the scope of protection of this application.

[0060] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0061] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here. Those skilled in the art will also clearly understand that the descriptions of the various embodiments of this application have different focuses. For the convenience and brevity of description, the same or similar parts may not be repeated in different embodiments. Therefore, for parts not described or not described in detail in a certain embodiment, reference can be made to the descriptions of other embodiments.

[0062] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A driver fatigue detection method, characterized in that: The following steps are involved: S1: The user gets on the car; determine whether the user is wearing a smart wearable device. If so, go to S2; otherwise, start the camera detection mode; S2: Determine whether the camera is authorized to record. If so, start the joint monitoring mode; otherwise, start the sensor monitoring mode; The combined monitoring mode is to fuse the monitoring results of the camera detection mode and the sensor monitoring mode; The camera detection mode includes: monitoring the cockpit situation through the camera terminal, storing the captured video images in the image database, and analyzing the human feature data of the images in the image database; The portrait feature data at least includes a facial image, pupil image data and a behavior video; The sensor monitoring mode includes: when a user wears a smart wearable device and enters a vehicle, the smart wearable device automatically or manually connects to the vehicle computer and starts real-time monitoring of human body characteristic data. According to the configuration parameters of the fatigue driving reminder function, it is determined whether the user is driving fatigued. If so, a fatigue alarm is issued to the user through the vehicle central control system. If not, the user continues driving. The fusion process includes: If the camera detection mode and the sensor monitoring mode both detect fatigue driving, a fatigue alarm is issued to the user through the vehicle's central control system; If the camera detection mode and the sensor monitoring mode both indicate non-fatigue driving, the status quo is maintained, monitoring continues, and the driving time is accumulated. When the driving time reaches the fatigue reminder time preset by the user, a fatigue alarm is actively issued to the user through the vehicle's central control system; The fusion process further includes: If the monitoring results of the camera detection mode and the sensor monitoring mode are inconsistent, the portrait feature data and the body feature data are compared respectively to determine whether the feature difference between any two data is within the preset range. If so, it is determined that the data source is abnormal and the user is prompted to perform data initialization processing; if not, it is determined that the data has mutated and abnormal data needs to be eliminated.

2. A driver fatigue detection method according to claim 1, characterized in that: The user is judged to be fatigued driving according to the analysis result of the portrait feature data. If so, a fatigue alarm is issued to the user through the vehicle-mounted central control system. If not, the user continues driving.

3. A driver fatigue detection method according to claim 1, characterized in that: The human body characteristic data at least includes blood pressure, heart rate and blood oxygen index.

4. A driver fatigue detection method according to claim 3, characterized in that: The smart wearable device communicates with the vehicle computer via the Bluetooth protocol.

5. A driver fatigue detection system, characterized in that: At least: An acquisition module, used to acquire the driver's portrait feature data and body feature data within a predetermined time period; a first fatigue determination module, configured to determine whether the driver is driving fatigued based on an analysis result of the driver's portrait feature data; a second fatigue determination module, configured to determine whether the driver is fatigued based on the analysis result of the human body characteristic data; a third fatigue determination module, configured to determine whether the driver is fatigued based on a combined analysis result of the portrait feature data and the body feature data; a fatigue driving determination module, configured to determine the current driver's state based on the first fatigue determination module, the second fatigue determination module, or the third fatigue determination module, and if the driver is fatigue driving, issue a fatigue alarm to the user through the vehicle's central control system; if not, continue driving; The driver fatigue detection system adopts the driver fatigue detection method as described in any one of claims 1-4.

Images