System for preventing fatigue of driver through active intervention

Abstract

The invention discloses a system for preventing fatigue of a driver through active intervention, and relates to the technical field of vehicle safety, the system comprises a vehicle-mounted terminal, and the vehicle-mounted terminal comprises a fatigue monitoring module, an active intervention module and an emergency physical intervention module; the fatigue monitoring module is used for evaluating the fatigue state of the driver according to the face image and / or physiological information of the driver to obtain the fatigue state degree of the driver; the fatigue state degree comprises a shallow fatigue state and a deep fatigue state; when the fatigue state degree of the driver is a shallow fatigue state, the active intervention module starts a man-machine conversation function, so that the driver performs voice conversation; when the fatigue state degree of the driver is a deep fatigue state, the emergency physical intervention module awakens the driver through a smell eruption device, a sound alarm device and a touch vibration device, effective intervention is conducted for the situation that parking and resting are inconvenient in the fatigue state, and the fatigue state of the driver is relieved.

Description

Technical Field

[0001] The present application relates to the field of vehicle safety technology, and in particular to an active intervention system for preventing driver fatigue. Background Art

[0002] Driver fatigue driving is one of the important causes of traffic accidents. At present, the existing technology only alarms and reports to the backend for fatigue driving and bad driving habits, but there is no effective means of intervention when it is inconvenient to stop and rest due to fatigue. There are no targeted measures to improve the driver's fatigue state. For example, it takes a certain distance and time for the driver to reach the rest point, and continuing to drive in a fatigued state poses a great safety hazard. Summary of the invention

[0003] The purpose of this application is to provide an active intervention system to prevent driver fatigue, which can effectively intervene in situations where it is inconvenient to stop and rest due to the fatigue state of the driver, relieve the driver's fatigue state, and improve driving safety.

[0004] To achieve the above objectives, this application provides the following solutions:

[0005] The present application provides an active intervention system for preventing driver fatigue, including an on-board terminal, wherein the on-board terminal includes a fatigue monitoring module, an active intervention module, and an emergency physical intervention module;

[0006] The fatigue monitoring module is used to: evaluate the fatigue state of the driver according to the driver's facial image and / or physiological information to obtain the degree of the driver's fatigue state; the degree of the fatigue state includes a shallow fatigue state and a deep fatigue state; output a first intervention instruction when the driver's fatigue state is a shallow fatigue state; output a second intervention instruction when the driver's fatigue state is a deep fatigue state;

[0007] The active intervention module is used to: after receiving the first intervention instruction, start the human-machine dialogue function to enable the driver to conduct a voice dialogue;

[0008] The emergency physical intervention module is used to wake up the driver through a scent spraying device, a sound alarm device and a tactile vibration device after receiving a second intervention instruction.

[0009] Optionally, the active intervention module further includes a voice chat unit; the voice chat unit is used to enable the driver, relatives, friends or vehicle transportation companies to have a voice conversation through an instant voice chat room.

[0010] Optionally, the active intervention module also includes a personalized warning unit; the personalized warning unit is used to: play a personalized warning sound to the driver; the personalized warning sound is a safety reminder recording using the voice of the driver's friends or relatives.

[0011] Optionally, the odor spraying device is used to release irritating odors; the sound alarm device is used to sound an alarm through a speaker or a vehicle audio system; the tactile vibration device, which is a steering wheel tactile vibration device or a seat cushion tactile vibration device, is used to wake up the driver by vibration.

[0012] Optionally, the active intervention to prevent driver fatigue system also includes an APP client; the APP client is used to view vehicle information, vehicle terminal settings and vehicle terminal calls in real time.

[0013] Optionally, the active intervention to prevent driver fatigue system also includes a server; the server is used to provide data connection services and data processing services.

[0014] Optionally, the vehicle-mounted terminal further includes an image acquisition device; the image acquisition device is used to acquire a facial image of the driver.

[0015] Optionally, the vehicle-mounted terminal further includes a physiological information acquisition device; the physiological information acquisition device is used to collect the driver's physiological information; the physiological information includes heart rate, skin electricity, myoelectricity and blood oxygen.

[0016] Optionally, the active intervention system for preventing driver fatigue also includes a monitoring center; the monitoring center is used to monitor the vehicle driving status in real time.

[0017] Optionally, the vehicle-mounted terminal further includes a communication module, and the vehicle-mounted terminal is connected to the server, the monitoring center and the APP client through the communication module.

[0018] According to the specific embodiments provided by the present application, the present application discloses the following technical effects: the present application provides an active intervention system to prevent driver fatigue. When the driver is in a fatigued state or may be in a fatigued state, the system actively intervenes and prevents the driving fatigue state that has occurred or may occur through human-computer dialogue or providing the driver with a customized instant voice chat room, thereby improving the driver's brain activity and preventing the driver from falling into a fatigued driving state; when it is monitored that the driver is in a deep fatigue state and voice dialogue intervention cannot effectively prevent driving fatigue, the driver is urgently physically stimulated to wake up through an odor spraying device, a sound alarm device and a tactile vibration device to prevent traffic accidents and improve transportation efficiency. For situations where it is inconvenient to stop and rest in a fatigued state, the above-mentioned means are used to effectively intervene, alleviate the driver's fatigue state and improve driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A structural diagram of an active intervention system for preventing driver fatigue provided in one embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0023] In an exemplary embodiment, Figure 1 As shown, a system for actively intervening to prevent driver fatigue is provided, including an on-board terminal, wherein the on-board terminal includes a fatigue monitoring module, an active intervention module and an emergency physical intervention module.

[0024] The fatigue monitoring module is used to: evaluate the driver's fatigue state based on the driver's facial image and / or physiological information to obtain the driver's fatigue state degree; the fatigue state degree includes a shallow fatigue state and a deep fatigue state; when the driver's fatigue state degree is a shallow fatigue state, output a first intervention instruction; when the driver's fatigue state degree is a deep fatigue state, output a second intervention instruction.

[0025] The active intervention module is used to start the human-machine dialogue function after receiving the first intervention instruction, so that the driver can have a voice dialogue.

[0026] The emergency physical intervention module is used to wake up the driver through a scent spraying device, a sound alarm device and a tactile vibration device after receiving a second intervention instruction.

[0027] The vehicle-mounted terminal also includes a video module, a video processing module, a positioning module, a Bluetooth module, an audio module, a data storage and communication module.

[0028] Among them, the video module contains two cameras, one is an in-car camera, which mainly monitors the driver's driving behavior; and the other is an outside-car camera, which mainly monitors the driving and driving records in front of the vehicle.

[0029] The vehicle terminal also includes a video processing module for processing video data from the in-vehicle camera and the vehicle driving record camera to determine the driver's fatigue condition, bad driving behavior, and the vehicle's driving trajectory and driving record function.

[0030] The positioning module is used to locate the vehicle and monitor the speed through GPS or Beidou positioning system.

[0031] The Bluetooth module can provide control for reminder devices such as odor / vibration through wireless connection, and can also enable users to communicate with the vehicle terminal through, for example, Bluetooth headsets, physiological information collection devices, etc.

[0032] The audio module can provide users with voice input and output, and complete voice-to-text (text-to-voice) conversion through audio processing software to achieve call or human-computer dialogue functions.

[0033] The data storage module is used to store necessary data, for example, important data such as driving recorders are stored locally.

[0034] 1) Fatigue monitoring module:

[0035] The vehicle terminal also includes an image acquisition device; the image acquisition device is the in-vehicle camera in the video module. The image acquisition device is used to acquire the driver's facial image. The vehicle terminal also includes a physiological information acquisition device; the physiological information acquisition device is used to acquire the driver's physiological information; the physiological information includes heart rate, skin electricity, myoelectricity and blood oxygen.

[0036] The fatigue monitoring module uses information collected by high-precision cameras or physiological information acquisition devices (heart rate, electrodermal conductivity, electromyography, blood oxygen, etc.), and uses a trained deep learning model to process facial images and / or physiological information, identify signs of fatigue, and evaluate the degree of fatigue to obtain the driver's fatigue status.

[0037] The fatigue monitoring module mainly uses AI visual models to detect fatigue status, and can also add physiological information equipment to collect physiological information data for more accurate fatigue detection. The trained deep learning models available are: 68-point face detection model in the Dlib library; MTCNN deep learning face detection model; pre-trained Haar cascade classifier and DNN model in the OpenCV library.

[0038] 2) Active intervention module:

[0039] The active intervention module can help the driver maintain driving alertness and brain activity for a long time, effectively prevent the driver from entering a state of driving fatigue, and improve the driver's transportation efficiency.

[0040] When the system detects signs of fatigue or possible fatigue (driving for a certain period of time), the human-computer dialogue function is activated through the on-board terminal system, and the driver is guided by voice to complete a series of cognitive tasks to improve his or her attention and brain activity.

[0041] The active intervention module also includes a voice chat unit; the voice chat unit is used to enable the driver, relatives, friends or vehicle transportation companies to have voice conversations through an instant voice chat room. The driver or vehicle transportation company (group) can establish a customized online instant voice chat room through this system, allowing the driver to complete the voice conversation, helping the driver to improve his concentration and brain activity.

[0042] The active intervention module also includes a personalized warning unit; the personalized warning unit is used to play a personalized warning sound to the driver; the personalized warning sound is a safety reminder recording using the voice of the driver's friends or relatives. The system has the function of recording and playing personalized warning sounds (such as safety reminder recordings of friends and relatives). When the system detects fatigue, it plays a personalized warning sound to the driver to help the driver improve alertness and brain activity.

[0043] When the active intervention module conducts voice dialogue, the voice-to-text conversion and text-to-voice conversion are realized through the audio module.

[0044] 2) Emergency physical intervention module:

[0045] When the driver is in a state of deep fatigue and human-computer dialogue and voice chat room can no longer effectively improve the driver's attention and brain activity, that is, when the driver's fatigue state is in a state of deep fatigue, the system activates the emergency physical intervention module according to the situation, and uses olfactory stimulation, sound stimulation, tactile vibration and other methods to quickly wake up the driver.

[0046] When in a state of deep fatigue, the brain's information processing capacity has been sharply reduced. At this time, human-computer dialogue and voice chat rooms have the risk of distracting attention and further reducing activity. In addition, relevant research results have shown that when in a state of deep fatigue, the stimulation of human-computer dialogue and voice chat rooms has limited effect on the recovery of the driver's physiological state.

[0047] The odor spraying device is used to release irritating odors, such as mint or lemon, to stimulate the driver's sense of smell; the sound alarm device is used to issue an alarm through a speaker or a car audio system to stimulate the driver's hearing; the tactile vibration device, which is a steering wheel tactile vibration device or a seat cushion tactile vibration device, is used to wake up the driver by vibration.

[0048] The emergency physical intervention module can effectively wake up the driver in an emergency when the driver falls into a state of deep driving fatigue after driving for a long time and the measures of active intervention modules such as human-computer dialogue and chatting have no effect on the driver, thereby effectively reducing the probability of traffic accidents caused by deep driving fatigue.

[0049] The active intervention system to prevent driver fatigue also includes an APP client; the APP client is used to view vehicle information, vehicle terminal settings, and vehicle terminal calls in real time. The APP client has functions such as system personalization settings or custom chat rooms, which can be set through the APP client. The APP client has the ability to communicate with the vehicle client. The driver's relatives and friends can enter the voice chat room through the APP client to interact with the driver by voice, or view the driver's vehicle location and other vehicle information according to the permission settings. Users (drivers, relatives and friends or fleet managers) can use the APP client to grasp vehicle information in real time and realize functions such as vehicle information viewing, vehicle terminal settings, and vehicle terminal calls. The APP client can be a device such as a mobile phone or a tablet computer.

[0050] The active intervention system for preventing driver fatigue also includes a monitoring center; the monitoring center is used to monitor the vehicle driving status in real time. The monitoring center can realize real-time monitoring of functions such as vehicle command, accident alarm, and vehicle monitoring through a server, and realize the management and deployment of the fleet or drivers.

[0051] The active intervention system for preventing driver fatigue also includes a server; the server is used to provide data connection services and data processing services for the vehicle terminal, the APP client and the monitoring center. The server has an application program interface, and the application program interface has an API access function for network information services, information exchange services for each terminal, and a server function for dialogue (chat) functions.

[0052] Among them, the application programming interface (API access) can provide the system with information sources through API access from traffic information platforms, meteorological information platforms, AI language model providers or other network consulting platforms, enabling the system to provide network information services for vehicle-mounted terminals.

[0053] The vehicle-mounted terminal also includes a communication module, through which the vehicle-mounted terminal is connected to a server, a monitoring center and an APP client based on an operator's network.

[0054] This application is a system that uses computer vision and multimodal active intervention to prevent driver fatigue. By comprehensively using computer technology, human-computer interaction, human-human interaction and physical stimulation measures, it can effectively monitor and prevent driving fatigue in real time, intervene in deep fatigue states in a timely manner, and significantly improve the safety and efficiency of transportation. The system is suitable for various vehicles or embedded in other fatigue monitoring systems, especially for long-distance transportation and professional driving occasions.

[0055] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0056] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. An active intervention system to prevent driver fatigue, characterized in that: The active intervention system for preventing driver fatigue includes a vehicle-mounted terminal, which includes a fatigue monitoring module, an active intervention module and an emergency physical intervention module; The fatigue monitoring module is used to: evaluate the fatigue state of the driver according to the driver's facial image and / or physiological information to obtain the degree of the driver's fatigue state; The degree of fatigue state includes shallow fatigue state and deep fatigue state; When the driver's fatigue state is a shallow fatigue state, a first intervention instruction is output; when the driver's fatigue state is a deep fatigue state, a second intervention instruction is output; The active intervention module is used to: after receiving the first intervention instruction, start the human-machine dialogue function to enable the driver to conduct a voice dialogue; The emergency physical intervention module is used to wake up the driver through a scent spraying device, a sound alarm device and a tactile vibration device after receiving a second intervention instruction.

2. The active intervention system for preventing driver fatigue according to claim 1, characterized in that: The active intervention module also includes a voice chat unit; the voice chat unit is used to enable the driver, relatives, friends or vehicle transportation companies to have a voice conversation through an instant voice chat room.

3. The active intervention system for preventing driver fatigue according to claim 1, characterized in that: The active intervention module also includes a personalized warning unit; The personalized warning unit is used to play a personalized warning sound to the driver; the personalized warning sound is a safety reminder recording using the voice of the driver's friend or relative.

4. The active intervention system for preventing driver fatigue according to claim 1, characterized in that: The odor spraying device is used to release irritating odors; the sound alarm device is used to sound an alarm through a speaker or a vehicle audio system; The tactile vibration device is a steering wheel tactile vibration device or a seat cushion tactile vibration device, and is used to wake up the driver through vibration.

5. The active intervention system for preventing driver fatigue according to claim 1, characterized in that: The active intervention system for preventing driver fatigue also includes an APP client; the APP client is used to view vehicle information, vehicle terminal settings, and vehicle terminal calls in real time.

6. The active intervention system for preventing driver fatigue according to claim 1, characterized in that: The active intervention system for preventing driver fatigue also includes a server; the server is used to provide data connection services and data processing services.

7. The active intervention system for preventing driver fatigue according to claim 1, characterized in that: The vehicle-mounted terminal also includes an image acquisition device; the image acquisition device is used to acquire the driver's facial image.

8. The active intervention system for preventing driver fatigue according to claim 1, characterized in that: The vehicle-mounted terminal also includes a physiological information collection device; The physiological information acquisition device is used to acquire the driver's physiological information; the physiological information includes heart rate, skin electricity, myoelectricity and blood oxygen.

9. The active intervention system for preventing driver fatigue according to claim 1, characterized in that: The active intervention system for preventing driver fatigue also includes a monitoring center; the monitoring center is used to monitor the vehicle driving status in real time.

10. The active intervention system for preventing driver fatigue according to claim 1, characterized in that: The vehicle-mounted terminal also includes a communication module, and the vehicle-mounted terminal is connected to the server, the monitoring center and the APP client through the communication module.

Images