A method for dealing with fatigue driving, terminal and vehicle

Abstract

The present disclosure relates to a method for coping with fatigue driving, a terminal and a car, and relates to the field of vehicle safety. When it is obtained that the driver is in a fatigue driving state, the vehicle is controlled to switch from a manual driving mode to an assisted driving mode; current address information, driving direction information and pre-stored map information are obtained, and the best safe parking address information is obtained according to the current address information, the driving direction information and the pre-stored map information; driving intention prompt information containing the best safe parking address information is output; when no feedback information from the driver is received for the driving intention prompt information, the vehicle is controlled to switch from the assisted driving mode to the automatic driving mode, and drive to a location corresponding to the best safe parking address information in the automatic driving mode; so as to ensure the driving safety of the vehicle after the driver is in a fatigue driving state.

Description

Technical Field

[0001] The present invention relates to the field of active protection for safe driving, and in particular to a method for dealing with fatigue driving, a terminal and a car. Background Art

[0002] With the rapid development of automotive electronics and Internet technology, drivers are prone to fatigue, which greatly increases the risk of driving. The current driver monitoring system only detects whether fatigue driving occurs, but does not further solve the problem of driving safety after fatigue driving is detected. Summary of the invention

[0003] The purpose of the present invention is to solve the driving safety problem after fatigue driving.

[0004] To achieve the above object, the present invention provides a method for dealing with fatigue driving, when it is obtained that the driver is in a fatigue driving state, controlling the vehicle to switch from a manual driving mode to an assisted driving mode;

[0005] Acquire current address information, driving direction information and pre-stored map information, and analyze and obtain optimal safe parking address information based on the current address information, the driving direction information and the pre-stored map information;

[0006] Outputting driving intention prompt information including the optimal safe parking address information;

[0007] When no feedback information is received from the driver regarding the driving intention prompt information, the vehicle is controlled to switch from the assisted driving mode to the automatic driving mode, and drives to a location corresponding to the optimal safe parking address information in the automatic driving mode, so as to solve the driving safety problem after the above-mentioned fatigue driving.

[0008] Preferably, when feedback information from the driver regarding the driving intention prompt information is received, the vehicle is controlled to switch from the assisted driving mode to the manual driving mode, or the vehicle is controlled to continue driving in the assisted driving mode.

[0009] Preferably, before the step of obtaining that the driver's state is a fatigue driving state, it also includes: obtaining the driver's facial information according to a first monitoring frequency, and judging whether the driver is in a fatigue state based on the facial information, wherein the facial information includes at least two of eye closure information, mouth opening and closing information, and head up and down movement information.

[0010] Preferably, after the step of determining whether the driver is in a fatigue state based on the facial information, it also includes: when it is determined that the driver is in a fatigue state based on the facial information, obtaining the driver's second facial information according to a second monitoring frequency, and determining whether the driver is in a fatigue state based on the second facial information, wherein the second monitoring frequency is greater than the first monitoring frequency.

[0011] Preferably, after the step of determining whether the driver is in a fatigue state according to the second facial information, the method further includes: outputting fatigue driving prompt information, wherein the fatigue driving prompt information includes at least one of voice prompts, pop-up prompts and light prompts, and a combination thereof.

[0012] Preferably, after the step of driving to the location corresponding to the optimal safe parking address information in the automatic driving mode, it also includes: controlling the vehicle to perform an automatic parking function in the automatic driving mode; when the vehicle completes automatic parking, controlling the vehicle to start the double flash function.

[0013] Preferably, before the step of controlling the vehicle to switch from the assisted driving mode to the automatic driving mode, it also includes: generating fatigue driving related information, and sending the fatigue driving related information to a server, the fatigue driving related information including the driver's contact information, relatives' contact information, vehicle driving related information and vehicle parking information.

[0014] Preferably, after the step of sending the fatigue driving related information to the server, the method further includes:

[0015] The cloud server forwards the information to the security management terminal, which calls the driver back and locates the vehicle information to notify nearby guards to ensure emergency response.

[0016] Preferably, it is characterized in that after the step of sending the fatigue driving related information to the server, it also includes: the cloud server forwards it to a reserved emergency contact, and the reserved emergency contact will obtain the real-time status of the driver and contact by phone.

[0017] A vehicle-mounted terminal comprises a processor and a memory, wherein the memory is used to store a computer program, and the computer program is recorded by the processor and executes the method for dealing with fatigue driving.

[0018] A car comprises the vehicle-mounted terminal. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a flow chart of a method for dealing with fatigue driving according to an embodiment of the present invention;

[0020] Figure 2It is a structural diagram of a method for dealing with fatigue driving according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0022] In order to make the technical problems, technical solutions and beneficial effects solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0023] Since the current driver fatigue monitoring system can only remind the driver through prompt sounds and display screens, but cannot provide assistance to fatigued drivers, driving risks still exist.

[0024] With the help of the vigorous development of image recognition algorithms and the continuous improvement of ADAS functions, the present invention provides a solution for driver fatigue monitoring and assisted driving based on an in-vehicle camera. When the driver is detected to be in a fatigued driving state, the system will give voice and warning icon reminders and continuously monitor the driver's state. If the driver's fatigue state is not eliminated after the reminder, the assisted driving system will intervene in the vehicle control and send out an emergency help message.

[0025] In one embodiment, a method for dealing with fatigue driving, when it is obtained that the driver is in a fatigue driving state, controlling the vehicle to switch from a manual driving mode to an assisted driving mode; obtaining current address information, driving direction information and pre-stored map information, and analyzing the current address information, the driving direction information and the pre-stored map information to obtain the best safe parking address information; outputting driving intention prompt information including the best safe parking address information; when no feedback information from the driver is received regarding the driving intention prompt information, controlling the vehicle to switch from the assisted driving mode to the automatic driving mode, and driving in the automatic driving mode to a location corresponding to the best safe parking address information.

[0026] Specifically, Figure 1As shown, when it is recognized that the driver is already in a fatigue state, the vehicle will be controlled to switch from the manual driving mode to the assisted driving mode, that is, it will automatically enter the assisted driving state, and according to the current address information, the driving direction information and the pre-stored map information, the best safe parking address information, such as a nearby service area, is analyzed according to the current address information, the driving direction information and the pre-stored map information; the driving intention prompt information containing the best safe parking address information is output, and after finding the service area, the driver is asked whether he wants to enter the service area, and a voice prompt is given to the driver to enter, and a pop-up window will also be displayed. If the prompt is repeated 10 times If the driver does not respond after the second time, or no feedback signal or operation is received from the driver, that is to say, the driver is still in a state of fatigue, the vehicle will switch from assisted driving to automatic driving mode, and drive to a safe place or safe service area in driving mode to ensure that the driver is already in a state of fatigue driving. The vehicle will automatically host an assisted driving to continue the current lane while searching for nearby safe stops. After searching for a nearby safe stop, the driver will be prompted to recover from the fatigue driving state. If the driver does not respond, the vehicle will change from assisted driving to automatic driving and drive to a safe stop.

[0027] In one embodiment, when feedback information from the driver regarding the driving intention prompt information is received, the vehicle is controlled to switch from the assisted driving mode to the manual driving mode, or the vehicle is controlled to continue driving in the assisted driving mode.

[0028] Specifically, after finding the service area, the driver is asked whether he wants to enter the service area. If the driver recovers from the fatigue driving state after receiving the prompt, or is no longer immersed in the driving state, such as controlling the steering wheel to turn, or performing other operations on the vehicle, the vehicle will immediately change from the assisted driving state to the manual driving state to ensure that after the driver is in a state of fatigue driving, the vehicle will automatically host an assisted driving to continue the current lane while searching for nearby safe stops.

[0029] In one embodiment, before the step of obtaining that the driver's state is a fatigue driving state, it also includes: obtaining the driver's facial information according to a first monitoring frequency, and judging whether the driver is in a fatigue state based on the facial information, wherein the facial information includes at least two of eye closure information, mouth opening and closing information, and head up and down movement information; when it is determined that the driver is in a fatigue state based on the facial information, obtaining the driver's second facial information according to a second monitoring frequency, and judging whether the driver is in a fatigue state based on the second facial information, wherein the second monitoring frequency is greater than the first monitoring frequency.

[0030] Specifically, Figure 2As shown, the driver fatigue monitoring and assisted driving system of this embodiment includes an infrared camera module 1 for collecting driver facial information, a central processing unit 3 for analyzing and processing the collected information and linking control of an ADAS system (advanced driver assistance system), an LCD display screen (liquid crystal display) 4 for displaying warning information, a HUD module (head-up display) 5 for displaying warning information, an ambient light 6 for adjusting the light in the cockpit, a speaker module 7 for voice prompts, a storage device 8 for storing the driver's driving status, a wireless network module 9 for sending emergency help information, an adaptive cruise system 10 for assisting the normal driving of the vehicle, an emergency braking system 11 for preventing vehicle collisions, and an automatic parking system 12 for safe parking of the vehicle.

[0031] The central processing unit determines whether the driver is in a fatigue state by analyzing the driver's facial information collected by the infrared camera module, such as the first monitoring frequency can be once every 10 seconds; the system determines whether the driver is in a fatigue driving state based on the collected facial information such as at least two groups of eye, mouth and head feature information. When the current driver's mouth opening and closing is greater than a preset mouth opening and closing threshold, and / or the driver's eyes closing is greater than a preset glasses closing threshold, and / or the driver's head up and down movement is greater than a preset threshold for head up and down movement, it is determined that the driver is in a fatigue state, and a second monitoring frequency will be used for monitoring, such as once every 5 seconds, wherein the second monitoring frequency is greater than the first monitoring frequency, and is used to detect the driver's driving state to determine whether he is in fatigue driving, and then to facilitate the vehicle's assisted driving operations and driving safety.

[0032] In one embodiment, after the step of determining whether the driver is in a fatigue state based on the second facial information, it also includes: outputting fatigue driving prompt information, and the fatigue driving prompt information includes at least one of voice prompts, pop-up prompts and light prompts, and a combination thereof.

[0033] Specifically, when the second monitoring frequency is adjusted to once every 5 seconds, fatigue driving prompt information is output, and the fatigue driving prompt information includes at least one of voice prompts, pop-up prompts and light prompts, and a combination thereof, wherein a voice prompt is performed and a flashing fatigue driving warning icon pops up on the central control screen and the HUD (head-up display), and the ambient light is also switched to a flashing mode. The vehicle system continuously monitors the driver's real-time driving status. When the driver's fatigue status improves, the above prompts are turned off. If the driver is still in a fatigue driving state, the automobile auxiliary driving system intervenes to keep the vehicle in driving status and search for nearby service areas at the same time. The search for nearby service areas also includes the nearest service areas in the same driving direction and in different driving directions. When service areas in two directions exist at the same time and the distances are the same, the nearest service area in the same driving direction is given priority. When service areas in the same driving direction and different driving directions are closer and the service distances in different driving directions are closer, the driving time and the real-time road conditions are judged, and a route with the shortest driving time and the best road conditions is preferred. When service areas in two directions exist at the same time and the service areas in the same driving direction are the closest, drive directly to the service area in the same driving direction.

[0034] In one embodiment, after the step of driving to the location corresponding to the optimal safe parking address information in the automatic driving mode, it also includes: controlling the vehicle to perform an automatic parking function in the automatic driving mode; when the vehicle completes automatic parking, controlling the vehicle to start a double flash function.

[0035] Specifically, when the vehicle drives to the nearest service area, which is a safe stop, through automatic driving, the vehicle is parked through the automatic parking system, and the lights are flashed to attract the attention of people around. The flashing lights generally refer to double flash lights.

[0036] In one embodiment, before the step of controlling the vehicle to switch from the assisted driving mode to the automatic driving mode, it also includes: generating fatigue driving related information and sending the fatigue driving related information to a server, wherein the fatigue driving related information includes the driver's contact information, relative contact information, vehicle driving related information and vehicle parking information.

[0037] Specifically, when the driver does not respond and is still in a fatigue driving state, and switches from the assisted driving mode to the automatic driving mode to drive to the service area, fatigue driving related information will be generated synchronously and sent to the server. The fatigue driving related information includes the driver's contact information, relatives' contact information, vehicle driving related information and vehicle parking information, etc., so as to retain this information so that it can be called in time when needed or shared synchronously with emergency contacts or car safety customer service, etc.

[0038] In one embodiment, after the step of sending the fatigue driving related information to the server, it also includes: the cloud server forwards it to the security management terminal, and the security management terminal calls the driver back and locates the vehicle information to notify the nearby guards to ensure emergency response. After the step of sending the fatigue driving related information to the server, it also includes: the cloud server forwards it to the reserved relatives, and the reserved emergency contact will obtain the real-time status of the driver and contact by phone.

[0039] Specifically, when the vehicle switches from assisted driving mode to automatic driving mode and drives to the service area, fatigue driving related information will be generated synchronously and sent to the server, which will be forwarded to the security management terminal and reserved relative contacts, that is, emergency contacts, through the cloud server. The owner will be informed of the current vehicle status and driver status, that is, the driver is in a fatigue driving state. After the security management terminal and the reserved relative contacts receive the information under normal network conditions, they will immediately make a telephone return call after receiving the information to wake up or prompt. If the driver's mobile terminal can receive such a telephone return call and can get out of the fatigue driving state, it can directly take over the vehicle, and the vehicle will change from automatic driving to manual driving and continue the previous driving; if the driver's mobile terminal does not receive such a telephone return call, the security management terminal will start the security management process, call the vehicle's real-time driving route map, locate and notify nearby traffic police or other safety assistance personnel for real-time rescue to avoid traffic accidents.

[0040] The driver fatigue monitoring and assisted driving solution provided by the present invention, wherein the monitoring module is an infrared camera module, which is connected to a central processing unit via a coaxial cable, and the solution comprises the following steps:

[0041] The central processing unit determines whether the driver is in a state of fatigue by analyzing the driver's facial information collected by the infrared camera module, and the monitoring frequency is once every 10 seconds. Based on the collected eye, mouth and head feature information, when the system determines that the driver is in a state of fatigue, the monitoring frequency is increased to once every 5 seconds. At the same time, a voice prompt is given and a flashing fatigue driving warning icon pops up on the central control screen and HUD. The ambient light also switches to flashing mode, and the system continuously monitors the driver's status.

[0042] When the driver's fatigue status improves, the above prompts will be turned off. If the driver is still in a state of fatigue driving, the car's assisted driving system will intervene to keep the vehicle in motion, search for nearby service areas, and prompt the driver to enter by voice. If the driver does not respond after repeated prompts for 10 times, which means the driver is still in a state of fatigue driving, a message will be sent to the emergency contact and car safety customer service via the wireless network, and the vehicle will switch the navigation route to the service area at this time;

[0043] When the vehicle automatically drives to the service area, the automatic parking system completes the parking and attracts the attention of people around by flashing lights.

[0044] In the specific implementation, in order to realize the driver fatigue monitoring and assisted driving solution, the steps specifically include:

[0045] Step 1, the infrared camera module 1 collects the driver's facial information at fixed time intervals and transmits it to the central processing unit 3 via a coaxial cable. The collected facial data is subjected to noise reduction processing, light adjustment, and contrast enhancement, and the central processing unit 3 marks the characteristic parts to form the characteristic information of the eyes, mouth and head.

[0046] Step 2, the fatigue monitoring module in the central processing unit 3 further analyzes the eye state as fully open, half open and half closed, and fully closed, the mouth state as fully open, half open and half closed, and the head state as sideways or not sideways, and compares it with the preset feature part data model, and then combines the blinking frequency and the direction of sight to determine whether the driver is in a fatigue state. If not, the next cycle sampling is performed at a fixed time interval. When the central processing unit 3 determines that the driver is in a fatigue state, the LCD module 4 and the HUD module 5 remind the driver by popping up a flashing warning icon, the ambient light module 6 reduces the driver's fatigue by increasing the brightness inside the car, and the speaker module 6 gives the driver a prompt through voice.

[0047] Step 3: When the driver resumes normal driving within the warning time, the LCD module 4, HUD module 5, ambient light module 6, and speaker module 7 stop the prompting function, and the infrared camera mode 1 performs the next cycle sampling according to a fixed time interval. When the warning time reaches the set time and the driver is still in a fatigue state, the adaptive cruise system 10 intervenes in the vehicle control, and the emergency brake system 11 increases the braking threshold to reduce the vehicle braking margin.

[0048] Step 4, the central control unit 3 searches for nearby service area sites, plans the driving route, and pushes it to the driver through the speaker module 6. If the push information is not responded to within the preset time, the central control unit 3 switches the navigation path to the service area, and the wireless network module 9 sends an emergency help message to the emergency contact and the car safety customer service.

[0049] Step 5: When the adaptive cruise system 10 reaches the service area according to the navigation path, the automatic parking system 12 will take over the vehicle, complete automatic parking, and attract the attention of people around by flashing lights.

[0050] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0051] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for dealing with fatigue driving, characterized by: When the driver's state is detected to be fatigue driving, the vehicle is controlled to switch from manual driving mode to assisted driving mode; Acquire current address information, driving direction information and pre-stored map information, and analyze and obtain optimal safe parking address information based on the current address information, the driving direction information and the pre-stored map information; Outputting driving intention prompt information including the optimal safe parking address information; When no feedback information from the driver is received in response to the driving intention prompt information, controlling the vehicle to switch from the assisted driving mode to the automatic driving mode, and driving the vehicle to a location corresponding to the optimal safe parking address information in the automatic driving mode; The coping methods also include: When feedback information from the driver regarding the driving intention prompt information is received, the vehicle is controlled to switch from the assisted driving mode to the manual driving mode, or the vehicle is controlled to continue driving in the assisted driving mode.

2. The method for dealing with fatigue driving according to claim 1, characterized in that: Before the step of obtaining that the driver's state is a fatigue driving state, the method further includes: The driver's facial information is acquired at a first monitoring frequency, and whether the driver is in a fatigue state is determined based on the facial information, wherein the facial information includes at least two of eye closure information, mouth opening and closing information, and head up and down movement information.

3. The method for dealing with fatigue driving according to claim 2, characterized in that: After the step of judging whether the driver is in a fatigue state according to the facial information, the method further includes: When it is determined that the driver is in a fatigue state based on the facial information, second facial information of the driver is obtained according to a second monitoring frequency, and whether the driver is in a fatigue state is determined based on the second facial information, wherein the second monitoring frequency is greater than the first monitoring frequency.

4. The method for dealing with fatigue driving according to claim 3, characterized in that: After the step of judging whether the driver is in a fatigue state according to the second facial information, the method further includes: Output fatigue driving prompt information, wherein the fatigue driving prompt information includes at least one of a voice prompt, a pop-up prompt and a light prompt, and a combination thereof.

5. The method for dealing with fatigue driving according to claim 1, characterized in that: After the step of driving to a location corresponding to the optimal safe parking address information in the automatic driving mode, the method further includes: In the automatic driving mode, controlling the vehicle to perform an automatic parking function; When the vehicle completes automatic parking, the vehicle's double flash function is activated.

6. The method for dealing with fatigue driving according to claim 1, characterized in that: Before the step of controlling the vehicle to switch from the assisted driving mode to the automatic driving mode, the method further includes: Generate fatigue driving related information and send the fatigue driving related information to a server, wherein the fatigue driving related information includes driver contact information, relative contact information, vehicle travel related information and vehicle parking information.

7. The method for dealing with fatigue driving according to claim 6, characterized in that: After the step of sending the fatigue driving related information to the server, the method further includes: The cloud server forwards the information to the security management terminal, which calls the driver back and locates the vehicle information to notify nearby guards to ensure emergency response.

8. The method for dealing with fatigue driving according to claim 6, characterized in that: After the step of sending the fatigue driving related information to the server, the method further includes: The cloud server forwards the information to the reserved relative, and the reserved emergency contact will obtain the real-time status of the driver and make a phone call.

9. A vehicle-mounted terminal, characterized in that: It comprises a processor and a memory, wherein the memory is used to store a computer program, and the computer program is recorded by the processor and executes the method for dealing with fatigue driving as described in one of claims 1-8.

10. A car, comprising the vehicle-mounted terminal according to claim 9.

Images