Driver body health state detection method based on driver monitoring system

Abstract

The invention is suitable for the technical field of vehicle intelligent driving, and provides a driver body health state detection method based on a driver monitoring system, and the method comprises the following steps: monitoring the facial expression and action of a driver in a vehicle through a health state detection module, and judging the health state of the driver; when the health state of the driver is abnormal and permission of the driver is obtained, the vehicle taking-over module takes over the running vehicle, so that the vehicle is controlled by the intelligent driving system to be pulled over and parked; telephone contact with an emergency center is established through the emergency contact module, and the emergency contact module is used for sending emergency rescue information in an emergency call mode. The problem of traffic accidents caused by the fact that a driver cannot control the vehicle due to sudden illness in the current vehicle driving process can be effectively solved; meanwhile, a rescue call can be dialed in time, and a driver can be rescued in time.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle intelligent driving, and specifically relates to a method for detecting the physical health status of a driver based on a driver monitoring system. Background Art

[0002] Monitoring the physical state and behavior of drivers has become increasingly important in the automotive industry. With the development of vehicle automation technology, a driver monitoring system (DMS) monitors the physical state and behavior of drivers in real time to timely detect potential dangerous situations and take necessary measures. Among them, the driver monitoring system is based on computer vision, deep learning, and sensor technology. By installing sensors such as infrared supplementary light cameras in the cockpit, the system can capture data such as the facial expressions, eye movements, head postures, and limb movements of drivers in real time. After these data are processed and analyzed by algorithms, the system can identify the bad driving behaviors of drivers and issue warning prompts.

[0003] Most of the existing driver monitoring technologies are for detecting and warning about relatively dangerous driving behaviors such as fatigue, distraction, yawning, and closing eyes. However, there is no perfect processing mechanism for monitoring behaviors such as a driver suddenly having a heart attack or cerebral infarction and instantly losing the ability to drive. Therefore, a method for detecting the physical health status of a driver based on a driver monitoring system is proposed to solve the above problems. Summary of the Invention

[0004] Aiming at the deficiencies of the existing technology, the purpose of the present invention is to provide a method for detecting the physical health status of a driver based on a driver monitoring system to solve the problems in the above background art.

[0005] The present invention is implemented as follows. A method for detecting the physical health status of a driver based on a driver monitoring system, the method includes the following steps: Monitor the facial expressions and actions of the driver in the vehicle through a health status detection module to judge the health status of the driver; When the health status of the driver is abnormal and the driver's permission is obtained, take over the vehicle in motion through a vehicle takeover module, so that the vehicle performs a pull-over stop process under the control of the intelligent driving system; Establish a telephone connection with the emergency center through an emergency contact module to send the emergency assistance information by means of an emergency call.

[0006] As a further solution of the present invention: the step of monitoring the facial expressions and actions of the driver in the vehicle through the health status detection module specifically includes: Establish a multi-functional classification model by means of real-person collection; Obtain the image data of the driver inside the vehicle through an imaging device, where the imaging device is an infrared supplementary light camera; Input the image data into a multi-functional classification model to identify the driver's facial expressions and actions.

[0007] As a further solution of the present invention: The steps of establishing the multi-functional classification model by using the method of collecting real people specifically include: Collect the facial expression data and action data of real people according to different lighting conditions encountered during driving; Classify and label the collected facial expressions and behavioral action data to guide the training work of the model; Update the weights of the multi-functional classification model through backpropagation of the loss function, and obtain the best-performing multi-functional classification model after multiple iterations.

[0008] As a further solution of the present invention: The loss function is a cross-entropy loss function or FocalLoss.

[0009] As a further solution of the present invention: The steps of inputting the image data into the multi-functional classification model to identify the driver's facial expressions and actions specifically include: Input the image data into the multi-functional classification model to obtain the model classification result, and use the image data to iteratively update the multi-functional classification model; Obtain the weighted result by using time series information for queue weighting according to the model classification result; When the weighted result is greater than the set threshold, it is determined that the current health status of the driver is an abnormal state.

[0010] As a further solution of the present invention: The specific steps of taking over the vehicle in motion through the vehicle takeover module specifically include: When the current health status of the driver is abnormal, send a vehicle takeover request to the vehicle through the vehicle takeover module, and the duration of the vehicle takeover request is a set value; When the driver accepts the request or does not respond within the set time, take over the operation right of the vehicle through the intelligent driving system; Identify the road conditions of the current road through the intelligent driving system and perform automatic pulling over to the side of the road and turn on the double flash function of the vehicle.

[0011] As a further solution of the present invention: The steps of establishing a telephone connection with the emergency center through the emergency contact module specifically include: When the current health status of the driver is abnormal, send a first aid call request to the vehicle through the emergency contact module; When the driver accepts the request or does not respond within the set time, dial the first aid call through the vehicle emergency communication system to feedback the corresponding first aid information; Obtain location information according to the high-precision map and transmit and feedback it through the emergency call.

[0012] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention collects the facial expressions and behavioral action data of the driver in real time through the health status detection module, and judges the health status of the driver. When the driver is in an unhealthy state, first, the intelligent driving system takes over the vehicle, controls the vehicle to gradually decelerate and pull over to the side of the road, and dials an emergency call according to the current location to establish a telephone connection with the emergency center, thereby sending out emergency information. In summary, the present invention can effectively solve the problem that when a driver suddenly falls ill during vehicle driving and cannot control the vehicle, resulting in traffic accidents; at the same time, it can also call the rescue phone in time to rescue the driver in time. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic flow chart of the method for detecting the physical health status of a driver based on a driver monitoring system in the implementation of the present technology.

[0014] Figure 2 It is a schematic flow chart of the implementation of the health status detection function in the implementation of the present technology. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0015] In order to make the objectives, technical solutions and advantages of the present invention clearer, the following further details the present invention with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0016] The following describes in detail the specific implementation of the present invention with reference to specific embodiments.

[0017] To achieve the above objective, the present invention provides a method for detecting the physical health status of a driver based on a driver monitoring system. The method for detecting the physical health status of a driver mainly relies on the collaborative action of three modules: a health status detection module 100, a vehicle takeover module 200, and an emergency contact module 300.

[0018] As Figure 2 shown, the health status detection module 100 mainly has two functions: driver facial expression monitoring and action recognition. Among them, the driver facial expression monitoring function uses real-time infrared fill light camera image data, and through a convolutional neural network, analyzes the current facial expression of the driver, so as to judge whether the physical health status is in a normal state; the action recognition function uses real-time and multi-frame infrared fill light camera image data, and through a convolutional neural network, analyzes the current action behavior of the driver, so as to judge whether the physical health status is in a normal state.

[0019] Further, the specific steps of the health status detection function are as follows: Step 1: Training the multi-functional classification model for driver facial expression and action recognition. The specific operation is as follows: 1. Considering the influence of light on image imaging, for common light scenarios in normal driving such as strong light, weak light, darkness and other different light conditions, through multiple real human collectors, under different light and different vehicle model scenarios, collect different expressions (such as happiness, anger, sorrow, joy, pain, etc.) and different actions (such as hand on forehead, body stiffness, hand covering chest; and other dangerous driving actions such as smoking, drinking water, making a phone call, dancing, etc.), and obtain facial expression data and action data through the above methods; 2. Classify and label the collected facial expression and behavioral action data to guide the training of the multi-functional classification model; 3. Update the weights of the multi-functional classification model through backpropagation using the loss function to minimize the loss. After multiple iterations, the multi-functional classification model with the best performance is obtained. Common loss functions include cross-entropy loss function and FocalLoss.

[0020] Step 2: Input the image data into the multi-functional classification model for driver facial expression and action recognition. The specific operation is as follows: 1. Input the real-time, multi-frame (such as fixed 4 frames) infrared supplementary light camera image data into the multi-functional classification model; 2. For the model classification results, use temporal information for queue weighting. Specifically: If the classification result is that the health status is normal, the result is recorded as -1; if the classification result is that there is a problem with the health status, the result is recorded as 1; assuming the weighted queue is 30 (the frame rate of the infrared supplementary light camera is generally 30 FPS), and weighting is performed according to the temporal proportion, the result weight of the frame closer to the current time is greater. When the weighted result is greater than a certain threshold, it is determined that the driver's current health status is abnormal.

[0021] As Figure 1 shown, when the detection result of the health status detection module 100 is that the driver's current health status is abnormal, the intelligent driving system starts to take over the vehicle and controls the vehicle to safely pull over. The specific steps are as follows: Step 1: Vehicle takeover request. When the detection result of the health status detection module is that the driver's current health status is abnormal, the intelligent driving system issues a vehicle takeover request, and the driver can manually or verbally reject or accept it; Step 2: Driver request acceptance. When the driver's current health is in an abnormal state, it is impossible to manually or verbally accept. After the intelligent driving system takeover request lasts for a certain period of time (such as 3 s) and there is no acceptance, it is defaulted to be accepted; Step 3: The vehicle pulls over to the side of the road. The intelligent driving system takes over the vehicle and gradually pulls over to the side of the road with safe control, and turns on the hazard lights; As Figure 1 shown, when the detection result of the health status detection module 100 is that the current health status of the driver is abnormal, the vehicle emergency communication system starts to call the emergency call (such as 120, etc.). The specific steps are as follows: Step 1: Emergency call request. When the detection result of the health status detection module 100 is that the current health status of the driver is abnormal, the vehicle emergency communication system issues an emergency call request, and the driver can manually, verbally refuse or accept it; Step 2: Driver's request acceptance. When the driver's current health is in an abnormal state and cannot be manually or verbally accepted, after the vehicle emergency communication system issues an emergency call request for a certain period of time (such as 3 s) and there is no acceptance request, it is defaulted to be accepted.

[0022] Step 3: Call the emergency call and feedback the corresponding emergency information. After the emergency call is connected, according to the high-precision map information, the location and situation information are fed back and wait for rescue.

[0023] Through the above-mentioned health status detection module 100, the driver's health status can be monitored. When the driver encounters a sudden health condition and is unable to control the vehicle, emergency assistance can be realized through the vehicle takeover module 200 and the emergency contact module 300 to prevent accidents and personal safety from being damaged.

[0024] The above only describes the preferred embodiments of the present invention in detail and is not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

[0025] It should be understood that although the steps in the flowcharts of the embodiments of the present invention are shown in sequence according to the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps does not have a strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in each embodiment may include multiple sub-steps or multiple stages. These sub-steps or stages do not necessarily have to be executed at the same moment, but can be executed at different moments. The execution order of these sub-steps or stages does not necessarily have to be sequential, but can be executed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

[0026] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0027] After considering the specification and the disclosure of the embodiments, those skilled in the art will readily conceive of other embodiments of the present disclosure. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and the embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the claims.

Claims

1. A method for detecting the physical health status of a driver based on a driver monitoring system, characterized in that, The method includes the following steps: Monitor the facial expressions and movements of the driver in the vehicle through a health status detection module to judge the health status of the driver; When the driver's health status is abnormal and the driver's permission is obtained, take over the moving vehicle through a vehicle takeover module, so that the vehicle performs a pull-over operation under the control of the intelligent driving system; Establish a telephone connection with the emergency center through an emergency contact module to send emergency rescue information by means of an emergency call.

2. The method for detecting the physical health status of a driver based on a driver monitoring system according to claim 1, wherein, The step of monitoring the facial expressions and movements of the driver in the vehicle through the health status detection module specifically includes: Establish a multi-functional classification model by using the method of collecting real people; Obtain the image data of the driver in the vehicle through an imaging device, and the imaging device is an infrared fill light camera; Input the image data into the multi-functional classification model to identify the facial expressions and movements of the driver.

3. The driver physical health status detection method based on the driver monitoring system according to claim 2, characterized in that, The step of establishing a multi-functional classification model by using the method of collecting real people specifically includes: Collect the facial expression data and movement data of real people according to different lighting conditions encountered during driving; Classify and label the collected facial expression and behavior movement data to guide the training work of the model; Update the weights of the multi-functional classification model through backpropagation of the loss function, and obtain the best-performing multi-functional classification model after multiple iterations.

4. The method for detecting the physical health status of a driver based on a driver monitoring system according to claim 2, wherein The loss function is the cross-entropy loss function or FocalLoss.

5. The method for detecting the physical health status of a driver based on a driver monitoring system according to claim 2, characterized in that, The step of inputting the image data into the multi-functional classification model to identify the facial expressions and movements of the driver specifically includes: Input the image data into the multi-functional classification model to obtain the model classification result, and use the image data to iteratively update the multi-functional classification model; Obtain a weighted result by weighting the queue according to the model classification result using temporal information; When the weighted result is greater than the set threshold, it is determined that the current health status of the driver is an abnormal state.

6. The method for detecting the physical health status of a driver based on a driver monitoring system according to claim 1, characterized in that, The specific steps of taking over the moving vehicle through the vehicle takeover module specifically include: When the current health status of the driver is abnormal, send a vehicle takeover request to the vehicle through the vehicle takeover module, and the duration of the vehicle takeover request is a set value; When the driver accepts the request or does not respond within the set time, take over the operation right of the vehicle through the intelligent driving system; Identify the road conditions of the current road through the intelligent driving system and perform an automatic pull-over operation and turn on the hazard warning lights of the vehicle.

7. The method for detecting the physical health status of a driver based on a driver monitoring system according to claim 1, characterized in that, The step of establishing a telephone connection with the emergency center through the emergency contact module specifically includes: When the current health status of the driver is abnormal, send an emergency call dialing request to the vehicle through the emergency contact module; When the driver accepts the request or does not respond within the set time, dial the emergency call through the vehicle emergency communication system to feedback the corresponding emergency information; Obtain the location information according to the high-precision map and transmit and feedback it through the emergency call.

Images