Intelligent monitoring and safety emergency system based on health state of vehicle driver

Abstract

The invention discloses an intelligent monitoring and safety emergency system based on the health state of a vehicle driver, and relates to the technical field of safe driving. Comprising the steps that the state of a current driving vehicle of a driver is obtained through a physical sign obtaining module and an image recognition module, and when an early warning prompting module recognizes that health data, gravity data and facial expressions of the driver exceed normal ranges, it is indicated that the driver is at risk of myocardial infarction and other abnormal states. Through the vehicle steering wheel and the vehicle seat cushion, direct contact with a driver can be achieved, various accurate driver physical sign data are obtained, the image recognition module is combined to recognize expression features when the driver pains or not, and whether the driver is at risk and has an abnormal state or not is jointly judged. When an abnormal state occurs due to risks, early warning is carried out to remind that the body is possibly abnormal, the vehicle is prevented from being driven again as much as possible, if the body cannot be controlled, rescue is directly requested, a driver is rescued as soon as possible, and loss is effectively reduced.

Description

Technical Field

[0001] The present application relates to the field of safe driving systems, and specifically to an intelligent monitoring and safety emergency system based on the health status of vehicle drivers. Background Art

[0002] The driver is responsible for the safety of himself and all the passengers when driving a car. However, due to their occupation, such as sitting for long periods of time, staying up late, lack of exercise and high mental stress, drivers have become a high-risk group for myocardial infarction. However, the signs before a myocardial infarction are easily ignored, and when a real attack occurs, people often lose consciousness within seconds. Therefore, if a driver suffers a myocardial infarction while driving, it is very likely to cause loss of life for himself and his passengers, and it will also threaten the safety of surrounding vehicles and personnel.

[0003] In the prior art, the patent with application number "CN202310968190.5" and invention name "Vehicle driver monitoring method, device, vehicle and computer storage medium" discloses a vehicle driver monitoring method, which obtains the real-time monitoring data of the driver's vital signs through an infrared binocular camera. The problems with the current prior art are: the accuracy of the driver's vital signs data obtained through the infrared binocular camera is limited, and when the driver's vital signs are identified to be abnormal, there is no corresponding emergency treatment method. Therefore, there is a need for an intelligent monitoring and safety emergency system and method based on the health status of the vehicle driver, which obtains vital sign data through direct contact with the vehicle steering wheel and seat cushion, and combines the camera to collect the driver's facial features to determine whether there is an abnormality at present. After confirming the abnormality, the driver will be warned and alarmed to ensure the safety of the driver and passengers and effectively reduce the risk of vehicle loss of control. Summary of the invention

[0004] This application aims to solve the problem that the accuracy of the driver's vital signs data currently obtained is limited, and there is no corresponding emergency treatment method when the driver's vital signs are identified to be abnormal. It provides an intelligent monitoring and safety emergency system and method based on the health status of the vehicle driver. Vital sign data is obtained through direct contact with the vehicle steering wheel and seat cushion. The camera is combined to collect the driver's facial features to determine whether there is an abnormality at present. After confirming the abnormality, the driver is warned and alarmed to ensure the safety of the driver and passengers and effectively reduce the risk of vehicle loss of control. The specific technical solution is as follows: In a first aspect of the present application, an intelligent monitoring and safety emergency system based on the health status of a vehicle driver is provided, comprising: The vital sign acquisition module is provided with data sensing devices on the vehicle steering wheel and in the vehicle seat, and the driver's health data is collected by the driver holding the vehicle steering wheel, and the health data includes heart rate, pulse, temperature and humidity data; at the same time, the driver's gravity data on the vehicle seat is collected according to the driver's sitting and lying posture on the vehicle seat, and the gravity data includes the driver's weight and sitting posture data; An image recognition module, wherein the vehicle is provided with an image acquisition device, the image acquisition device acquires a driver image in real time, obtains the driver's facial features and facial expressions in the driver image, and classifies the facial expressions; The early warning module collects the changes in the health data and the gravity data, as well as the facial expressions of the driver when an abnormality occurs during the driving of the vehicle, and inputs them into the neural network model for training to obtain a recognition model that can identify the driver's risk of abnormal state. When the health data, gravity data and facial expressions of the driver are identified to be out of the normal range during the driving process of the vehicle, an early warning is issued; The emergency avoidance module turns on the vehicle's hazard warning lights when an early warning is issued; if the driver does not process the early warning within a preset time, a preset phone number is dialed to inform the other party of the current location, preset sentence and the health data.

[0005] In one embodiment of the present application, the emergency avoidance module also includes a lane avoidance sub-module, and the image acquisition device also includes identifying the road on which the vehicle is traveling. When the driver does not process the warning within a preset time, the current lane of the vehicle is determined, the steering wheel angle of the vehicle is controlled, and the vehicle is driven toward the emergency lane or the edge of the road, and stops after entering the predetermined lane.

[0006] In one embodiment of the present application, the image recognition module also includes a personalization submodule, which establishes or retrieves the identity information of the current driver based on the facial features, and the identity information includes the driver image, and the health data and gravity data of the driver's driving history of the current vehicle, to form a health record of the driver. The driver's gender and height data are obtained from the health record through the driver image, and the weight data is obtained through the gravity data, to obtain the normal range of the driver's current health data and gravity data.

[0007] In one embodiment of the present application, the image recognition module also includes a state recognition submodule. By classifying facial expressions, different facial expressions correspond to different states, and corresponding processing is performed for different states. The facial expressions include calmness, fatigue and anger. The calm driving state is not processed; the processing performed in the fatigued driving state includes voice reminders, vehicle seat vibrations and vehicle steering wheel vibrations; the processing performed in the angry driving state includes voice soothing and speed limit.

[0008] In one embodiment of the present application, the state recognition submodule also includes a first pre-recognition submodule, which recognizes the facial features and health data of the driver when the driver enters the vehicle. When it is recognized that the driver's current mental state and / or health data are not good, a reminder is given in advance to rest and avoid driving for a long time.

[0009] In one embodiment of the present application, the state recognition submodule also includes a second pre-recognition submodule. When the image acquisition device captures a situation that may cause the driver to drive in an angry state during driving, the driver is calmed in advance. The situations that may cause the driver to drive in an angry state include early driving, traffic jams, other vehicles cutting in, the driver's facial expression is serious, frowning, wrinkling the nose and squinting, and the driver speaks in an increased volume.

[0010] In one embodiment of the present application, the data sensing device includes a temperature collection device disposed on the vehicle steering wheel. When the driver's hand touches the vehicle steering wheel, the hand temperature is collected to determine whether to turn on the air conditioner or the heater.

[0011] In one embodiment of the present application, the data sensing device includes a gravity sensing device and / or a humidity sensing device arranged in the steering wheel of the vehicle. The humidity sensing device determines whether the driver's hands are sweating, the gravity sensing device determines whether the driver's hands are shaking, and the sitting posture data determines whether the body is shaking. When the driver's hands are sweating, or his hands and body are shaking, and the image sensing device recognizes that the driver's facial expression is nervous or fearful, it is determined that there is danger in the car, and the vehicle's position and the audio and images in the car are synchronized to the outside world.

[0012] In one embodiment of the present application, the gravity sensor device is provided in the vehicle steering wheel, and different pressing times and numbers of pressing the vehicle steering wheel correspond to different information. When it is judged that danger occurs in the vehicle, the driver transmits corresponding information to the outside world by pressing the vehicle steering wheel for different times.

[0013] In a second aspect of the present application, a smart monitoring and safety emergency method based on the health status of a vehicle driver is provided, including: Data sensing devices are arranged on the vehicle steering wheel and in the vehicle seat to collect the driver's health data through the driver holding the vehicle steering wheel, and the health data includes heart rate, pulse, temperature and humidity data; at the same time, according to the driver's sitting and lying posture on the vehicle seat, the driver's gravity data on the vehicle seat is collected, and the gravity data includes the driver's weight and sitting posture data; The vehicle is provided with an image acquisition device, the image acquisition device acquires a driver image in real time, obtains the driver's facial features and facial expressions in the driver image, and classifies the facial expressions; Collect the changes in the health data and gravity data, as well as the facial expressions of existing drivers when an abnormality occurs during driving a vehicle, input them into a neural network model for training, and obtain a recognition model that can identify drivers who are at risk of abnormal conditions. When it is identified that the health data, gravity data, and facial expressions of the driver are out of the normal range during vehicle driving, an early warning is issued; When a warning is issued, the vehicle's hazard warning lights are turned on; if the driver does not handle the warning within a preset time, a preset phone number is dialed to inform the other party of the current location, preset sentence and the health data.

[0014] In one embodiment of the present application, the image acquisition device also includes identifying the road on which the vehicle is traveling. When the driver does not process the warning within a preset time, it determines the current lane of the vehicle, controls the steering wheel angle of the vehicle, drives the vehicle to the emergency lane or the edge of the road, and stops after entering the predetermined lane.

[0015] In one embodiment of the present application, the identity information of the current driver is established or retrieved based on the facial features, and the identity information includes the driver image, and the health data and gravity data of the driver's driving history of the current vehicle, forming a health record of the driver. The driver's gender and height data are obtained from the health record through the driver image, and the weight data is obtained through the gravity data, thereby obtaining the normal range of the driver's current health data and gravity data.

[0016] In one embodiment of the present application, by classifying facial expressions, different facial expressions correspond to different states, and corresponding processing is performed for different states. The facial expressions include calmness, fatigue and anger. The calm driving state is not processed; the processing performed in the fatigued driving state includes voice reminders, vehicle seat vibrations and vehicle steering wheel vibrations; the processing performed in the angry driving state includes voice soothing and speed limit.

[0017] In one embodiment of the present application, when the driver enters the vehicle, the facial features and health data of the driver are identified. When it is identified that the driver's current mental state is not good and / or the health data is not good, a reminder is given in advance to rest and avoid driving for a long time.

[0018] In one embodiment of the present application, when the image acquisition device captures situations that may cause the driver to drive in anger during driving, the driver is calmed in advance. The situations that may cause the driver to drive in anger include early driving, traffic jams, other vehicles cutting in, the driver's facial expression is serious, frowning, wrinkling the nose and squinting, and the driver speaks in an increased volume.

[0019] In one embodiment of the present application, a temperature collection device on the vehicle steering wheel collects the temperature of the driver's hand when the driver's hand touches the vehicle steering wheel to determine whether to turn on the cooling or heating.

[0020] In one embodiment of the present application, a gravity sensor device and / or a humidity sensor device is further provided in the steering wheel of the vehicle, the humidity sensor device determines whether the driver's hands are sweating, the gravity sensor device determines whether the driver's hands are shaking, and the sitting posture data determines whether the body is shaking. When the driver's hands are sweating, or his hands and body are shaking, and the image sensor device recognizes that the driver's facial expression is nervous or fearful, it is determined that there is danger in the car, and the vehicle's position and the audio and images in the car are synchronized to the outside world.

[0021] In one embodiment of the present application, the gravity sensor device is provided in the vehicle steering wheel, and different pressing times and numbers of pressing the vehicle steering wheel correspond to different information. When it is judged that danger occurs in the vehicle, the driver transmits corresponding information to the outside world by pressing the vehicle steering wheel for different times.

[0022] This application has the following beneficial effects: 1. The driver's current driving status is obtained through the vital sign acquisition module and the image recognition module. When the early warning prompt module recognizes that the driver's health data, gravity data and facial expression are beyond the normal range, it means that the driver is at risk of abnormal conditions such as myocardial infarction. At this time, if the driver is still awake, the early warning is turned off, and the driver drives the vehicle to the side to wait for rescue according to his own situation, or he can go to the hospital for treatment according to the planned route given by the vehicle; if the driver can no longer effectively control his body and cannot turn off the early warning, it means that the driver is in a bad state. He directly calls the preset phone number and informs the other party of the current location and the preset sentence through the vehicle GPS. At the same time, the health data is sent together for reference by rescue personnel, so as to prepare first aid tools for corresponding diseases in advance; through the vehicle steering wheel and the vehicle seat cushion, direct contact with the driver can be made to obtain a variety of accurate driver vital signs data, and combined with the image recognition module to identify whether the driver has uncomfortable facial features, and jointly judge whether the driver is at risk of abnormal state. When there is a risk of abnormal state, an early warning is issued to remind the driver that the body may be abnormal and that driving the vehicle should be avoided as much as possible. If the driver can no longer control the body normally, a request for rescue is made directly, including calling relatives and friends and the police, so as to treat the driver as soon as possible and effectively reduce losses.

[0023] 2. Considering that the driver may not be able to control the vehicle when an abnormality occurs, it may cause the vehicle to lose control, posing a threat to the driver, passengers, and vehicles and pedestrians around the vehicle. Therefore, when the driver fails to process the warning within the preset time, the vehicle is controlled to enter the emergency lane or the edge of the road, and then stops to wait for rescue, which further ensures that the vehicle does not lose control when the driver suddenly fails to control the vehicle. Specifically, the image acquisition device identifies the road and specific lane where the vehicle is traveling, and adjusts the angle of the vehicle's steering wheel according to the distance from the emergency lane or the edge of the road, so that the vehicle is changed to a relatively safe lane. Stopping on the roadside and waiting for rescue greatly reduces the risk of vehicle collision.

[0024] 3. Since the normal ranges of the health data and gravity data are different for different drivers, a personalized module is established to create a file for the driver driving the vehicle. The identity information of the current driver is obtained by collecting facial features through an image acquisition device, and the normal ranges of the current health data and gravity data are obtained based on the driver's own identity information. The early warning module will only issue an early warning when it recognizes that the data exceeds the normal range corresponding to the current driver. Furthermore, the driver's medical history can also be entered in the identity information, and the normal ranges of the health data and gravity data can be adjusted according to the previous medical history. When the driver feels unwell, the driver can be provided with suggestions for alleviating the condition through voice based on the treatment methods in the previous medical history.

[0025] 4. Considering that it is not only the driver who has a disease that makes it difficult to move that requires emergency treatment, but also when the driver is driving fatigued or angry, emergency treatment is also required according to the driver's condition. For example, in the state of fatigue driving, voice reminders, playing music with a strong sense of rhythm, etc. are used to make the driver concentrate on driving the vehicle; for example, the corresponding treatment in the angry driving state includes corresponding voice soothing and speed limit to prevent the driver from speeding due to fighting with other vehicles; for example, in the state of driving in physical discomfort, when the image acquisition device recognizes that the driver's face is in pain and the mouth and nose are breathing heavily, it can also be combined with gravity data. When it is recognized that the driver is sitting in an irregular posture, the driver is reminded to stop driving the vehicle and ask whether the old disease has recurred, and the corresponding relief method is provided; if the driver still cannot concentrate on driving the vehicle within the preset time, it also means that the driver is at risk of abnormal conditions, and warning and risk avoidance operations are also performed; further, when the driver is about to start the vehicle, , judge whether the driver is driving fatigued, including identifying whether there are "dark circles" and whether the pulse is accelerated, judge whether the driver may be tired in the next driving process, remind the driver in advance, and plan a parking and rest location, such as a parking lot, a highway service area, and determine the mandatory rest time according to the driver's status; at the same time, the image recognition module inputs images that may cause the driver to be angry driving state for training, and when the data acquisition device collects, for example, early driving, traffic jams, other vehicles cutting in, the driver's facial expression is serious, frowning, wrinkling the nose, squinting, and the driver's voice volume increases, the driver is comforted in advance to avoid the driver being in an angry state and difficult to accept any comfort; further, personalized settings can be made according to the reasons why the driver was angry when driving the vehicle in the past. When the data acquisition device recognizes the same situation that caused the driver to be angry as before, the driver is comforted in advance.

[0026] 5. Considering that the driver's body temperature is cold, it is easy to cause stiff hands and slow reaction, and considering that the driver's body temperature is high, it is easy to cause heatstroke and fainting. Therefore, the temperature sensor on the steering wheel of the vehicle is used to identify the temperature of the driver's palms. When the temperature exceeds the normal value, the cold air or warm air is turned on accordingly. Further, considering that the driver may be in good physical condition, but it is difficult to make a normal alarm due to the threat of hijacking and other reasons, emergency treatment is also required. Therefore, by identifying whether the driver is shaking, and combining the image sensor device to identify that the driver's facial expression is nervous or fearful, it is judged that there is a danger in the car, and the vehicle position and the audio and image in the car are synchronized to the outside world, so that the outside world can obtain the situation in the car, which is convenient for rescuing the driver. Further, combined with the pressure sensor set on the steering wheel of the vehicle, different information is pre-set, and the driver presses the steering wheel of the vehicle with his fingers for different times and times to achieve simple communication with the outside world. The most important thing is that the communication method is relatively concealed and not easy to be discovered, so that the outside world can understand the situation in the car more clearly, which is more helpful for rescuing the driver. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0028] Figure 1 A schematic diagram of the electronic device structure of the hardware operating environment involved in an embodiment of the present application.

[0029] Figure 2 It is a schematic diagram of the functional modules of the intelligent monitoring and safety emergency system based on the health status of the vehicle driver provided in the embodiment of the present application.

[0030] Figure 3 It is a schematic diagram of the module architecture of the intelligent monitoring and safety emergency system based on the health status of the vehicle driver provided in an embodiment of the present application.

[0031] Figure 4 It is a step flow chart of the intelligent monitoring and safety emergency method based on the health status of the vehicle driver provided in the embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0033] The solution of the present application is further described below in conjunction with the accompanying drawings.

[0034] like Figure 1 As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0035] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0036] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a data storage module, a network communication module, a user interface module and an electronic program.

[0037] exist Figure 1 In the electronic device shown, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be set in the electronic device, and the electronic device calls the intelligent monitoring and safety emergency system based on the health status of the vehicle driver stored in the memory 1005 through the processor 1001, and executes the intelligent monitoring and safety emergency system based on the health status of the vehicle driver provided in the embodiment of the present application.

[0038] Based on the aforementioned hardware operating environment and system architecture, in the first aspect of the present application, refer to Figure 2 and Figure 3 As shown, it provides an intelligent monitoring and safety emergency system based on the health status of the vehicle driver, including: The vital sign acquisition module is provided with data sensing devices on the vehicle steering wheel and in the vehicle seat, and the driver's health data is collected by the driver holding the vehicle steering wheel, and the health data includes heart rate, pulse, temperature and humidity data; at the same time, the driver's gravity data on the vehicle seat is collected according to the driver's sitting and lying posture on the vehicle seat, and the gravity data includes the driver's weight and sitting posture data; It should be noted that there are multiple blood vessels on the palm, and various physiological data of the driver can be obtained by directly contacting the vehicle steering wheel with the palm. According to the needs, multiple data acquisition devices are set on the vehicle steering wheel, such as a temperature sensor to collect the temperature data of the palm, and a pulse sensor to collect the beating data of the palm artery; similarly, by setting pressure sensors at different positions on the vehicle seat cushion, the driver sits on the vehicle seat cushion, and the driver's current sitting posture and weight data are obtained according to the data obtained by the pressure sensors at different positions; An image recognition module, wherein the vehicle is provided with an image acquisition device, the image acquisition device acquires a driver image in real time, obtains the driver's facial features and facial expressions in the driver image, and classifies the facial expressions; It should be noted that the image acquisition device is a device that can acquire, record and upload images, including an infrared camera, and the neural network is trained so that the image recognition module can recognize the facial features and facial expressions of the driver from the acquired driver image, and further classify the facial expressions according to the features, including smiling, frowning, wrinkling the nose, squinting the eyes and pursing the lips; The early warning module collects the changes in the health data and the gravity data, as well as the facial expressions of the existing drivers when an abnormality occurs during driving the vehicle, and inputs them into the neural network model for training to obtain a recognition model that can identify the driver's risk of abnormal state. When the health data, gravity data and facial expressions of the driver are identified to be out of the normal range during the vehicle driving process, a warning is issued; It should be noted that many acute diseases usually cause varying degrees of discomfort before onset, such as the discomfort before a myocardial infarction, including heartache, sweating, palpitations or irregular heartbeats, but many drivers do not care and continue to drive the vehicle. Therefore, the neural network is trained by inputting the data range of abnormal conditions such as myocardial infarction, including irregular heartbeats, uncomfortable expressions and unstable sitting postures. When the vehicle driving process identifies that the driver's health data, gravity data and facial expressions are beyond the normal range, it means that the driver is at risk of abnormal conditions, and an early warning is issued at this time; in addition, it is dangerous driving for the driver to take both hands off the steering wheel of the vehicle while driving. At this time, since the data sensor device in the steering wheel of the vehicle needs to directly contact the hands to collect data, the driver's hands off the steering wheel of the vehicle during driving will cause the data sensor device to not collect data, and an early warning will also be triggered to remind the driver to avoid dangerous driving; The emergency avoidance module turns on the vehicle's hazard warning lights when an early warning is issued; if the driver does not process the early warning within a preset time, a preset phone number is dialed to inform the other party of the current location, preset sentence and the health data.

[0039] It should be noted that the hazard warning flasher is also the vehicle's "double flash" warning light, which is used to alert surrounding vehicles; In this embodiment, the driver's current driving status of the vehicle is obtained through the vital sign acquisition module and the image recognition module. When the early warning prompt module recognizes that the driver's health data, gravity data and facial expression are beyond the normal range, it means that the driver is at risk of abnormal conditions such as myocardial infarction. At this time, if the driver is still conscious, the early warning is turned off, and the driver drives the vehicle to the side of the road to wait for rescue according to his own situation. He can also go to the hospital for treatment according to the planned route given by the vehicle. If the driver can no longer effectively control his body and cannot turn off the early warning, it means that the driver is in a bad state. He directly calls the preset phone number and informs the driver of the current location and the preset statement through the vehicle GPS. The other party can also receive the health data and send it to the rescue personnel for reference, so as to prepare first aid tools for corresponding diseases in advance. Through the vehicle steering wheel and the vehicle seat cushion, the system can directly contact the driver to obtain a variety of accurate driver vital signs data. The system can identify whether the driver has uncomfortable facial expressions in combination with the image recognition module, and jointly judge whether the driver is at risk of abnormal state. When there is a risk of abnormal state, an early warning will be issued to remind the driver that the body may be abnormal and that driving the vehicle should be avoided as much as possible. If the driver can no longer control the body normally, a request for rescue will be made directly, including calling relatives and friends and the police, so as to treat the driver as soon as possible and effectively reduce the loss.

[0040] In one embodiment of the present application, the emergency avoidance module also includes a lane avoidance sub-module, and the image acquisition device also includes identifying the road on which the vehicle is traveling. When the driver does not process the warning within a preset time, the current lane of the vehicle is determined, the steering wheel angle of the vehicle is controlled, and the vehicle is driven toward the emergency lane or the edge of the road, and stops after entering the predetermined lane.

[0041] In this embodiment, considering that the driver may not be able to control the vehicle when an abnormality occurs, it may cause the vehicle to lose control, posing a threat to the driver, passengers, and vehicles and pedestrians around the vehicle. Therefore, when the driver fails to process the warning within a preset time, the vehicle is controlled to enter the emergency lane or the edge of the road, and then stops to wait for rescue, which further ensures that the vehicle does not lose control when the driver suddenly fails to control the vehicle. Specifically, the image acquisition device identifies the road and specific lane where the vehicle is traveling, and adjusts the angle of the vehicle's steering wheel according to the distance from the emergency lane or the edge of the road, so that the vehicle is changed to a relatively safe lane. Stopping on the roadside to wait for rescue greatly reduces the risk of vehicle collision.

[0042] In one embodiment of the present application, the image recognition module also includes a personalization submodule, which establishes or retrieves the identity information of the current driver based on the facial features, and the identity information includes the driver image, and the health data and gravity data of the driver's driving history of the current vehicle, to form a health record of the driver, and the driver's gender and height data in the health record are obtained through the driver image, and the weight data is obtained through the gravity data, to obtain the normal range of the driver's current health data and gravity data.

[0043] It should be noted that the normal range of data for different groups of people is different. For example, the standard blood pressure for different genders and ages is different, and the standard weight for people of different heights is different. When a driver is overweight, he or she is more likely to have a heart attack. In this embodiment, since the normal ranges of the health data and gravity data are different for different drivers, a personalized module is established to create a file for the driver driving the vehicle. The identity information of the current driver is obtained by collecting facial features based on the image acquisition device, and the normal ranges of the current health data and gravity data are derived based on the driver's own identity information. The early warning module will only issue an early warning when it recognizes that the data exceeds the normal range corresponding to the current driver. Furthermore, the driver's medical history can also be entered in the identity information, and the normal ranges of the health data and gravity data can be adjusted according to the previous medical history. When the driver feels unwell, the driver can be provided with suggestions for alleviating the condition through voice based on the treatment methods in the previous medical history.

[0044] In one embodiment of the present application, the image recognition module also includes a state recognition submodule. By classifying facial expressions, different facial expressions correspond to different states, and corresponding processing is performed for different states. The facial expressions include calmness, fatigue and anger. The calm driving state is not processed; the processing performed in the fatigued driving state includes voice reminders, vehicle seat vibrations and vehicle steering wheel vibrations; the processing performed in the angry driving state includes voice soothing and speed limit.

[0045] In one embodiment of the present application, the state recognition submodule also includes a first pre-recognition submodule, which recognizes the facial features and health data of the driver when the driver enters the vehicle. When it is recognized that the driver's current mental state and / or health data are not good, a reminder is given in advance to rest and avoid driving for a long time.

[0046] In one embodiment of the present application, the state recognition submodule also includes a second pre-recognition submodule. When the image acquisition device captures a situation that may cause the driver to drive in an angry state during driving, the driver is calmed in advance. The situations that may cause the driver to drive in an angry state include early driving, traffic jams, other vehicles cutting in, the driver's facial expression is serious, frowning, wrinkling the nose and squinting, and the driver speaks in an increased volume.

[0047] In this embodiment, it is considered that not only the driver's illness that makes it difficult to move requires emergency treatment, but also when the driver is driving fatigued or angry, emergency treatment is also required according to the driver's state. For example, in the state of fatigue driving, voice reminders, playing music with a strong sense of rhythm, etc. are used to make the driver concentrate on driving the vehicle; for example, the corresponding treatment in the angry driving state includes corresponding voice soothing and speed limit to prevent the driver from speeding due to fighting with other vehicles; for example, in the state of driving due to physical discomfort, when the image acquisition device recognizes that the driver's face is in pain and the mouth and nose are breathing heavily, it can also be combined with gravity data. When it is recognized that the driver is sitting in an irregular posture, the driver is reminded to stop driving the vehicle and asked whether the old disease has recurred, and the corresponding relief method is provided; if the driver still cannot concentrate on driving the vehicle within the preset time, it also means that the driver is at risk of abnormal conditions, and early warning and risk avoidance operations are also performed; further, when the driver is about to start When the vehicle is detected, it is judged whether the driver is driving fatigued, including whether there are "dark circles" and whether the pulse is accelerated, and it is judged that the driver may be tired in the next driving process, and the driver is reminded in advance, and a parking and rest location is planned, such as a parking lot, a highway service area, and a mandatory rest time is determined according to the driver's state; at the same time, the image recognition module inputs images that may cause the driver to be angry driving state for training, and when the data acquisition device collects, for example, early driving, traffic jams, other vehicles cutting in, the driver's facial expression is serious, frowning, wrinkling the nose, squinting, and the driver's voice volume increases, the driver is comforted in advance to avoid the driver being in an angry state and difficult to accept any comfort; further, personalized settings can be made according to the reasons why the driver was angry when driving the vehicle in the past. When the data acquisition device recognizes the same situation that caused the driver to be angry as before, the driver is comforted in advance.

[0048] In one embodiment of the present application, the data sensing device includes a temperature collection device disposed on the vehicle steering wheel. When the driver's hand touches the vehicle steering wheel, the hand temperature is collected to determine whether to turn on the air conditioner or the heater.

[0049] In one embodiment of the present application, the data sensing device includes a gravity sensing device and / or a humidity sensing device arranged in the steering wheel of the vehicle. The humidity sensing device determines whether the driver's hands are sweating, the gravity sensing device determines whether the driver's hands are shaking, and the sitting posture data determines whether the body is shaking. When the driver's hands are sweating, or his hands and body are shaking, and the image sensing device recognizes that the driver's facial expression is nervous or fearful, it is determined that there is danger in the car, and the vehicle's position and the audio and images in the car are synchronized to the outside world.

[0050] In one embodiment of the present application, the gravity sensor device is provided in the vehicle steering wheel, and different pressing times and numbers of pressing the vehicle steering wheel correspond to different information. When it is judged that danger occurs in the vehicle, the driver transmits corresponding information to the outside world by pressing the vehicle steering wheel for different times.

[0051] In this embodiment, considering that the driver's body temperature is cold, it is easy to cause stiff hands and slow reaction, and considering that the driver's body temperature is high, it is easy to cause heatstroke and fainting, so the temperature sensor on the steering wheel of the vehicle is used to identify the temperature of the driver's palms, and when the temperature exceeds the normal value, the cold air or warm air is turned on accordingly; further, considering that the driver may be in good physical condition, but it is difficult to make a normal alarm due to the threat of hijacking and other reasons, emergency treatment is also required. Therefore, by identifying whether the driver is shaking, and combining the image sensor device to identify that the driver's facial expression is nervous or fearful, it is judged that there is a danger in the car, and the vehicle position and the audio and image in the car are synchronized to the outside world, so that the outside world can obtain the situation in the car, which is convenient for rescuing the driver; further, combined with the pressure sensor set on the steering wheel of the vehicle, different information is pre-set, and the driver presses the steering wheel of the vehicle with his fingers for different times and times to achieve simple communication with the outside world. The most important thing is that the communication method is relatively concealed and not easy to be discovered, so that the outside world can understand the situation in the car more clearly, which is more helpful for rescuing the driver.

[0052] In the second aspect of this application, see Figure 4 As shown, it provides intelligent monitoring and safety emergency response methods based on the health status of vehicle drivers, including: Data sensing devices are arranged on the vehicle steering wheel and in the vehicle seat to collect the driver's health data through the driver holding the vehicle steering wheel, and the health data includes heart rate, pulse, temperature and humidity data; at the same time, according to the driver's sitting and lying posture on the vehicle seat, the driver's gravity data on the vehicle seat is collected, and the gravity data includes the driver's weight and sitting posture data; The vehicle is provided with an image acquisition device, the image acquisition device acquires a driver image in real time, obtains the driver's facial features and facial expressions in the driver image, and classifies the facial expressions; Collect the changes in the health data and gravity data, as well as the facial expressions of existing drivers when an abnormality occurs during driving a vehicle, input them into a neural network model for training, and obtain a recognition model that can identify drivers who are at risk of abnormal conditions. When it is identified that the health data, gravity data, and facial expressions of the driver are out of the normal range during vehicle driving, an early warning is issued; When a warning is issued, the vehicle's hazard warning lights are turned on; if the driver does not handle the warning within a preset time, a preset phone number is dialed to inform the other party of the current location, preset sentence and the health data.

[0053] In one embodiment of the present application, the image acquisition device also includes identifying the road on which the vehicle is traveling. When the driver does not process the warning within a preset time, it determines the current lane of the vehicle, controls the steering wheel angle of the vehicle, drives the vehicle to the emergency lane or the edge of the road, and stops after entering the predetermined lane.

[0054] In one embodiment of the present application, the identity information of the current driver is established or retrieved based on the facial features, and the identity information includes the driver image, and the health data and gravity data of the driver's driving history of the current vehicle, forming a health record of the driver. The driver's gender and height data are obtained from the health record through the driver image, and the weight data is obtained through the gravity data, thereby obtaining the normal range of the driver's current health data and gravity data.

[0055] In one embodiment of the present application, by classifying facial expressions, different facial expressions correspond to different states, and corresponding processing is performed for different states. The facial expressions include calmness, fatigue and anger. The calm driving state is not processed; the processing performed in the fatigued driving state includes voice reminders, vehicle seat vibrations and vehicle steering wheel vibrations; the processing performed in the angry driving state includes voice soothing and speed limit.

[0056] In one embodiment of the present application, when the driver enters the vehicle, the facial features and health data of the driver are identified. When it is identified that the driver's current mental state is not good and / or the health data is not good, a reminder is given in advance to rest and avoid driving for a long time.

[0057] In one embodiment of the present application, when the image acquisition device captures situations that may cause the driver to drive in anger during driving, the driver is calmed in advance. The situations that may cause the driver to drive in anger include early driving, traffic jams, other vehicles cutting in, the driver's facial expression is serious, frowning, wrinkling the nose and squinting, and the driver speaks in an increased volume.

[0058] In one embodiment of the present application, a temperature collection device on the vehicle steering wheel collects the temperature of the driver's hand when the driver's hand touches the vehicle steering wheel to determine whether to turn on the cooling or heating.

[0059] In one embodiment of the present application, a gravity sensor device and / or a humidity sensor device is further provided in the steering wheel of the vehicle, the humidity sensor device determines whether the driver's hands are sweating, the gravity sensor device determines whether the driver's hands are shaking, and the sitting posture data determines whether the body is shaking. When the driver's hands are sweating, or his hands and body are shaking, and the image sensor device recognizes that the driver's facial expression is nervous or fearful, it is determined that there is danger in the car, and the vehicle's position and the audio and images in the car are synchronized to the outside world.

[0060] In one embodiment of the present application, the gravity sensor device is provided in the vehicle steering wheel, and different pressing times and numbers of pressing the vehicle steering wheel correspond to different information. When it is judged that danger occurs in the vehicle, the driver transmits corresponding information to the outside world by pressing the vehicle steering wheel for different times.

[0061] It should be noted that the specific implementation method of the intelligent monitoring and safety emergency system based on the health status of the vehicle driver in the embodiment of the present application refers to the specific implementation method of the intelligent monitoring and safety emergency system based on the health status of the vehicle driver proposed in the first aspect of the embodiment of the present application, and will not be repeated here.

[0062] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that an article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such article or device. In the absence of more restrictions, the elements defined by the sentence "includes..." do not exclude the existence of other identical elements in the article or device including the elements.

[0063] The above is a detailed introduction to the intelligent monitoring and safety emergency response system based on the health status of the vehicle driver. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the intelligent monitoring and safety emergency response system based on the health status of the vehicle driver and its core idea of ​​the present application; at the same time, for general technical personnel in this field, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. Intelligent monitoring and safety emergency system based on the health status of vehicle drivers, characterized by: include: The vital sign acquisition module is provided with data sensing devices on the vehicle steering wheel and in the vehicle seat, and the driver's health data is collected by the driver holding the vehicle steering wheel, and the health data includes heart rate, pulse, temperature and humidity data; at the same time, the driver's gravity data on the vehicle seat is collected according to the driver's sitting and lying posture on the vehicle seat, and the gravity data includes the driver's weight and sitting posture data; An image recognition module, wherein the vehicle is provided with an image acquisition device, the image acquisition device acquires a driver image in real time, obtains the driver's facial features and facial expressions in the driver image, and classifies the facial expressions; The early warning module collects the changes in the health data and the gravity data, as well as the facial expressions of the driver when an abnormality occurs during the driving of the vehicle, and inputs them into the neural network model for training to obtain a recognition model that can identify the driver's risk of abnormal state. When the health data, gravity data and facial expressions of the driver are identified to be out of the normal range during the driving process of the vehicle, an early warning is issued; The emergency avoidance module turns on the vehicle's hazard warning lights when an early warning is issued; If the driver does not process the warning within the preset time, the preset phone number will be dialed to inform the other party of the current location, preset sentence and the health data.

2. The intelligent monitoring and safety emergency system based on the health status of vehicle drivers according to claim 1 is characterized in that: The emergency avoidance module also includes a lane avoidance submodule, and the image acquisition device also includes identifying the road on which the vehicle is traveling. When the driver does not process the warning within a preset time, the current lane of the vehicle is determined, the steering wheel angle of the vehicle is controlled, the vehicle is driven to the emergency lane or the edge of the road, and stops after entering the predetermined lane.

3. The intelligent monitoring and safety emergency system based on the health status of vehicle drivers according to claim 1 is characterized in that: The image recognition module also includes a personalization submodule, which establishes or retrieves the identity information of the current driver based on the facial features, the identity information including the driver image, and the health data and gravity data of the driver's driving history of the current vehicle, to form a health record of the driver, and obtains the driver's gender and height data from the health record through the driver image, and obtains the weight data through the gravity data, to derive the normal range of the driver's current health data and gravity data.

4. The intelligent monitoring and safety emergency system based on the health status of vehicle drivers according to claim 1 is characterized in that: The image recognition module also includes a state recognition submodule, which classifies facial expressions. Different facial expressions correspond to different states, and corresponding processing is performed for different states. The facial expressions include calmness, fatigue and anger. The calm driving state is not processed; the processing performed in the fatigued driving state includes voice reminders, vehicle seat vibrations and vehicle steering wheel vibrations; the processing performed in the angry driving state includes voice soothing and speed limit.

5. The intelligent monitoring and safety emergency system based on the health status of vehicle drivers according to claim 4 is characterized in that: The state recognition submodule also includes a first pre-recognition submodule, which recognizes the driver's facial features and health data when the driver enters the vehicle. When it is recognized that the driver's current mental state and / or health data are not good, it prompts the driver to rest and avoid driving for a long time in advance.

6. The intelligent monitoring and safety emergency system based on the health status of vehicle drivers according to claim 4 is characterized in that: The state recognition submodule also includes a second pre-recognition submodule. When the image acquisition device captures a situation that may cause the driver to drive in anger during driving, the driver is comforted in advance. The situations that may cause the driver to drive in anger include early driving, traffic jams, other vehicles cutting in, the driver's facial expression is serious, frowning, wrinkling the nose and squinting, and the driver speaks in an increased volume.

7. The intelligent monitoring and safety emergency system based on the health status of the vehicle driver according to any one of claims 1 to 6, characterized in that: The data sensing device includes a temperature collecting device arranged on the vehicle steering wheel. When the driver's hand touches the vehicle steering wheel, the hand temperature is collected to determine whether to turn on the cooling or heating.

8. The intelligent monitoring and safety emergency system based on the health status of vehicle drivers according to claim 7 is characterized in that: The data sensing device includes a gravity sensing device and / or a humidity sensing device arranged in the steering wheel of the vehicle. The humidity sensing device determines whether the driver's hands are sweating, the gravity sensing device determines whether the driver's hands are shaking, and the sitting posture data determines whether the body is shaking. When the driver's hands are sweating, or the hands and body are shaking, and the image sensing device is combined with the driver's facial expression that is nervous or fearful, it is determined that there is danger in the car, and the vehicle position and the audio and image in the car are synchronized to the outside world.

9. The intelligent monitoring and safety emergency system based on the health status of vehicle drivers according to claim 8 is characterized in that: The gravity sensor device is arranged in the vehicle steering wheel. Different pressing times and pressing times of the vehicle steering wheel correspond to different information. When it is judged that danger occurs in the vehicle, the driver transmits corresponding information to the outside world by pressing the vehicle steering wheel for different times.

Images