Driver state monitoring method, device, equipment and medium

Abstract

The invention discloses a driver state monitoring method, device and equipment and a medium. The method comprises the following steps: obtaining a state monitoring result according to visual observation data, contact observation data, electrocardiogram observation data and infrared observation data; when the state monitoring result is a fatigue state, the upper limit value of the vehicle speed is adjusted, and rest prompt information is output; when the state monitoring result is a sudden physical state, parking prompt information is output, and if it is detected that the driver does not execute parking operation within the target duration, position information of the vehicle and visual observation data, contact observation data, electrocardio observation data and infrared observation data of the driver are sent to an emergency contact user and a rescue user. According to the embodiment of the invention, the state of the driver can be comprehensively and accurately monitored based on the multi-dimensional data, vehicle speed control and warning are carried out in time when the driver is in a fatigue state, and warning and rescue are carried out in time when the driver is in a sudden physical state.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus, device, and medium for monitoring driver status. Background Technology

[0002] During vehicle operation, if the driver is fatigued or experiences a sudden physical distress, it can pose significant safety hazards. When a driver is fatigued, they may experience decreased concentration, slowed reaction time, and sluggish operation, increasing the risk of road accidents. In the event of a driver experiencing a sudden physical distress, timely warnings and assistance are necessary. To ensure the safety of both the driver and the vehicle, driver status monitoring is essential to detect fatigue or sudden physical distress, and timely intervention should be implemented when such conditions are detected.

[0003] In related technologies, a common driver status monitoring scheme involves the vehicle's controller monitoring the driver's status based on in-vehicle images during vehicle operation. This monitors for driver fatigue and alerts the driver when fatigue is detected. However, this method relies on in-vehicle images, resulting in a limited monitoring scope. It is susceptible to influences from the vehicle environment and driver emotions, failing to provide comprehensive and accurate driver status monitoring and hindering timely alerts and assistance in cases of sudden driver health issues. Summary of the Invention

[0004] This invention provides a driver status monitoring method, device, equipment, and medium to solve the problem that existing driver status monitoring schemes rely on in-vehicle images to monitor the driver's status. However, these schemes have limited monitoring dimensions and are easily affected by factors such as the vehicle environment and the driver's emotions. Consequently, they cannot comprehensively and accurately monitor the driver's status and cannot provide timely warnings and assistance when the driver experiences a sudden physical condition.

[0005] According to one aspect of the present invention, a driver status monitoring method is provided, comprising:

[0006] The data acquisition component collects the driver's visual observation data, tactile observation data, electrocardiogram observation data, and infrared observation data.

[0007] Through the data analysis component, the system periodically detects whether the driver is in a state of fatigue or sudden physical condition based on the driver's visual observation data, contact observation data, electrocardiogram observation data and infrared observation data, and obtains the driver's condition monitoring results.

[0008] When the driver's condition monitoring result indicates fatigue, the vehicle's speed limit is adjusted through the action execution component, and a rest reminder message is output.

[0009] When the driver's status monitoring result indicates a sudden physical condition, the action execution component outputs a parking prompt message and detects whether the driver performs a parking operation. If the driver does not perform a parking operation within the target time period, the vehicle's location information and the driver's visual observation data, contact observation data, electrocardiogram observation data, and infrared observation data are sent to the emergency contact user and the rescue user.

[0010] According to another aspect of the present invention, a driver status monitoring device is provided, comprising:

[0011] The data acquisition module is used to collect the driver's visual observation data, tactile observation data, electrocardiogram observation data, and infrared observation data through the data acquisition components.

[0012] The monitoring module is used to periodically detect whether the driver is fatigued or experiencing a sudden physical condition based on the driver's visual observation data, contact observation data, electrocardiogram observation data, and infrared observation data through the data analysis component, and obtain the driver's condition monitoring results.

[0013] The first state processing module is used to adjust the vehicle's speed limit and output a rest reminder message when the driver's state monitoring result indicates fatigue.

[0014] The second state processing module is used to output a parking prompt message through the action execution component when the driver's state monitoring result indicates a sudden physical condition, and to detect whether the driver has performed a parking operation. If the driver has not performed a parking operation within the target time period, the module will send the vehicle's location information and the driver's visual observation data, contact observation data, electrocardiogram observation data and infrared observation data to the emergency contact user and the rescue user.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] At least one processor;

[0017] and a memory communicatively connected to the at least one processor;

[0018] The memory stores a computer program that is executed by the at least one processor, which enables the at least one processor to perform the driver state monitoring method according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the driver state monitoring method according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the driver state monitoring method according to any embodiment of the present invention.

[0021] The technical solution of this invention collects driver's visual observation data, contact observation data, electrocardiogram (ECG) observation data, and infrared observation data through a data acquisition component. A data analysis component periodically detects whether the driver is fatigued or experiencing a sudden physical condition based on these data, obtaining driver status monitoring results. When the driver's status monitoring result indicates fatigue, an action execution component adjusts the vehicle's maximum speed and outputs a rest reminder. When the driver's status monitoring result indicates a sudden physical condition, the action execution component outputs a parking reminder and detects whether the driver performs a parking operation. If the driver does not perform a parking operation within a target time period, the vehicle's location information and the driver's visual observation data, contact observation data, ECG observation data, and infrared observation data are sent to emergency contact users and rescue users. This solves the problem that related driver status monitoring schemes rely on in-vehicle images for driver status monitoring, which has a relatively single monitoring dimension and is easily affected by factors such as the vehicle environment and driver emotions, failing to comprehensively and accurately monitor the driver's condition. The problem of insufficient timely alerts and rescues when drivers experience sudden physical distress during status monitoring can be addressed by comprehensively and accurately monitoring driver fatigue or sudden physical distress based on visual observation data, contact observation data, electrocardiogram (ECG) observation data, and infrared observation data. The monitoring results can be used to adjust the vehicle's speed limit and output rest reminders when driver fatigue is detected, enabling timely intervention. Conversely, when a sudden physical distress is detected, a stop reminder can be output, and the system can detect whether the driver stops. If the driver fails to stop within a target timeframe, the vehicle's location information and the driver's visual, contact, ECG, and infrared observation data are sent to emergency contact and rescue personnel, allowing for timely intervention. This multi-dimensional data-driven approach enables comprehensive and accurate driver status monitoring, timely speed control and alerts when the driver is fatigued, and timely alerts and rescues when the driver experiences a sudden physical distress.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of a driver status monitoring method provided in Embodiment 1 of the present invention.

[0025] Figure 2 This is a flowchart of a driver status monitoring method provided in Embodiment 2 of the present invention.

[0026] Figure 3 This is a schematic diagram of a driver status monitoring device provided in Embodiment 3 of the present invention.

[0027] Figure 4 A schematic diagram of the structure of an electronic device for implementing the driver status monitoring method of this invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "target," "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising," "including," and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices. The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solutions of this disclosure all comply with relevant laws and regulations and do not violate public order and good morals.

[0030] Example 1

[0031] Figure 1 This is a flowchart illustrating a driver state monitoring method according to Embodiment 1 of the present invention. This embodiment is applicable to monitoring the state of a vehicle driver, detecting whether the driver is fatigued or experiencing a sudden physical condition, and intervening promptly when fatigue or a sudden physical condition is detected. This method can be executed by a driver state monitoring device, which can be implemented in hardware and / or software and can be configured in the vehicle's overall controller. Figure 1 As shown, the method includes:

[0032] Step 101: Collect the driver's visual observation data, contact observation data, electrocardiogram observation data, and infrared observation data through the data acquisition component.

[0033] Optionally, during vehicle operation, the driver may be in a normal state, a fatigued state, or a sudden physical condition. A driver in a normal state means the driver is driving when both their physical and emotional state are good. A driver in a fatigued state means the driver is driving after a long period of driving or without sufficient rest. A driver experiencing a sudden physical condition means the driver is driving when their physical and emotional state is abnormal due to a sudden illness.

[0034] Optionally, driver visual observation data can refer to driver-related data determined from video or images of the driver captured by cameras installed in the vehicle with the driver's authorization. Visual observation data may include blink frequency, eyelid closure, yawn frequency, and facial expression. Blink frequency can refer to the number of times the driver blinks per unit of time. Eyelid closure can refer to the percentage of the driver's pupil covered by their eyelids. Yawn frequency can refer to the number of times the driver yawns per unit of time. Facial expression can be text describing the driver's facial expressions. The unit of time can be 1 minute.

[0035] Optionally, driver contact observation data can refer to data related to the driver's vital signs measured by sensors located on the vehicle's steering wheel and in contact with the driver, with the driver's authorization. These sensors on the steering wheel can be sensors used to measure heart rate, blood oxygen saturation, and blood pressure. Driver contact observation data can include the driver's heart rate, blood oxygen saturation, and blood pressure.

[0036] Optionally, driver ECG observation data can refer to driver-related data determined based on ECG signals measured by ECG sensors installed in the vehicle, with the driver's authorization. Driver ECG observation data may include the driver's electrocardiogram (ECG).

[0037] Optionally, the driver's infrared observation data can refer to driver-related data measured by infrared sensors installed on the vehicle with the driver's authorization. The driver's electrocardiogram observation data may include the driver's body temperature.

[0038] The vehicle's driver authorizes the hardware and software modules installed on the vehicle to collect, store, use, process, transmit, provide, and disclose video and images, electrocardiogram signals, visual observation data, contact observation data, electrocardiogram observation data, and infrared observation data containing the driver.

[0039] Optionally, the data acquisition component can be a hardware module installed in the vehicle for collecting the driver's visual observation data, contact observation data, electrocardiogram (ECG) observation data, and infrared observation data. The data acquisition component may include a visual acquisition sub-component, a contact acquisition sub-component, an ECG acquisition sub-component, and an infrared acquisition sub-component. The visual acquisition sub-component can be a hardware module installed in the vehicle for collecting the driver's visual observation data. The contact acquisition sub-component can be a hardware module installed in the vehicle for collecting the driver's contact observation data. The ECG acquisition sub-component can be a hardware module installed in the vehicle for collecting the driver's electrocardiogram (ECG) observation data. The infrared acquisition sub-component can be a hardware module installed in the vehicle for collecting the driver's infrared observation data.

[0040] Optionally, the data acquisition component collects the driver's visual observation data, contact observation data, electrocardiogram (ECG) observation data, and infrared observation data, including: collecting the driver's visual observation data through a visual acquisition sub-component; wherein the visual observation data includes blink frequency, eyelid closure degree, yawn frequency, and facial expression state; collecting the driver's contact observation data through a contact acquisition sub-component; wherein the contact observation data includes heart rate, blood oxygen, and blood pressure; collecting the driver's ECG observation data through an ECG acquisition sub-component; wherein the ECG observation data includes an electrocardiogram; and collecting the driver's infrared observation data through an infrared acquisition sub-component; wherein the infrared observation data includes body temperature.

[0041] Optionally, the visual acquisition sub-component can analyze and detect videos or images of the driver captured by the cameras installed on the vehicle with the driver's authorization at a preset frequency during vehicle operation, in order to determine the driver's blinking frequency, eyelid closure degree, yawning frequency and facial expression state, thereby collecting the driver's visual observation data at regular intervals.

[0042] Optionally, the contact acquisition sub-component can measure the driver's heart rate, blood oxygen, and blood pressure at a preset frequency during vehicle operation, with the driver's authorization, through sensors located on the vehicle's steering wheel that are in contact with the driver, thereby periodically collecting the driver's contact observation data.

[0043] Optionally, the ECG acquisition sub-component can analyze and detect the ECG signals measured by the ECG sensors installed on the vehicle at a preset frequency during vehicle operation, with the driver's authorization, to determine the driver's ECG and thus collect the driver's ECG observation data at regular intervals.

[0044] Optionally, the infrared acquisition sub-component can measure the driver's body temperature at a preset frequency using an infrared sensor installed on the vehicle, with the driver's authorization, thereby periodically collecting the driver's infrared observation data.

[0045] Step 102: Using the data analysis component, the driver is periodically monitored for fatigue or sudden physical condition based on the driver's visual observation data, contact observation data, electrocardiogram observation data, and infrared observation data, and the driver's condition monitoring results are obtained.

[0046] Optionally, the driver's condition monitoring results can be information obtained after monitoring the driver's condition to characterize whether the driver is fatigued or experiencing a sudden physical condition. The driver's condition monitoring results can be categorized as normal, fatigued, or experiencing a sudden physical condition. A normal driver condition indicates that the driver is in a normal state. A fatigued driver condition indicates that the driver is fatigued. A sudden physical condition indicates that the driver is experiencing a sudden physical condition.

[0047] Optionally, driver visual observation data, contact observation data, electrocardiogram (ECG) observation data, and infrared observation data can be used to assess whether the driver is fatigued or experiencing a sudden physical condition. Typically, when a driver is fatigued, their blinking frequency slows down evenly, their eyes remain closed for extended periods, and they yawn a set number of times. Simultaneously, the driver may experience a decrease in low-frequency heart rate variability, reduced blood oxygen levels, and increased blood pressure fluctuations. When a driver experiences a sudden physical condition, they may exhibit postural deviation, pain, and difficulty breathing.

[0048] Optionally, the data analysis component can be a software or hardware module installed in the vehicle to detect whether the driver is fatigued or experiencing a sudden physical condition based on the driver's visual observation data, contact observation data, electrocardiogram (ECG) observation data, and infrared observation data, and to obtain the driver's condition monitoring results. The input to the data analysis component is the driver's visual observation data, contact observation data, ECG observation data, and infrared observation data collected within a preset time period prior to the current moment. The preset time period can be a pre-set duration. For example, the preset time period can be 3 minutes, 5 minutes, or 10 minutes. The output of the data analysis component is the driver's condition monitoring results. The driver's visual observation data, contact observation data, ECG observation data, and infrared observation data collected within the preset time period prior to the current moment can be input into the data analysis component. The data analysis component will analyze and detect the input driver's visual observation data, contact observation data, ECG observation data, and infrared observation data collected within the preset time period prior to the current moment to determine whether the driver is in a normal state, a fatigued state, or a sudden physical condition. If the data analysis component determines that the driver is in a normal state, it will record the driver's state monitoring result as normal and output the result. If the data analysis component determines that the driver is fatigued, it will record the driver's state monitoring result as fatigued and output the result. If the data analysis component determines that the driver is experiencing a sudden physical emergency, it will record the driver's state monitoring result as a sudden physical emergency and output the result. The driver's state monitoring result output by the data analysis component can be obtained to determine the driver's overall state.

[0049] Optionally, the data analysis component periodically detects whether the driver is in a state of fatigue or sudden physical condition based on the driver's visual observation data, contact observation data, electrocardiogram observation data, and infrared observation data, and obtains the driver's state monitoring results. This includes periodically inputting the driver's visual observation data, contact observation data, electrocardiogram observation data, and infrared observation data collected within a preset time period before the current moment into the data analysis component, and obtaining the driver's state monitoring results output by the data analysis component; wherein, the driver's state monitoring results are normal state, fatigue state, or sudden physical condition.

[0050] Optionally, during vehicle operation, at preset time intervals, the driver's visual observation data, contact observation data, electrocardiogram (ECG) observation data, and infrared observation data collected within a preset duration prior to the current moment can be input into the data analysis component to obtain the driver's status monitoring results output by the data analysis component. The preset time interval can be a pre-set time interval. For example, the preset time interval could be 3 minutes, 5 minutes, or 10 minutes. The driver's visual observation data, contact observation data, ECG observation data, and infrared observation data collected by the data acquisition component within the preset duration prior to the current moment can be input into the data analysis component. The data analysis component will analyze and detect the input driver's visual observation data, contact observation data, ECG observation data, and infrared observation data collected by the data acquisition component within the preset duration prior to the current moment to determine whether the driver is in a normal state, a fatigued state, or a sudden physical condition, determine the driver's status monitoring results, and then output the driver's status monitoring results. The driver's status monitoring results output by the data analysis component can be obtained to obtain the driver's status monitoring results.

[0051] Step 103: When the driver's condition monitoring result indicates fatigue, the vehicle speed limit is adjusted through the action execution component, and a rest reminder message is output.

[0052] Optionally, the action execution component can be a software or hardware module installed on the vehicle to perform intervention operations when the driver is detected to be in a state of fatigue or a sudden physical condition.

[0053] Optionally, when the driver's condition monitoring result indicates fatigue, indicating that the driver is fatigued, the action execution component adjusts the vehicle's maximum speed limit and outputs a rest reminder message. This allows the action execution component to perform intervention operations specifically for the driver's fatigue condition. The vehicle's maximum speed limit can be the maximum speed the vehicle can reach, stored in the vehicle's database. The rest reminder message can be a pre-set message to prompt the driver to rest.

[0054] Optionally, when the driver's condition monitoring result indicates fatigue, the action execution component adjusts the vehicle's maximum speed and outputs a rest reminder message. This includes: when the driver's condition monitoring result indicates fatigue, the action execution component updates the vehicle's maximum speed to a preset speed value, controls the display device to display the rest reminder message, and controls the audible and visual alarm device to output an alarm message. The preset speed value can be a pre-set value that the vehicle's speed must be less than or equal to when the driver is fatigued. The display device can be a display screen installed on the vehicle. The audible and visual alarm device can be a buzzer and indicator lights installed on the vehicle.

[0055] Optionally, the action execution component updates the vehicle's maximum speed to a preset speed value, controls the display device to display a rest reminder message, and controls the audible and visual alarm device to output alarm information. This includes: performing the following operations through the action execution component: updating the vehicle's maximum speed stored in the vehicle to a preset speed value; controlling the display device to display a rest reminder message; controlling the buzzer installed on the vehicle to emit a buzzing sound; and controlling the indicator light installed on the vehicle to flash.

[0056] Step 104: When the driver's status monitoring result indicates a sudden physical condition, the action execution component outputs a parking prompt message and detects whether the driver performs a parking operation. If the driver does not perform a parking operation within the target time period, the vehicle's location information and the driver's visual observation data, contact observation data, electrocardiogram observation data, and infrared observation data are sent to the emergency contact user and the rescue user.

[0057] Optionally, when the driver's status monitoring result indicates a sudden physical condition, it means that the driver has been detected to be in a sudden physical condition. The action execution component outputs a parking prompt message and checks whether the driver performs a parking maneuver. If the driver does not perform a parking maneuver within a target time period, the vehicle's location information and the driver's visual observation data, contact observation data, ECG observation data, and infrared observation data are sent to the emergency contact user and the rescue user. The action execution component then performs intervention operations for the driver's sudden physical condition. The parking prompt message can be a pre-set message to remind the driver to stop driving, confirm their current physical condition, and take timely rest and check. The vehicle's location information can be the latest latitude and longitude coordinates measured by the vehicle's positioning module. The vehicle's positioning module can be a hardware module installed on the vehicle to measure the vehicle's latitude and longitude coordinates. The emergency contact user can be a person designated by the driver to handle the driver's sudden physical condition. The rescue user can be a technician used to rescue the driver in a sudden physical condition.

[0058] Optionally, when the driver's condition monitoring result indicates a sudden physical condition, the action execution component outputs a parking prompt message and detects whether the driver performs a parking operation. If the driver does not perform a parking operation within the target time period, the vehicle's location information and the driver's visual observation data, contact observation data, ECG observation data, and infrared observation data are sent to the emergency contact user and the rescue user. This includes: when the driver's condition monitoring result indicates a sudden physical condition, the action execution component outputs a parking prompt message and detects whether the driver performs a parking operation; if the driver does not perform a parking operation within the target time period, the vehicle's location information and the latest collected driver's visual observation data, contact observation data, ECG observation data, and infrared observation data are sent to the emergency contact user and the rescue user based on the communication information between the emergency contact user and the rescue user. The target time period can be a preset duration. For example, the target time period is 1 minute or 3 minutes.

[0059] Optionally, the action execution component outputs a parking prompt message and detects whether the driver has performed a parking operation, including: the action execution component performs the following operations: controls the display device to display a rest prompt message and continuously detects whether the vehicle speed has decreased to zero; if the vehicle speed is detected to have decreased to zero before the continuous detection period reaches the target period, it is determined that the driver has performed a parking operation within the target period, and the detection of whether the vehicle speed has decreased to zero is stopped; if the vehicle speed is not detected to have decreased to zero after the continuous detection period reaches the target period, it is determined that the driver has not performed a parking operation within the target period, and the detection of whether the vehicle speed has decreased to zero is stopped.

[0060] Optionally, if it is detected that the driver has performed a parking operation within the target time period, it is determined that the driver has viewed the parking prompt information, stops driving according to the parking prompt information, confirms the current physical condition, and promptly checks and rests, thus determining that the process of performing intervention and handling operations for the driver's sudden physical condition has ended.

[0061] Optionally, the vehicle controller stores communication information for emergency contact users and rescue users. The communication information for emergency contact users can be used to establish a communication connection with the terminal equipment used by the emergency contact user. The communication information for rescue users can also be used to establish a communication connection with the terminal equipment used by the rescue user. A communication connection can be established with the terminal equipment used by the emergency contact user based on the communication information, and information exchange can be conducted based on the established communication connection. Similarly, a communication connection can be established with the terminal equipment used by the rescue user based on the communication information, and information exchange can be conducted based on the established communication connection.

[0062] Optionally, if the driver is detected not to have stopped within the target time period, a communication connection is established with the terminal device used by the emergency contact user based on the communication information. The vehicle's location information and the latest collected visual observation data, contact observation data, ECG observation data, and infrared observation data of the driver are sent to the emergency contact user's terminal device. Similarly, based on the communication information of the rescue user, a communication connection is established with the terminal device used by the rescue user. The vehicle's location information and the latest collected visual observation data, contact observation data, ECG observation data, and infrared observation data of the driver are sent to the rescue user's terminal device. This allows the emergency contact user and the rescue user to determine that the driver is in a sudden physical emergency and to provide assistance. Thus, an alarm is triggered when a driver is in a sudden physical emergency, and the emergency contact user and the rescue user provide assistance, enabling timely alarm and rescue when a driver is in a sudden physical emergency.

[0063] Optionally, before collecting the driver's visual observation data, contact observation data, electrocardiogram observation data, and infrared observation data through the data acquisition component, the method may also include: detecting whether the driver has set up emergency contact user communication information; if it is detected that the driver has not set up emergency contact user communication information, the driver is prompted to set up emergency contact user communication information.

[0064] Optionally, the detection of whether the driver has set up emergency contact user communication information includes: after the vehicle controller is powered on, detecting whether the vehicle controller stores emergency contact user communication information; if the vehicle controller stores emergency contact user communication information, it is determined that the driver has set up emergency contact user communication information, and the detection process ends; if the vehicle controller does not store emergency contact user communication information, it is determined that the driver has not set up emergency contact user communication information.

[0065] Optionally, the system prompts the driver to set up emergency contact user communication information, including displaying an emergency setting prompt message on the control display device. The emergency setting prompt message can be information used to prompt the driver to enter the emergency contact user's communication information. The driver can enter the emergency contact user's communication information on the display device. The system can acquire the emergency contact user's communication information entered by the driver on the display device, store the emergency contact user's communication information in the vehicle's overall controller, confirm that the driver has set up the emergency contact user's communication information, and determine that the current testing process is complete.

[0066] Optionally, it also includes: when the driver's status monitoring result is normal, the control display device displays the driver's visual observation data, contact observation data, electrocardiogram observation data and infrared observation data collected within a preset time period before the current moment.

[0067] Optionally, when the driver's condition monitoring result is normal, it indicates that the driver is not fatigued or experiencing a sudden physical condition. The display device will only show the driver's visual observation data, contact observation data, electrocardiogram observation data, and infrared observation data collected within a preset time period before the current moment.

[0068] The technical solution of this invention collects driver's visual observation data, contact observation data, electrocardiogram (ECG) observation data, and infrared observation data through a data acquisition component. A data analysis component periodically detects whether the driver is fatigued or experiencing a sudden physical condition based on these data, obtaining driver status monitoring results. When the driver's status monitoring result indicates fatigue, an action execution component adjusts the vehicle's maximum speed and outputs a rest reminder. When the driver's status monitoring result indicates a sudden physical condition, the action execution component outputs a parking reminder and detects whether the driver performs a parking operation. If the driver does not perform a parking operation within a target time period, the vehicle's location information and the driver's visual observation data, contact observation data, ECG observation data, and infrared observation data are sent to emergency contact users and rescue users. This solves the problem that related driver status monitoring schemes rely on in-vehicle images for driver status monitoring, which has a relatively single monitoring dimension and is easily affected by factors such as the vehicle environment and driver emotions, failing to comprehensively and accurately monitor the driver's condition. The problem of insufficient timely alerts and rescues when drivers experience sudden physical distress during status monitoring can be addressed by comprehensively and accurately monitoring driver fatigue or sudden physical distress based on visual observation data, contact observation data, electrocardiogram (ECG) observation data, and infrared observation data. The monitoring results can be used to adjust the vehicle's speed limit and output rest reminders when driver fatigue is detected, enabling timely intervention. Conversely, when a sudden physical distress is detected, a stop reminder can be output, and the system can detect whether the driver stops. If the driver fails to stop within a target timeframe, the vehicle's location information and the driver's visual, contact, ECG, and infrared observation data are sent to emergency contact and rescue personnel, allowing for timely intervention. This multi-dimensional data-driven approach enables comprehensive and accurate driver status monitoring, timely speed control and alerts when the driver is fatigued, and timely alerts and rescues when the driver experiences a sudden physical distress.

[0069] The technical solution of this invention can integrate and process various data that can be used to measure whether a driver is fatigued or experiencing a sudden physical condition, enabling real-time monitoring and recording of the driver's condition and timely intervention in some emergencies. This effectively prevents safety hazards caused by the driver's condition, reduces driving safety risks, and improves road safety.

[0070] The technical solution of this invention determines the driver's state through multi-data fusion analysis, avoiding the problem of data deviation caused by a single monitoring dimension, which is easily affected by factors such as the vehicle environment and the driver's emotions, and thus fails to accurately reflect the driver's state. The technical solution of this invention can control the vehicle speed to a certain extent when the driver is fatigued, reducing the impact of poor physical condition on vehicle control. Furthermore, the technical solution of this invention can promptly send the vehicle's location information and the driver's visual observation data, contact observation data, electrocardiogram observation data, and infrared observation data to emergency contact users and rescue users when the driver experiences a sudden physical emergency, buying time for handling the emergency.

[0071] Example 2

[0072] Figure 2 This is a flowchart illustrating a driver status monitoring method according to Embodiment 2 of the present invention. Embodiments of the present invention can be combined with various optional solutions from one or more of the above embodiments. For example... Figure 2 As shown, the method includes:

[0073] Step 201: Check whether the driver has set up emergency contact information for the user.

[0074] Step 202: After confirming that the driver has set up emergency contact information, collect the driver's visual observation data, contact observation data, electrocardiogram observation data and infrared observation data through the data acquisition component.

[0075] Step 203: Using the data analysis component, the driver is periodically monitored for fatigue or sudden physical condition based on the driver's visual observation data, contact observation data, electrocardiogram observation data, and infrared observation data, and the driver's condition monitoring results are obtained.

[0076] Step 204: When the driver's status monitoring result is normal, the control display device displays the driver's visual observation data, contact observation data, electrocardiogram observation data and infrared observation data collected within a preset time period before the current moment.

[0077] Step 205: When the driver's condition monitoring result indicates fatigue, the vehicle speed limit is adjusted through the action execution component, and a rest reminder message is output.

[0078] Step 206: When the driver's status monitoring result indicates a sudden physical condition, the action execution component outputs a parking prompt message and detects whether the driver performs a parking operation. If the driver does not perform a parking operation within the target time period, the vehicle's location information and the driver's visual observation data, contact observation data, electrocardiogram observation data, and infrared observation data are sent to the emergency contact user and the rescue user.

[0079] The technical solution of this invention, when it is determined that the driver has set up emergency contact information, can comprehensively and accurately monitor whether the driver is in a state of fatigue or sudden physical condition based on visual observation data, contact observation data, electrocardiogram observation data, and infrared observation data. The driver's status monitoring results allow for adjustments to the vehicle's speed limit and the output of rest reminders when driver fatigue is detected. Similarly, when a sudden physical condition is detected, a stop reminder is output, and the system checks whether the driver stops. If the driver does not stop within a target time, the vehicle's location information and the driver's visual, contact, electrocardiogram, and infrared observation data are sent to the emergency contact and rescue users. This enables timely intervention in cases of sudden physical conditions. This multi-dimensional data-driven approach allows for comprehensive and accurate driver status monitoring, enabling timely speed control and alerts when the driver is fatigued, and timely alerts and rescue in cases of sudden physical conditions.

[0080] Example 3

[0081] Figure 3 This is a schematic diagram of a driver status monitoring device provided in Embodiment 3 of the present invention. The device can be configured in an electronic device. Figure 3 As shown, the device includes: a data acquisition module 301, a monitoring module 302, a first state processing module 303, and a second state processing module 304.

[0082] The system includes the following modules: Acquisition module 301, which collects the driver's visual observation data, contact observation data, electrocardiogram (ECG) observation data, and infrared observation data via a data acquisition component; Monitoring module 302, which periodically detects whether the driver is fatigued or experiencing a sudden physical condition based on the driver's visual observation data, contact observation data, ECG observation data, and infrared observation data via a data analysis component, and obtains the driver's status monitoring results; First status processing module 303, which adjusts the vehicle's speed limit and outputs a rest reminder message when the driver's status monitoring result indicates fatigue, via an action execution component; and Second status processing module 304, which outputs a parking reminder message and detects whether the driver performs a parking operation when the driver's status monitoring result indicates a sudden physical condition, and if the driver does not perform a parking operation within a target time period, sends the vehicle's location information and the driver's visual observation data, contact observation data, ECG observation data, and infrared observation data to emergency contact users and rescue users.

[0083] The technical solution of this invention collects driver's visual observation data, contact observation data, electrocardiogram (ECG) observation data, and infrared observation data through a data acquisition component. A data analysis component periodically detects whether the driver is fatigued or experiencing a sudden physical condition based on these data, obtaining driver status monitoring results. When the driver's status monitoring result indicates fatigue, an action execution component adjusts the vehicle's maximum speed and outputs a rest reminder. When the driver's status monitoring result indicates a sudden physical condition, the action execution component outputs a parking reminder and detects whether the driver performs a parking operation. If the driver does not perform a parking operation within a target time period, the vehicle's location information and the driver's visual observation data, contact observation data, ECG observation data, and infrared observation data are sent to emergency contact users and rescue users. This solves the problem that related driver status monitoring schemes rely on in-vehicle images for driver status monitoring, which has a relatively single monitoring dimension and is easily affected by factors such as the vehicle environment and driver emotions, failing to comprehensively and accurately monitor the driver's condition. The problem of insufficient timely alerts and rescues when drivers experience sudden physical distress during status monitoring can be addressed by comprehensively and accurately monitoring driver fatigue or sudden physical distress based on visual observation data, contact observation data, electrocardiogram (ECG) observation data, and infrared observation data. The monitoring results can be used to adjust the vehicle's speed limit and output rest reminders when driver fatigue is detected, enabling timely intervention. Conversely, when a sudden physical distress is detected, a stop reminder can be output, and the system can detect whether the driver stops. If the driver fails to stop within a target timeframe, the vehicle's location information and the driver's visual, contact, ECG, and infrared observation data are sent to emergency contact and rescue personnel, allowing for timely intervention. This multi-dimensional data-driven approach enables comprehensive and accurate driver status monitoring, timely speed control and alerts when the driver is fatigued, and timely alerts and rescues when the driver experiences a sudden physical distress.

[0084] In an optional embodiment of the present invention, the acquisition module 301 is optionally configured to: acquire visual observation data of the driver via a visual acquisition sub-component; wherein the visual observation data includes blink frequency, eyelid closure degree, yawn frequency, and facial expression state; acquire contact observation data of the driver via a contact acquisition sub-component; wherein the contact observation data includes heart rate, blood oxygen, and blood pressure; acquire electrocardiogram (ECG) observation data of the driver via an ECG acquisition sub-component; wherein the ECG observation data includes an electrocardiogram; and acquire infrared observation data of the driver via an infrared acquisition sub-component; wherein the infrared observation data includes body temperature.

[0085] In an optional embodiment of the present invention, the monitoring module 302 is specifically used to: periodically input the driver's visual observation data, contact observation data, electrocardiogram observation data and infrared observation data collected within a preset time period before the current time to the data analysis component, and obtain the driver's status monitoring result output by the data analysis component; wherein, the driver's status monitoring result is a normal state, a fatigue state or a sudden physical condition.

[0086] In an optional embodiment of the present invention, the driver status monitoring device may further include: a communication information detection module, used to detect whether the driver has set up communication information for emergency contact users; and a communication information prompting module, used to prompt the driver to set up communication information for emergency contact users if it is detected that the driver has not set up communication information for emergency contact users.

[0087] In an optional embodiment of the present invention, the first state processing module 303 is specifically used to: when the driver's state monitoring result is fatigued, update the vehicle's speed limit to a preset speed value through the action execution component, control the display device to display rest reminder information, and control the audible and visual alarm device to output alarm information.

[0088] In an optional embodiment of the present invention, the second state processing module 304 is specifically used to: when the driver's state monitoring result is a sudden physical condition, output a parking prompt message through the action execution component, and detect whether the driver has performed a parking operation; if it is detected that the driver has not performed a parking operation within the target time period, then according to the communication information of the emergency contact user and the rescue user, send the vehicle's location information and the latest collected driver's visual observation data, contact observation data, electrocardiogram observation data and infrared observation data to the emergency contact user and the rescue user.

[0089] In an optional embodiment of the present invention, the driver status monitoring device may further include: a third status processing module, used to control the display device to display the driver's visual observation data, contact observation data, electrocardiogram observation data and infrared observation data collected within a preset time period before the current moment when the driver's status monitoring result is normal.

[0090] The driver status monitoring device provided in the embodiments of the present invention can execute the driver status monitoring method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0091] Example 4

[0092] Figure 4 A schematic diagram of an electronic device 10, which can be used to implement the driver state monitoring method of embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, electronic devices, blade electronic devices, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0093] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0094] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0095] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as driver state monitoring methods.

[0096] In some embodiments, the driver state monitoring method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on a heterogeneous hardware accelerator via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by a processor, one or more steps of the driver state monitoring method described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform the driver state monitoring method by any other suitable means (e.g., by means of firmware).

[0097] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0098] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or electronic device.

[0099] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0100] To provide user interaction, the systems and techniques described herein can be implemented on a heterogeneous hardware accelerator, which includes: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the heterogeneous hardware accelerator. Other types of devices can also be used to provide user interaction; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or haptic feedback); and input from the user can be received in any form (including sound input, voice input, or haptic input).

[0101] The systems and technologies described herein can be implemented in computing systems that include back-end components (e.g., as data electronic devices), or computing systems that include middleware components (e.g., application electronic devices), or computing systems that include front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0102] A computing system can include clients and electronic devices. Clients and electronic devices are generally geographically separated and typically interact via communication networks. The client-electronic device relationship is created by computer programs running on the respective computers and establishing a client-electronic device relationship between them. Electronic devices can be cloud electronic devices, also known as cloud computing electronic devices or cloud servers, which are hosting products within the cloud computing service ecosystem. These address the shortcomings of traditional physical hosting and VPS services, such as high management difficulty and weak business scalability.

[0103] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0104] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for monitoring driver status, characterized in that, include: The data acquisition component collects the driver's visual observation data, tactile observation data, electrocardiogram observation data, and infrared observation data. Through the data analysis component, the system periodically detects whether the driver is in a state of fatigue or sudden physical condition based on the driver's visual observation data, contact observation data, electrocardiogram observation data and infrared observation data, and obtains the driver's condition monitoring results. When the driver's condition monitoring result indicates fatigue, the vehicle's speed limit is adjusted through the action execution component, and a rest reminder message is output. When the driver's status monitoring result indicates a sudden physical condition, the action execution component outputs a parking prompt message and detects whether the driver performs a parking operation. If the driver does not perform a parking operation within the target time period, the vehicle's location information and the driver's visual observation data, contact observation data, electrocardiogram observation data, and infrared observation data are sent to the emergency contact user and the rescue user.

2. The driver status monitoring method according to claim 1, characterized in that, The data acquisition component collects the driver's visual observation data, tactile observation data, electrocardiogram observation data, and infrared observation data, including: The visual acquisition sub-component collects the driver's visual observation data, including blinking frequency, eyelid closure degree, yawning frequency, and facial expression status. The driver's contact observation data is collected through the contact acquisition sub-component; wherein, the contact observation data includes heart rate, blood oxygen and blood pressure; The driver's electrocardiogram (ECG) observation data is collected through the ECG acquisition sub-component; wherein, the ECG observation data includes an electrocardiogram (ECG). The infrared acquisition sub-component collects infrared observation data of the driver; wherein, the infrared observation data includes body temperature.

3. The driver status monitoring method according to claim 1, characterized in that, Through data analysis components, the system periodically detects whether the driver is fatigued or experiencing a sudden physical condition based on visual observation data, tactile observation data, electrocardiogram observation data, and infrared observation data, obtaining driver status monitoring results, including: The system will periodically input the driver's visual observation data, contact observation data, electrocardiogram observation data, and infrared observation data collected within a preset time period before the current moment into the data analysis component, and obtain the driver's status monitoring result output by the data analysis component; wherein, the driver's status monitoring result is a normal state, a fatigued state, or a sudden physical condition.

4. The driver status monitoring method according to claim 1, characterized in that, Before collecting the driver's visual observation data, tactile observation data, electrocardiogram observation data, and infrared observation data through the data acquisition components, the following is also included: Detect whether the driver has set up emergency contact information for the user; If it is detected that the driver has not set up emergency contact user communication information, the driver will be prompted to set up emergency contact user communication information.

5. The driver status monitoring method according to claim 1, characterized in that, When the driver's condition monitoring indicates fatigue, the action execution component adjusts the vehicle's maximum speed and outputs a rest reminder message, including: When the driver's condition monitoring result indicates fatigue, the action execution component updates the vehicle's speed limit to a preset speed value, controls the display device to show a rest reminder message, and controls the audible and visual alarm device to output an alarm message.

6. The driver status monitoring method according to claim 1, characterized in that, When the driver's condition monitoring result indicates a sudden physical condition, the action execution component outputs a parking prompt message and detects whether the driver performs a parking operation. If the driver does not perform a parking operation within the target time period, the vehicle's location information and the driver's visual observation data, contact observation data, electrocardiogram observation data, and infrared observation data are sent to emergency contact users and rescue users, including: When the driver's status monitoring result indicates a sudden physical condition, the action execution component outputs a parking prompt message and detects whether the driver performs a parking operation. If it is detected that the driver has not performed a parking operation within the target time period, the vehicle's location information and the latest collected visual observation data, contact observation data, electrocardiogram observation data and infrared observation data of the driver will be sent to the emergency contact user and the rescue user based on the communication information of the emergency contact user and the rescue user.

7. The driver status monitoring method according to claim 1, characterized in that, Also includes: When the driver's condition monitoring result is normal, the control display device displays the driver's visual observation data, contact observation data, electrocardiogram observation data, and infrared observation data collected within a preset time period prior to the current moment.

8. A driver status monitoring device, characterized in that, include: The data acquisition module is used to collect the driver's visual observation data, tactile observation data, electrocardiogram observation data, and infrared observation data through the data acquisition components. The monitoring module is used to periodically detect whether the driver is fatigued or experiencing a sudden physical condition based on the driver's visual observation data, contact observation data, electrocardiogram observation data, and infrared observation data through the data analysis component, and obtain the driver's condition monitoring results. The first state processing module is used to adjust the vehicle's speed limit and output a rest reminder message when the driver's state monitoring result indicates fatigue. The second state processing module is used to output a parking prompt message through the action execution component when the driver's state monitoring result indicates a sudden physical condition, and to detect whether the driver has performed a parking operation. If the driver has not performed a parking operation within the target time period, the module will send the vehicle's location information and the driver's visual observation data, contact observation data, electrocardiogram observation data and infrared observation data to the emergency contact user and the rescue user.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that is executed by the at least one processor, which enables the at least one processor to perform the driver state monitoring method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the driver state monitoring method according to any one of claims 1-7.

Images