Vehicle warning method and device based on driver state monitoring and electronic equipment

Abstract

This invention discloses a vehicle warning method, device, and electronic device based on driver status monitoring. The method includes: acquiring facial image data and vital sign data of the target driver, and vehicle behavior data of the target vehicle; inputting the facial image data, vital sign data, and vehicle behavior data into a pre-trained anomaly detection model to obtain the health risk level of the target driver and the vehicle hazard level of the target vehicle; determining the priority of vehicle assistance requests and hazard warnings according to the health risk level and vehicle hazard level, and generating vehicle assistance request information and hazard warning information; broadcasting the vehicle assistance request information to emergency service vehicles and roadside terminals according to priority, and broadcasting the hazard warning information to non-emergency service vehicles. This invention balances road traffic safety and driver personal safety, improves vehicle driving safety, and can be applied to the field of vehicle monitoring technology.

Description

Technical Field

[0001] This invention relates to the field of vehicle monitoring technology, and in particular to a vehicle warning method, device, and electronic device based on driver status monitoring. Background Technology

[0002] With the development of intelligent vehicles, driver monitoring systems have been widely used for fatigue driving and distracted driving warnings. However, existing driver status monitoring functions only serve in-vehicle warnings (such as reminding the driver to drive safely), and their information does not interact with the external environment. When a driver suddenly falls ill, they are often unable to effectively operate the vehicle or call for help, leading to potential driving safety hazards and delaying the driver's rescue efforts, thus affecting road traffic safety and the driver's personal safety.

[0003] The above problems urgently need to be addressed. Summary of the Invention

[0004] The purpose of this invention is to at least partially solve one of the technical problems existing in the prior art.

[0005] Therefore, one objective of this invention is to provide a vehicle warning method based on driver status monitoring. This method identifies the driver's health risk level and the degree of vehicle danger based on the driver's facial image, vital signs, and vehicle behavior. It determines the priority of vehicle assistance requests and danger warnings according to the health risk level and the degree of vehicle danger, so that when the driver suffers a sudden illness, vehicle assistance requests and danger warnings can be broadcast and pushed according to priority. This allows emergency service vehicles to quickly rescue the driver, and non-emergency service vehicles to avoid dangerous vehicles in time, thus taking into account both road traffic safety and driver personal safety, and improving vehicle driving safety.

[0006] Another objective of this invention is to provide a vehicle warning device based on driver status monitoring.

[0007] To achieve the above-mentioned technical objectives, the technical solutions adopted in the embodiments of the present invention include: On one hand, embodiments of the present invention provide a vehicle warning method based on driver status monitoring, comprising the following steps: Acquire facial image data, vital sign data of the target driver, and vehicle behavior data of the target vehicle; The facial image data, vital sign data, and vehicle behavior data are input into a pre-trained anomaly detection model to obtain the health risk level of the target driver and the vehicle danger level of the target vehicle. Based on the health risk level and the vehicle danger level, the priorities of vehicle distress calls and danger warnings are determined respectively, and vehicle distress call information and danger warning information are generated. The vehicle distress call information is broadcast to emergency service vehicles and roadside terminals according to priority, while the danger warning information is broadcast to non-emergency service vehicles.

[0008] Furthermore, in one embodiment of the present invention, the anomaly detection model is trained through the following steps: We acquire facial image samples, vital sign samples, and vehicle behavior samples of historical drivers in historical driving scenarios, and determine the corresponding health risk level labels and vehicle danger level labels through manual annotation. The facial image samples, vital sign samples, and vehicle behavior samples are input into a pre-constructed multi-branch convolutional neural network to obtain the predicted health risk level and the predicted vehicle danger level. The loss value is determined based on the predicted health risk level, the predicted vehicle hazard level, the health risk level label, and the vehicle hazard level label. The parameters of the multi-branch convolutional neural network are updated using the backpropagation algorithm based on the loss value to obtain the trained anomaly detection model.

[0009] Further, in one embodiment of the present invention, the multi-branch convolutional neural network includes a facial image convolutional branch, a vital sign convolutional branch, a vehicle behavior convolutional branch, a first feature fusion layer, a second feature fusion layer, a first fully connected layer, and a second fully connected layer. The step of inputting the facial image samples, the vital sign samples, and the vehicle behavior samples into the pre-constructed multi-branch convolutional neural network to obtain the predicted health risk level and the predicted vehicle hazard level specifically includes: The facial image samples, vital sign samples, and vehicle behavior samples are respectively input into the facial image convolution branch, the vital sign convolution branch, and the vehicle behavior convolution branch to obtain facial image features, vital sign features, and vehicle behavior features. The first feature fusion layer performs feature fusion on the facial image features and the vital sign features to obtain the first fused feature; The first fused feature is mapped to a health risk level probability distribution through the first fully connected layer; The second feature fusion layer performs feature fusion on the health risk level probability distribution and the vehicle behavior features to obtain the second fused feature. The second fused feature is mapped to a vehicle danger level probability distribution through the second fully connected layer; The predicted health risk level is determined based on the probability distribution of the health risk level, and the predicted vehicle danger level is determined based on the probability distribution of the vehicle danger level.

[0010] Furthermore, in one embodiment of the present invention, the step of determining the priority of vehicle assistance and hazard warning based on the health risk level and the vehicle danger level, respectively, and generating vehicle assistance information and hazard warning information, specifically includes: The health risk level is mapped to vehicle assistance priority using a preset first priority mapping rule, and the vehicle danger level is mapped to danger warning priority using a preset second priority mapping rule. When the vehicle distress request priority is higher than the danger warning priority, the vehicle distress request information is generated based on the facial image data, the vital signs data, and the real-time location of the target vehicle, and the danger warning information is generated based on the vehicle danger level and the vehicle distress request information. When the vehicle distress request priority is not higher than the danger warning priority, the danger warning information is generated based on the vehicle danger level and the real-time location of the target vehicle, and the vehicle distress request information is generated based on the facial image data, the vital signs data, and the danger warning information.

[0011] Furthermore, in one embodiment of the present invention, the step of broadcasting the vehicle distress call information to emergency service vehicles and roadside terminals according to priority, and broadcasting the hazard warning information to non-emergency service vehicles, specifically includes: The assistance range is determined based on the health risk level and the real-time location of the target vehicle, and the warning range is determined based on the vehicle's danger level and the real-time location of the target vehicle. When the vehicle distress request priority is higher than the danger warning priority, the vehicle distress request information is broadcast to emergency service vehicles and roadside terminals within the distress request range via satellite communication and V2X communication, and the danger warning information is broadcast to non-emergency service vehicles within the warning range via Bluetooth communication and 5G communication. When the vehicle's distress request priority is not higher than the danger warning priority, the danger warning information is broadcast to non-emergency service vehicles within the warning range via satellite communication and V2X communication, and to emergency service vehicles and roadside terminals within the distress request range via Bluetooth communication and 5G communication.

[0012] Furthermore, in one embodiment of the present invention, the vehicle warning method further includes the following steps: The vehicle distress call information and the danger warning information are sent to the vehicle's external display screen, so that the vehicle's external display screen displays the vehicle distress call information and the danger warning information according to priority.

[0013] Furthermore, in one embodiment of the present invention, the vehicle warning method further includes the following steps: Determine whether the target driver has the ability to drive manually based on the health risk level; When the target driver does not have the ability to drive manually, the target vehicle is controlled to switch to automatic driving mode, and the optimal parking path is obtained by planning the parking path according to the surrounding environment of the target vehicle. Then, the target vehicle is controlled to brake and stop according to the optimal parking path.

[0014] On the other hand, embodiments of the present invention provide a vehicle warning device based on driver status monitoring, comprising: The data acquisition module is used to acquire facial image data, vital sign data of the target driver, and vehicle behavior data of the target vehicle. An anomaly detection module is used to input the facial image data, the vital signs data, and the vehicle behavior data into a pre-trained anomaly detection model to obtain the health risk level of the target driver and the vehicle danger level of the target vehicle. The information generation module is used to determine the priority of vehicle distress call and danger warning based on the health risk level and the vehicle danger level, respectively, and generate vehicle distress call information and danger warning information. The information broadcasting module is used to broadcast the vehicle distress information to emergency service vehicles and roadside terminals according to priority, and to broadcast the danger warning information to non-emergency service vehicles.

[0015] On the other hand, embodiments of the present invention provide an electronic device, including: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the above-described vehicle warning method based on driver state monitoring.

[0016] On the other hand, embodiments of the present invention also provide a computer-readable storage medium storing a processor-executable computer program that, when executed by a processor, implements the above-described vehicle warning method based on driver status monitoring.

[0017] On the other hand, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the above-described vehicle warning method based on driver status monitoring.

[0018] The advantages and beneficial effects of the present invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention: This invention acquires facial image data and vital sign data of the target driver, as well as vehicle behavior data of the target vehicle. The facial image data, vital sign data, and vehicle behavior data are input into a pre-trained anomaly detection model to obtain the target driver's health risk level and the target vehicle's hazard level. Based on the health risk level and vehicle hazard level, the priorities for vehicle assistance requests and hazard warnings are determined, and vehicle assistance requests and hazard warnings are generated. The vehicle assistance requests are broadcast to emergency service vehicles and roadside terminals according to priority, while the hazard warnings are broadcast to non-emergency service vehicles. This invention identifies the driver's health risk level and vehicle hazard level based on the driver's facial image, vital signs, and vehicle behavior, and determines the priorities for vehicle assistance requests and hazard warnings based on these priorities. This allows for the broadcasting of vehicle assistance requests and hazard warnings according to priority when a driver experiences a sudden illness, enabling emergency service vehicles to quickly rescue the driver and non-emergency service vehicles to avoid dangerous vehicles in a timely manner. This balances road traffic safety and driver safety, improving overall vehicle safety. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments of the present invention are described below. It should be understood that the drawings described below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating the steps of a vehicle warning method based on driver status monitoring, provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a multi-branch convolutional neural network according to an embodiment of the present invention; Figure 3 A structural block diagram of a vehicle warning device based on driver status monitoring provided in an embodiment of the present invention; Figure 4 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of this invention; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this invention as detailed in the appended claims.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0023] The vehicle warning method based on driver status monitoring provided in this invention can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application that implements the vehicle warning method based on driver status monitoring, but is not limited to the above forms.

[0024] This invention can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0025] It should be noted that in various specific embodiments of the present invention, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user parking space location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of the present invention require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to a confirmation page. Only after obtaining the user's separate permission or consent is the necessary user-related data for the normal operation of the embodiments of the present invention acquired.

[0026] Reference Figure 1 This invention provides a vehicle warning method based on driver status monitoring, specifically including the following steps: S101. Obtain facial image data, vital sign data of the target driver, and vehicle behavior data of the target vehicle. S102. Input facial image data, vital sign data and vehicle behavior data into a pre-trained anomaly detection model to obtain the health risk level of the target driver and the vehicle danger level of the target vehicle. S103. Determine the priority of vehicle distress calls and hazard warnings based on the health risk level and the degree of vehicle danger, and generate vehicle distress call information and hazard warning information respectively; S104. Broadcast vehicle distress information to emergency service vehicles and roadside terminals according to priority, and broadcast hazard warning information to non-emergency service vehicles.

[0027] This invention identifies the driver's health risk level and vehicle danger level based on the driver's facial image, vital signs, and vehicle behavior. It determines the priority of vehicle assistance requests and danger warnings according to the health risk level and vehicle danger level, respectively. This allows for the broadcasting and push of vehicle assistance and danger warning information according to priority when the driver experiences a sudden illness. This enables emergency service vehicles to quickly rescue the driver, and non-emergency service vehicles to avoid dangerous vehicles in a timely manner, thus balancing road traffic safety and driver safety and improving vehicle driving safety.

[0028] Specifically, the system collects facial image data of the driver through in-vehicle cameras, focusing on capturing eyelid opening and closing, eye movement frequency, facial muscle state, and yawning frequency to determine facial abnormalities caused by fatigue, absent-mindedness, or sudden illness. It also collects vital sign data of the driver through devices such as heart rate sensors on the steering wheel grips, seat pressure sensors, and smart bracelets, obtaining real-time indicators such as heart rate, blood pressure, blood oxygen saturation, and respiratory rate, which directly reflect health status. Finally, it obtains vehicle behavior data from the vehicle's CAN bus, including steering wheel angle, accelerator pedal depth, braking frequency, vehicle trajectory, speed changes, and lane departure, to determine whether vehicle operation conforms to normal driving logic.

[0029] After obtaining facial image data, vital sign data, and vehicle behavior data, these are input into a pre-trained anomaly detection model for bidirectional correlation analysis. For determining the level of health risk, the model cross-validates facial features with vital sign data. For example, continuous eye closure combined with a sudden drop in heart rate may be classified as extremely high risk, while frequent yawning combined with heart rate fluctuations may be classified as moderate risk. For determining the degree of vehicle danger, considering the safety hazards caused by the driver's health risks preventing proper vehicle operation, a comprehensive judgment must be made by combining the identified health risk level with vehicle behavior data. For example, sudden acceleration while the driver is distracted or lane departure without operation may be classified as high-risk.

[0030] It should be noted that the model training phase needs to cover various abnormal scenarios, such as fatigued driving, sudden cardiovascular and cerebrovascular diseases, epileptic seizures, hypoglycemic fainting and other health abnormalities, as well as corresponding vehicle loss of control scenarios.

[0031] As an optional implementation, the anomaly detection model is trained through the following steps: S201. Obtain facial image samples, vital sign samples, and vehicle behavior samples of historical drivers in historical driving scenarios, and determine the corresponding health risk level labels and vehicle danger level labels through manual annotation. S202. Input facial image samples, vital sign samples, and vehicle behavior samples into a pre-constructed multi-branch convolutional neural network to obtain the predicted health risk level and the predicted vehicle danger level. S203. Determine the loss value based on the predicted health risk level, the predicted vehicle hazard level, the health risk level label, and the vehicle hazard level label. S204. Update the parameters of the multi-branch convolutional neural network based on the loss value using the backpropagation algorithm to obtain the trained anomaly detection model.

[0032] Specifically, facial image samples and vital sign samples of historical drivers and vehicle behavior samples from historical driving scenarios are acquired, and corresponding health risk level labels and vehicle hazard level labels are determined through manual annotation. The facial image samples, vital sign samples, and vehicle behavior samples are input into a pre-constructed multi-branch convolutional neural network to obtain the predicted health risk level and predicted vehicle hazard level. The loss value is determined based on the predicted health risk level, predicted vehicle hazard level, health risk level label, and vehicle hazard level label. The parameters of the multi-branch convolutional neural network are updated using the backpropagation algorithm based on the loss value to complete one iteration of training. When the number of iterations reaches a preset threshold, or the loss value falls below a preset threshold, training stops, and the trained anomaly detection model is obtained.

[0033] As a further optional implementation, the multi-branch convolutional neural network includes a facial image convolutional branch, a vital sign convolutional branch, a vehicle behavior convolutional branch, a first feature fusion layer, a second feature fusion layer, a first fully connected layer, and a second fully connected layer. Facial image samples, vital sign samples, and vehicle behavior samples are input into the pre-constructed multi-branch convolutional neural network to obtain predicted health risk levels and predicted vehicle hazard levels, specifically including: S2021. Input the facial image samples, vital sign samples, and vehicle behavior samples into the facial image convolution branch, vital sign convolution branch, and vehicle behavior convolution branch respectively to obtain facial image features, vital sign features, and vehicle behavior features. S2022. Facial image features and vital sign features are fused through the first feature fusion layer to obtain the first fused feature; S2023. The first fused feature is mapped to a health risk level probability distribution through the first fully connected layer; S2024. The second feature fusion layer is used to fuse the probability distribution of health risk level and vehicle behavior features to obtain the second fused feature. S2025. The second fused feature is mapped to a probability distribution of vehicle hazard level through the second fully connected layer; S2026. Determine the predicted health risk level based on the probability distribution of health risk level, and determine the predicted vehicle danger level based on the probability distribution of vehicle danger level.

[0034] like Figure 2The diagram shows the structure of a multi-branch convolutional neural network according to an embodiment of the present invention. In this embodiment, facial image samples, vital sign samples, and vehicle behavior samples are input into the facial image convolutional branch, vital sign convolutional branch, and vehicle behavior convolutional branch, respectively, to obtain facial image features, vital sign features, and vehicle behavior features. The facial image features and vital sign features output from the facial image convolutional branch and the vital sign convolutional branch are input into a first feature fusion layer for feature fusion to obtain a first fused feature. This feature is then input into a first fully connected layer and mapped to a health risk level probability distribution. The vehicle behavior features output from the vehicle behavior convolutional branch and the health risk level probability distribution are then input into a second feature fusion layer for feature fusion to obtain a second fused feature. Finally, this feature is input into a second fully connected layer and mapped to a vehicle danger level probability distribution. Based on the health risk level probability distribution and the vehicle danger level probability distribution, the predicted health risk level and the predicted vehicle danger level can be determined, respectively.

[0035] As an optional implementation, the priorities of vehicle assistance requests and hazard warnings are determined based on the health risk level and the degree of vehicle danger, respectively, and vehicle assistance information and hazard warning information are generated, specifically including: S1031. The health risk level is mapped to the vehicle assistance priority through the preset first priority mapping rule, and the vehicle danger level is mapped to the danger warning priority through the preset second priority mapping rule. S1032. When the priority of vehicle distress call is higher than the priority of danger warning, generate vehicle distress call information based on facial image data, vital sign data and the real-time location of the target vehicle, and generate danger warning information based on the degree of danger of the vehicle and the vehicle distress call information. S1033. When the priority of vehicle distress call is not higher than the priority of danger warning, generate danger warning information based on the degree of danger of the vehicle and the real-time location of the target vehicle, and generate vehicle distress call information based on facial image data, vital sign data and danger warning information.

[0036] Specifically, a first priority mapping rule is pre-constructed that corresponds to the health risk level and the vehicle assistance priority score, and a second priority mapping rule is constructed that corresponds to the vehicle danger level and the danger warning level score. For example, the health risk level can be divided into six levels from 0 to 5, and the vehicle danger level can be divided into six levels from 0 to 5. When the health risk level is 0 or 1, the corresponding vehicle assistance priority score is 0. When the health risk level is 2-5, the corresponding vehicle assistance priority score is 2-5. And when the vehicle danger level is 0-5, the corresponding danger warning priority score is 0-5.

[0037] The first priority mapping rule maps health risk levels to vehicle distress priority, while the second priority mapping rule maps vehicle hazard level to hazard warning priority. The two are then compared. When the vehicle distress priority is higher than the hazard warning priority, such as when a driver suffers a sudden heart attack (extremely high health risk) but the vehicle is traveling at a low speed and smoothly (low hazard level), it indicates that vehicle distress should be prioritized over hazard warning. In this case, precise vehicle distress information is generated based on facial image data, vital sign data, and the target vehicle's real-time location. Then, a hazard warning is generated by combining the vehicle hazard level and the distress information, such as "The driver of the vehicle ahead has suffered a sudden illness; please give way." When the priority of vehicle distress calls is not higher than that of hazard warnings, such as when a vehicle experiences a tire blowout and loses control (extremely high vehicle danger) but the driver is conscious (low health risk), or when a driver has an epileptic seizure (extremely high health risk) and the vehicle is out of control on a highway (extremely high vehicle danger), in order to avoid causing a traffic accident and greater harm, hazard warnings should be prioritized over vehicle distress calls. In this case, accurate hazard warning information is generated based on the vehicle's danger level and the target vehicle's real-time location. Then, facial image data, vital sign data, and the hazard warning information are combined to generate vehicle distress call information.

[0038] It should be noted that when the vehicle assistance priority score / danger warning priority score is 0, it means that no vehicle assistance / danger warning is required, and correspondingly, no vehicle assistance / danger warning information will be generated.

[0039] As a further optional implementation, vehicle distress calls are broadcast to emergency service vehicles and roadside terminals according to priority, while hazard warnings are broadcast to non-emergency service vehicles. Specifically, this includes: S1041. Determine the scope of the distress call based on the health risk level and the real-time location of the target vehicle, and determine the warning scope based on the degree of danger of the vehicle and the real-time location of the target vehicle. S1042. When the priority of vehicle distress call is higher than the priority of hazard warning, the vehicle distress call information is broadcast to emergency service vehicles and roadside terminals within the distress call range via satellite communication and V2X communication, and the hazard warning information is broadcast to non-emergency service vehicles within the warning range via Bluetooth communication and 5G communication. S1043. When the priority of vehicle assistance is not higher than the priority of hazard warning, the hazard warning information is broadcast to non-emergency service vehicles within the warning range via satellite communication and V2X communication, and the vehicle assistance information is broadcast to emergency service vehicles and roadside terminals within the assistance range via Bluetooth communication and 5G communication.

[0040] Specifically, different health risk levels correspond to different assistance ranges. The higher the health risk level, the larger the assistance range. Different vehicle danger levels also correspond to different warning ranges. The greater the vehicle danger level, the larger the warning range.

[0041] In this embodiment of the invention, primary and secondary tasks are broadcast using different communication methods. When the priority of vehicle assistance requests is higher than that of hazard warnings, vehicle assistance requests are prioritized over hazard warnings. Hazard warning information is broadcast to non-emergency service vehicles within the warning range via satellite and V2X communication, while vehicle assistance requests are broadcast to emergency service vehicles and roadside terminals within the assistance range via Bluetooth and 5G communication. Conversely, when the priority of vehicle assistance requests is no higher than that of hazard warnings, hazard warnings are prioritized over vehicle assistance requests. Hazard warning information is broadcast to non-emergency service vehicles within the warning range via satellite and V2X communication, while vehicle assistance requests are broadcast to emergency service vehicles and roadside terminals within the assistance range via Bluetooth and 5G communication. Because satellite and V2X communication offer higher stability and timeliness compared to Bluetooth and 5G communication, the completion of the primary task can be guaranteed.

[0042] When emergency service vehicles (police cars, ambulances, etc.) in the area where a dangerous vehicle is located receive a distress call, they can quickly proceed to the target location to provide vehicle assistance. Meanwhile, non-emergency service vehicles around the dangerous vehicle, upon receiving a hazard warning, can avoid the dangerous vehicle to prevent traffic accidents, and this also facilitates emergency service vehicles in rescuing the dangerous vehicle.

[0043] As an optional implementation, the vehicle warning method further includes the following steps: S105. Send vehicle distress information and hazard warning information to the vehicle's external display screen, so that the vehicle's external display screen displays the vehicle distress information and hazard warning information according to priority.

[0044] Specifically, in this embodiment of the invention, corresponding simplified distress text and warning text are generated based on vehicle distress information and danger warning information, and then displayed on the vehicle's external display screen according to priority. If the vehicle distress priority is higher, the distress text is displayed first; if the vehicle distress priority is not higher than the danger warning priority, the warning text is displayed first.

[0045] As an optional implementation, the vehicle warning method further includes the following steps: S106. Determine whether the target driver has the ability to drive manually based on the health risk level; S107. When the target driver does not have the ability to drive manually, control the target vehicle to switch to automatic driving mode, and plan the optimal parking path according to the surrounding environment of the target vehicle, and then control the target vehicle to brake and stop according to the optimal parking path.

[0046] Specifically, in this embodiment of the invention, the system determines whether the target driver has the ability to drive manually based on the health risk level. If the target driver does not have the ability to drive manually, the autonomous driving system takes over in an emergency, plans the optimal parking path based on the surrounding environment of the target vehicle, and then controls the target vehicle to brake and stop according to the optimal parking path, waiting for rescue personnel.

[0047] The method steps of the embodiments of the present invention have been described above. It can be understood that the embodiments of the present invention, based on the driver's facial image, vital signs, and vehicle behavior, identify the driver's health risk level and the degree of vehicle danger. The priorities of vehicle assistance requests and danger warnings are determined according to the health risk level and the degree of vehicle danger, respectively. This allows for the broadcasting and push of vehicle assistance and danger warning information according to priority when the driver experiences a sudden illness. This enables emergency service vehicles to quickly rescue the driver, and non-emergency service vehicles to avoid dangerous vehicles in a timely manner, thus balancing road traffic safety and driver safety, and improving vehicle driving safety.

[0048] Reference Figure 3 This invention provides a vehicle warning device based on driver status monitoring, comprising: The data acquisition module is used to acquire facial image data, vital sign data of the target driver, and vehicle behavior data of the target vehicle. The anomaly detection module is used to input facial image data, vital sign data, and vehicle behavior data into a pre-trained anomaly detection model to obtain the health risk level of the target driver and the vehicle hazard level of the target vehicle. The information generation module is used to determine the priority of vehicle distress calls and hazard warnings based on the health risk level and the degree of vehicle danger, and to generate vehicle distress call information and hazard warning information. The information broadcasting module is used to broadcast vehicle distress requests to emergency service vehicles and roadside terminals according to priority, and to broadcast hazard warning information to non-emergency service vehicles.

[0049] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0050] Reference Figure 4This invention provides an electronic device, comprising: At least one processor; At least one memory for storing at least one program; When the above-mentioned at least one program is executed by the above-mentioned at least one processor, the above-mentioned at least one processor implements the above-mentioned vehicle warning method based on driver status monitoring.

[0051] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0052] This invention also provides a computer-readable storage medium storing a processor-executable computer program that, when executed by a processor, implements the above-described vehicle warning method based on driver status monitoring.

[0053] This invention provides a computer-readable storage medium that can execute a vehicle warning method based on driver status monitoring provided in the method embodiments of this invention. It can execute any combination of the implementation steps of the method embodiments and has the corresponding functions and beneficial effects of the method.

[0054] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described vehicle warning method based on driver status monitoring.

[0055] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0056] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0057] The embodiments described in this invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0058] The terms "first," "second," "third," "fourth," etc. (if present) in the specification and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0059] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the aforementioned blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0060] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the aforementioned functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0061] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0062] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0063] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the aforementioned program can be printed, because the aforementioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0064] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0065] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0066] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0067] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A vehicle alerting method based on driver state monitoring, characterized by, Includes the following steps: Acquire facial image data, vital sign data of the target driver, and vehicle behavior data of the target vehicle; The facial image data, vital sign data, and vehicle behavior data are input into a pre-trained anomaly detection model to obtain the health risk level of the target driver and the vehicle danger level of the target vehicle. Based on the health risk level and the vehicle danger level, the priorities of vehicle distress calls and danger warnings are determined respectively, and vehicle distress call information and danger warning information are generated. The vehicle distress call information is broadcast to emergency service vehicles and roadside terminals according to priority, while the danger warning information is broadcast to non-emergency service vehicles.

2. The vehicle alert method based on driver state monitoring as claimed in claim 1, wherein, The anomaly detection model is trained through the following steps: We acquire facial image samples, vital sign samples, and vehicle behavior samples of historical drivers in historical driving scenarios, and determine the corresponding health risk level labels and vehicle danger level labels through manual annotation. The facial image samples, vital sign samples, and vehicle behavior samples are input into a pre-constructed multi-branch convolutional neural network to obtain the predicted health risk level and the predicted vehicle danger level. The loss value is determined based on the predicted health risk level, the predicted vehicle hazard level, the health risk level label, and the vehicle hazard level label. The parameters of the multi-branch convolutional neural network are updated using the backpropagation algorithm based on the loss value to obtain the trained anomaly detection model.

3. A vehicle alerting method based on driver state monitoring according to claim 2, characterized in that, The multi-branch convolutional neural network includes a facial image convolutional branch, a vital signs convolutional branch, a vehicle behavior convolutional branch, a first feature fusion layer, a second feature fusion layer, a first fully connected layer, and a second fully connected layer. The step of inputting the facial image samples, the vital signs samples, and the vehicle behavior samples into the pre-constructed multi-branch convolutional neural network to obtain predicted health risk levels and predicted vehicle hazard levels specifically includes: The facial image samples, vital sign samples, and vehicle behavior samples are respectively input into the facial image convolution branch, the vital sign convolution branch, and the vehicle behavior convolution branch to obtain facial image features, vital sign features, and vehicle behavior features. The first feature fusion layer performs feature fusion on the facial image features and the vital sign features to obtain the first fused feature; The first fused feature is mapped to a health risk level probability distribution through the first fully connected layer; The second feature fusion layer performs feature fusion on the health risk level probability distribution and the vehicle behavior features to obtain the second fused feature. The second fused feature is mapped to a vehicle danger level probability distribution through the second fully connected layer; The predicted health risk level is determined based on the probability distribution of the health risk level, and the predicted vehicle danger level is determined based on the probability distribution of the vehicle danger level.

4. The vehicle alert method based on driver state monitoring as claimed in claim 1, wherein, The process of determining the priority of vehicle distress calls and hazard warnings based on the health risk level and the vehicle hazard level, and generating vehicle distress call information and hazard warning information, specifically includes: The health risk level is mapped to vehicle assistance priority using a preset first priority mapping rule, and the vehicle danger level is mapped to danger warning priority using a preset second priority mapping rule. When the vehicle distress request priority is higher than the danger warning priority, the vehicle distress request information is generated based on the facial image data, the vital signs data, and the real-time location of the target vehicle, and the danger warning information is generated based on the vehicle danger level and the vehicle distress request information. When the vehicle distress request priority is not higher than the danger warning priority, the danger warning information is generated based on the vehicle danger level and the real-time location of the target vehicle, and the vehicle distress request information is generated based on the facial image data, the vital signs data, and the danger warning information.

5. A vehicle alerting method based on driver state monitoring according to claim 4, characterized in that, The step of broadcasting the vehicle distress call information to emergency service vehicles and roadside terminals according to priority, and broadcasting the hazard warning information to non-emergency service vehicles, specifically includes: The assistance range is determined based on the health risk level and the real-time location of the target vehicle, and the warning range is determined based on the vehicle's danger level and the real-time location of the target vehicle. When the vehicle distress request priority is higher than the danger warning priority, the vehicle distress request information is broadcast to emergency service vehicles and roadside terminals within the distress request range via satellite communication and V2X communication, and the danger warning information is broadcast to non-emergency service vehicles within the warning range via Bluetooth communication and 5G communication. When the vehicle's distress request priority is not higher than the danger warning priority, the danger warning information is broadcast to non-emergency service vehicles within the warning range via satellite communication and V2X communication, and to emergency service vehicles and roadside terminals within the distress request range via Bluetooth communication and 5G communication.

6. The vehicle warning method based on driver status monitoring according to claim 1, characterized in that, The vehicle warning method also includes the following steps: The vehicle distress call information and the danger warning information are sent to the vehicle's external display screen, so that the vehicle's external display screen displays the vehicle distress call information and the danger warning information according to priority.

7. A vehicle warning method based on driver status monitoring according to any one of claims 1 to 6, characterized in that, The vehicle warning method also includes the following steps: Determine whether the target driver has the ability to drive manually based on the health risk level; When the target driver does not have the ability to drive manually, the target vehicle is controlled to switch to automatic driving mode, and the optimal parking path is obtained by planning the parking path according to the surrounding environment of the target vehicle. Then, the target vehicle is controlled to brake and stop according to the optimal parking path.

8. A vehicle warning device based on driver status monitoring, characterized in that, include: The data acquisition module is used to acquire facial image data, vital sign data of the target driver, and vehicle behavior data of the target vehicle. An anomaly detection module is used to input the facial image data, the vital signs data, and the vehicle behavior data into a pre-trained anomaly detection model to obtain the health risk level of the target driver and the vehicle danger level of the target vehicle. The information generation module is used to determine the priority of vehicle distress call and danger warning based on the health risk level and the vehicle danger level, respectively, and generate vehicle distress call information and danger warning information. The information broadcasting module is used to broadcast the vehicle distress information to emergency service vehicles and roadside terminals according to priority, and to broadcast the danger warning information to non-emergency service vehicles.

9. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a vehicle warning method based on driver status monitoring as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements a vehicle warning method based on driver status monitoring as described in any one of claims 1 to 7.

Images