A method and system for onboard driver health detection

Abstract

This invention discloses a method and system for in-vehicle driver health monitoring. The method includes the following steps: collecting driver health indicator information to form perception data; fusing and analyzing the perception data, and determining the execution level based on a personalized database; and providing corresponding reminders or interventions to the vehicle according to the execution level. This invention can systematically determine whether a customer has any conditions that affect driving safety, and can promptly influence or intervene in the driving state to ensure the personal and property safety of the customer; it can provide a better monitoring environment and conditions for people who need long-term health monitoring, saving customers hospital monitoring costs and providing greater convenience; and it can provide most people with a health report and a caring experience, taking a significant step forward in improving the quality of vehicle use.

Description

Technical Field

[0001] This invention belongs to the field of intelligent vehicles, specifically to a method and system for detecting the health of in-vehicle drivers. Background Technology

[0002] As living standards rise, people are paying more and more attention to health. This is not only reflected in various popular full-body checkups, but also in the emergence of more intelligent health monitoring methods and scenarios, such as various small home monitoring instruments and devices, online remote diagnosis, and remote medical treatment. At the same time, health monitoring in special scenarios is also receiving increasing attention, such as vehicle-mounted health monitoring.

[0003] There are some solutions for cabin health monitoring, but single-mode health monitoring has a series of problems, mainly in the following aspects.

[0004] First, health indicator monitoring is incomplete. For example, current monitoring solutions include those based on the conductive material of the steering wheel, those based on sensors installed on the gear shift lever, and those based on Doppler monitoring using millimeter-wave radar in the cabin. These solutions can only monitor heart rate and have limitations in monitoring conditions, such as requiring the driver to keep both hands on the steering wheel or the gear shift lever at all times, which is obviously not suitable for driving habits. At the same time, there are also issues with cost and placement for these monitoring solutions. Especially with the continuous refinement of vehicle costs, an increase of more than 100 yuan in cost for a single function undoubtedly brings greater pressure. In addition, millimeter-wave radar has relatively strict requirements for installation location, and the inherent space and fixed position in the cabin will also face great challenges.

[0005] Secondly, even if health monitoring is relatively comprehensive, it cannot be personalized for every individual. For example, a solution based on the cockpit DMS camera can monitor heart rate, respiratory heart rate variability, and even blood pressure, but it cannot accurately determine which indicators are normal and which are abnormal by combining each person's abnormal indicators. As a result, what was originally intended to give users a brand-new experience ends up with frequent errors, which will make customers distrustful and even lead to complaints.

[0006] Third, even if the monitored indicators are relatively comprehensive and can be personalized to each individual, it is still possible that many different driving scenarios may occur, leading to incorrect judgments by the system. For example, when a driver has just finished exercising and is driving, it is obvious that his various indicators are much higher than the normal range. Another example is that when a driver's mood or other reasons cause abnormal non-health indicators, these scenarios need to be fully considered, otherwise the customer experience will be greatly diminished.

[0007] In summary, existing vehicle health monitoring methods suffer from inaccurate monitoring and limited application scenarios. Summary of the Invention

[0008] To address the problems existing in the prior art, the present invention provides a method and system for detecting the health of vehicle drivers, thereby solving the aforementioned problems.

[0009] To achieve the above objectives, the present invention provides the following technical solution:

[0010] A method for detecting the health of a vehicle driver includes the following steps:

[0011] Collect driver health indicator information to form perception data;

[0012] The system integrates and analyzes the perceived data, and uses a personalized database to determine the execution level.

[0013] Provide corresponding reminders or interventions to vehicles based on the level of enforcement.

[0014] Preferably, it includes the following steps:

[0015] Establish an ID account for each user on the cloud platform to store personal health indicator data, forming a personalized database for each individual.

[0016] Driver health indicator information is collected to form perception data. This perception data is compared with a personalized database. When the perception data is abnormal, scenario analysis is performed, and the vehicle is given corresponding reminders or interventions based on the analysis results.

[0017] Preferably, the personalized database is based on the average data collected after each ride. The personal data on the cloud platform is corrected every hundred average data points. Combined with the initial personalized database, the first correction is made to achieve the first version of personalized data. The personalized database is based on the user attribute identification and reference to medical statistical databases to classify health indicators into 10 levels.

[0018] Preferably, the perceived data includes attribute perception, emotion perception, behavior perception, and customer data perception.

[0019] Preferably, the execution level specifically includes,

[0020] If the abnormal range of health indicators is below 15% and all other test dimensions are normal, we will only issue a friendly reminder.

[0021] When health indicators are below 15% abnormal, emotional abnormalities occur, but other conditions are normal, implement mild intervention actions and provide gentle reminders.

[0022] When health indicators are below 15% abnormal, emotions are normal, but driving is abnormal, perform mild intervention actions and provide gentle reminders.

[0023] When the abnormal range of health indicators is below 15%, and all other test dimensions are abnormal, a combination of severe and mild interventions will be implemented after user confirmation.

[0024] When the abnormal range of health indicators is 15%-30% and all monitoring dimensions are normal, only a friendly reminder will be given.

[0025] When health indicators are abnormal within 15%-30%, emotions are abnormal, but other conditions are normal, perform mild intervention actions and provide gentle reminders.

[0026] When the abnormal range of health indicators is 15%-30%, the mood is normal, and other abnormal situations occur, after the user confirms, a combination of severe and mild interventions will be implemented.

[0027] When the abnormal range of health indicators is 15%-30% and all monitoring dimensions are abnormal, drive take-off and mild intervention will be performed after user confirmation.

[0028] When the abnormal range of health indicators exceeds 30% and all monitoring dimensions are positive, implement mild intervention measures and provide a friendly reminder;

[0029] When health indicators are abnormal by more than 30%, or when there are emotional abnormalities, but other conditions are normal, a combination of severe and mild interventions will be implemented after user confirmation.

[0030] When health indicators are abnormal by more than 30%, or when there are abnormal emotions or behaviors, take over driving and perform mild intervention after user confirmation.

[0031] When the abnormal range of health indicators exceeds 30% and all monitoring dimensions are abnormal, driving take-off and mild intervention will be performed after user confirmation.

[0032] Preferably, the vehicle will provide corresponding reminders or interventions in the following four ways:

[0033] Generate a health record report and display the owner's health record report on the vehicle's infotainment interface;

[0034] A friendly reminder will be given when systemic health indicators show mild abnormalities, with a caring greeting delivered via in-car voice prompts and a user-friendly interface.

[0035] Mild intervention is carried out when systemic health indicators reach the mild intervention threshold. Mild intervention actions include adjusting ambient lighting, turning on the air conditioner to cool down to refresh the mind, or adjusting to soothing music to relax.

[0036] Severe intervention is implemented when systemic indicators exceed thresholds. Severe intervention includes connecting to telemedicine, automatically connecting to ECALL function, and allowing the vehicle to pull over or be taken over by autonomous driving to drive directly to the nearest hospital.

[0037] An in-vehicle driver health monitoring system includes a sensing module, a decision-making module, and an execution module;

[0038] The sensing module is used to detect driver health indicators and generate sensing data.

[0039] The decision-making module is used to fuse and analyze the perceived data, combine it with personalized data, and rely on a continuously revised algorithm model to analyze and make decisions to form decision instructions.

[0040] The execution module is used to receive decision instructions and execute actions.

[0041] An in-vehicle driver health monitoring system, including a vehicle-side system;

[0042] The vehicle-mounted system is used to collect driver health indicators to form perception data and upload it to the cloud system.

[0043] The vehicle-mounted system receives decision instructions generated by the cloud system through analysis and decision-making, and executes corresponding reminders or interventions based on the decision instructions.

[0044] An in-vehicle driver health monitoring system, including a cloud system;

[0045] The cloud system is used to receive driver health indicators collected by the vehicle system to form perception data; the cloud system stores the received data; the cloud system processes the received data based on a personalized database and sends the decision instructions to the vehicle system.

[0046] Compared with the prior art, the present invention has the following beneficial technical effects:

[0047] This invention provides an in-vehicle driver health detection method that can systematically determine whether a customer has any conditions affecting driving safety, and can promptly influence or intervene in the driving state to ensure the personal and property safety of the customer. It provides a better monitoring environment and conditions for people requiring long-term health monitoring, saving customers hospital monitoring costs and providing greater convenience. It offers health reports and a caring experience to most people, taking a significant step forward in improving the quality of vehicle use. The invention features an industry-first multi-dimensional perception fusion algorithm that can achieve personalized detection for each individual with accuracy comparable to medical standards. It also has numerous intelligent application scenarios, embodying a technologically advanced and emotionally resonant intelligent cockpit, empowering OEM brands. Attached Figure Description

[0048] Figure 1 This is a block diagram of the system principle.

[0049] Figure 2 This is a diagram of the fusion algorithm model.

[0050] Figure 3 This is a system architecture diagram. Detailed Implementation

[0051] The present invention will be further described in detail below with reference to specific embodiments. These descriptions are for explanation purposes only and are not intended to limit the scope of the invention.

[0052] This invention provides an in-vehicle driver health detection method, which is a fusion monitoring method and algorithm model. The fusion monitoring method includes, but is not limited to, the following: Based on a visual approach, it utilizes the optical reflection principle of an in-cabin DMS infrared camera to acquire driver health indicators, including heart rate, heart rate variability, blood pressure, and respiration; simultaneously, it provides mood characteristics based on camera footage, including the recognition of more than ten emotions such as normal facial expressions, happiness, sadness, and anger; driver attribute detection, including behaviors that affect driving safety such as smoking, looking around, and making phone calls; driver attribute monitoring, including information such as gender and age; and continuously learning and recording driver health indicators, facial expression characteristics, driving habits, and corresponding vehicle data; fusion perception and monitoring are performed based on these dimensions.

[0053] The algorithm model of this invention mainly refers to the algorithm system deployed in the cloud, which is continuously updated in combination with the above data and managed separately by personal tags; based on age, gender, etc., ten inherent algorithm models are initially divided into basic models, which are analyzed and learned in combination with a continuous stream of personal data, and finally grow into a personalized algorithm model that is unique to each individual.

[0054] The application scenarios protected by this invention mainly refer to different application scenarios under the cloud algorithm model, combined with different stages of model growth. For example, in the initial stage, it is only used to push health reports to customers periodically, mainly through push notifications on the vehicle's terminal and mobile phone, and customers actively checking the information. In the background model growth stage, the vehicle can make corresponding reminders and simple interventions, such as providing a warm voice greeting and making corresponding changes to the in-vehicle environment when abnormalities occur, such as adjusting the air conditioning, playing soothing music, and reminding customers of the outside lights. In the later stage of model development, it can be used for remote medical care or even take over autonomous driving.

[0055] For users, this invention offers several advantages: First, it provides a systematic assessment of whether a customer is experiencing any issues that could affect driving safety, allowing for timely intervention or control of the driving process to ensure personal and property safety. Second, it offers a better monitoring environment and conditions for individuals requiring long-term health monitoring, saving them hospital monitoring costs and providing greater convenience. Third, it provides most people with health reports and a caring experience, representing a significant step forward in improving the quality of vehicle use.

[0056] The invention features an industry-first multi-dimensional perception fusion algorithm that can achieve personalized results for each user with detection accuracy comparable to medical standards. It also has numerous intelligent application scenarios, embodying a technologically advanced and emotionally resonant intelligent cockpit, thus empowering OEM brands.

[0057] Example

[0058] like Figure 1 As shown, an in-vehicle driver health monitoring system of the present invention includes a sensing module, a decision-making module and an execution module;

[0059] The perception module includes driver health indicator detection, including but not limited to heart rate, respiration, blood pressure, and heart rate variability indicators. This detection is the core of the perception content.

[0060] Attribute perception includes driver gender detection and age detection; its function is to initially classify the general population and establish preliminary health indicator ranges.

[0061] Emotional perception mainly includes normal emotions such as happiness, sadness, and grief. Its role is to assist in determining whether health indicators are truly abnormal.

[0062] Behavioral perception mainly includes the driver's posture and eye contact; it detects whether the driver's posture is tilted forward, backward, left, or right, and whether there are any abnormalities in the head position; eye contact detection mainly looks for abnormalities such as dull eyes or closed eyes, and its main function is to assist in the accuracy of health indicators.

[0063] Customer data perception mainly includes customer driving data and vehicle data. The driving data is mainly used to see if there is abnormal driving behavior when health indicators are abnormal, and the purpose is to verify the accuracy of health indicators.

[0064] The decision-making module mainly focuses on the fusion and analysis of perceived data, combining personalized data from different individuals, and relying on continuously improved algorithm models to analyze and make decisions, ultimately providing several levels of execution for the vehicle to handle.

[0065] The execution module primarily receives decision-making instructions and executes actions. Based on health indicator levels, it is divided into four levels: gentle reminder, mild intervention, and severe intervention. Application scenarios include:

[0066] Health record reports can be viewed by clicking on the vehicle's infotainment system's human-machine interface. Users can also be notified via voice prompts and pop-up windows whenever a new report is available. Furthermore, reminders and viewing can also be performed on a mobile device.

[0067] A friendly reminder: when there are mild abnormalities in systemic health indicators, a friendly reminder can be sent via in-car voice prompts and a friendly interface to offer care and greetings.

[0068] Mild intervention is implemented when systemic health indicators reach the threshold for mild intervention. Mild intervention actions include adjusting ambient lighting, turning on the air conditioner to cool down and playing soothing music to relax.

[0069] Severe intervention is implemented when systemic indicators exceed thresholds. Severe intervention includes connecting to telemedicine, automatically connecting to ECALL function, and allowing the vehicle to pull over or be taken over by autonomous driving to drive directly to the nearest hospital.

[0070] like Figure 2 As shown, the present invention provides a method for detecting the health of a vehicle driver. The fusion algorithm is deployed in the cloud and is based on the fusion processing of data uploaded from the vehicle. The basic research content includes the following aspects:

[0071] First, an ID account is created for each user on the cloud platform to store personal health indicator data, gender, age, mood, driving behavior, and other personal data. An initial range of health indicators is established based on the typical health status of Chinese people. Combined with user attribute identification (gender, age), the health indicators are divided into 10 levels, which can be referenced from medical statistical databases. This first step aims to achieve a personalized approach for each individual.

[0072] By continuously analyzing user health data and averaging the data collected after each ride, the cloud platform's personal data is corrected every hundred average data points. This initial correction, combined with the initial personalized database, achieves the first version of a personalized experience. Simultaneously, as the number of drivers' data increases, the platform continuously refreshes and iterates to ensure that personal data is always up-to-date and maintains real-time personalized updates. When health indicators are abnormal, the following flowchart is used to integrate and analyze different moods, driving behaviors, and user data, and the results are exported according to the logic diagram.

[0073] When abnormal detection indicators occur, but emotions and driving behavior are within the normal range, scenario analysis is required. In particular, after user confirmation, the fusion algorithm is continuously corrected, updated, and self-learned to continuously enhance the scenario and accuracy of the fusion algorithm. At the same time, it is combined with a continuous stream of personal data for analysis and learning, and finally grows into a personalized algorithm model that is tailored to each individual.

[0074] The system combines the levels output by the fusion algorithm to perform vehicle-related interventions. For example, it can provide a friendly reminder when there is a minor abnormality, mainly through voice and interface display. The interface includes a user confirmation feedback entry to continuously improve the algorithm model. At the same time, health records can be pushed and queried on mobile phones and in-vehicle systems.

[0075] At the second level, mild interventions can be made, such as ambient lighting, air conditioning, and soothing music, to make the cabin environment more comfortable. There are also button or voice confirmation feedback entry points to correct the fusion algorithm.

[0076] At the third level, telemedicine and ECALL can be used to handle emergencies; user confirmation is also required before responding.

[0077] The fourth level integrates autonomous driving scenarios, also for handling emergency situations.

[0078] The protected fusion algorithm is as follows:

[0079] If the abnormal range of health indicators is below 15% and all other test dimensions are normal, we will only issue a friendly reminder.

[0080] When health indicators are below 15% abnormal, emotional abnormalities occur, but other conditions are normal, mild intervention actions and gentle reminders are implemented.

[0081] When health indicators are below 15% abnormal, emotions are normal, but driving is abnormal, perform mild intervention actions and provide gentle reminders.

[0082] When the abnormal range of health indicators is below 15%, and all other test dimensions are abnormal, a combination of severe and mild interventions will be implemented after user confirmation.

[0083] When the abnormal range of health indicators is 15%-30% and all monitoring dimensions are normal, only a friendly reminder is given.

[0084] When health indicators are abnormal within 15%-30%, emotions are abnormal, but other conditions are normal, perform mild intervention actions and provide gentle reminders.

[0085] When the abnormal range of health indicators is 15%-30%, the mood is normal, and other abnormal conditions are confirmed by the user, a combination of severe and mild interventions will be implemented.

[0086] When the abnormal range of health indicators is 15%-30% and all monitoring dimensions are abnormal, driving take-off and mild intervention will be performed after user confirmation.

[0087] When the abnormal range of health indicators exceeds 30%, and all monitoring dimensions are positive, implement mild intervention measures and provide a friendly reminder.

[0088] When health indicators are abnormal by more than 30%, or when there are emotional abnormalities, but other conditions are normal, a combination of severe and mild interventions will be implemented after user confirmation.

[0089] When health indicators are abnormal by more than 30%, or when there are abnormal emotions or behaviors, take over driving and perform mild intervention after user confirmation.

[0090] When the abnormal range of health indicators exceeds 30% and all monitoring dimensions are abnormal, driving take-off and mild intervention will be performed after user confirmation.

[0091] The above distinctions between abnormal value ranges for health indicators are merely examples and do not constitute a fixed protection level of 15% or 30%.

[0092] The monitored health indicators, mood, driving behavior, and other data should be refreshed at fixed intervals, and monitoring could be considered every 10 seconds.

[0093] like Figure 3 As shown, the architecture of the in-vehicle driver health monitoring system of the present invention is mainly divided into a vehicle-side system and a cloud-side system. The vehicle-side system is responsible for fusion sensing and data uploading; the cloud-side system includes a big data platform, a TSP platform, and a health cloud platform. The big data platform mainly stores and transmits personal health indicator data, attributes, behavioral analysis, emotion, and other data; the TSP platform mainly stores and transmits user driving behavior data to the health cloud platform; after receiving the data from each dimension, the health cloud platform analyzes and processes the data based on the fusion algorithm and outputs the health level result to the vehicle-side system; after receiving the level conclusion, the vehicle-side system executes vehicle control, as shown in the system architecture diagram below.

[0094] The vehicle-side system packages the collected personal health data, driver attribute data, emotion data, and driver behavior perception data, adds ID tags, and uploads them to the big data platform in sequence number 1. The big data platform stores the data and simultaneously transmits it to the health cloud platform.

[0095] The driver's driving behavior data is uploaded directly to the TSP platform according to the original architecture of the traditional OEM, via Line 2. At the same time, the TSP directly transmits this data to the health cloud platform.

[0096] After obtaining data from various dimensions, the health cloud platform performs data fusion analysis, according to the attached... Figure 2 The fusion algorithm is used for decision processing, and the processed ranking results are then distributed.

[0097] The health monitoring application in the vehicle-side DMC will connect with the health cloud. After receiving the results according to path 3, it will control and process them according to the level. At the same time, the vehicle-side has an HMI human-machine interface and a voice interaction channel. Based on the user feedback from the previous few times and uploaded to the cloud platform according to path 3, the individual algorithm model will be continuously calibrated and optimized.

[0098] According to the anomaly level, DMC will issue and execute the action commands to be performed by the vehicle through the in-vehicle network architecture in the categories of 4, 5, and 6 respectively;

[0099] If remote medical intervention is involved, the remote medical platform will be connected via path 7 through a 4G / 5G network;

[0100] If reminders and interventions are made via mobile devices, the health level will be sent to the TSP via the cloud platform according to path 2. The TSP will then translate the instructions and send them to the smartphone according to path 8.

Claims

1. A method for detecting the health of a vehicle driver, characterized in that, Includes the following processes, Establish an ID account for each user on the cloud platform to store personal health indicator data, forming a personalized database for each individual. The system collects driver health indicator information to form perception data, which is then compared with a personalized database. When the perception data is abnormal, scenario analysis is performed, and the vehicle provides corresponding reminders or interventions based on the analysis results. The personalized database is compiled by statistically analyzing the average data collected after each ride. The personal data on the cloud platform is corrected every hundred average data points. Combined with the initial personalized database, the first correction is performed to achieve the first version of personalized driving. The personalized database is divided into 10 levels based on user attribute identification and reference to medical statistical databases. Collect driver health indicator information to form perception data; The system integrates and analyzes the perceived data, and uses a personalized database to determine the execution level. Provide corresponding reminders or interventions to the vehicle based on the level of enforcement; The perceived data includes attribute perception, emotion perception, behavior perception, and customer data perception. The execution levels specifically include, If the abnormal range of health indicators is below 15% and all other test dimensions are normal, we will only issue a friendly reminder. When health indicators are below 15% abnormal, emotional abnormalities occur, but other conditions are normal, implement mild intervention actions and provide gentle reminders. When health indicators are below 15% abnormal, emotions are normal, but driving is abnormal, perform mild intervention actions and provide gentle reminders. When the abnormal range of health indicators is below 15%, and all other test dimensions are abnormal, a combination of severe and mild interventions will be implemented after user confirmation. When the abnormal range of health indicators is 15%-30% and all monitoring dimensions are normal, only a friendly reminder will be given. When health indicators are abnormal within 15%-30%, emotions are abnormal, but other conditions are normal, perform mild intervention actions and provide gentle reminders. When the abnormal range of health indicators is 15%-30%, the mood is normal, and other abnormal situations occur, after the user confirms, a combination of severe and mild interventions will be implemented. When the abnormal range of health indicators is 15%-30% and all monitoring dimensions are abnormal, drive take-off and mild intervention will be performed after user confirmation. When the abnormal range of health indicators exceeds 30% and all monitoring dimensions are positive, implement mild intervention measures and provide a friendly reminder; When health indicators are abnormal by more than 30%, or when there are emotional abnormalities, but other conditions are normal, a combination of severe and mild interventions will be implemented after user confirmation. When health indicators are abnormal by more than 30%, or when there are abnormal emotions or behaviors, take over driving and perform mild intervention after user confirmation. When the abnormal range of health indicators exceeds 30% and all monitoring dimensions are abnormal, drive take-off and mild intervention will be performed after user confirmation. The vehicle will provide corresponding reminders or interventions in the following four ways: Generate a health record report and display the owner's health record report on the vehicle's infotainment interface; A friendly reminder will be given when systemic health indicators show mild abnormalities, with a caring greeting delivered via in-car voice prompts and a user-friendly interface. Mild intervention is carried out when systemic health indicators reach the mild intervention threshold. Mild intervention actions include adjusting ambient lighting, turning on the air conditioner to cool down to refresh the mind, or adjusting to soothing music to relax. Severe intervention is implemented when systemic indicators exceed thresholds. Severe intervention includes connecting to telemedicine, automatically connecting to ECALL function, and allowing the vehicle to pull over or be taken over by autonomous driving to drive directly to the nearest hospital.

2. An on-board driver health detection system, based on the on-board driver health detection method according to claim 1, characterized in that, It includes a perception module, a decision-making module, and an execution module; The sensing module is used to detect driver health indicators and generate sensing data. The decision-making module is used to fuse and analyze the perceived data, combine it with personalized data, and rely on a continuously revised algorithm model to analyze and make decisions to form decision instructions. The execution module is used to receive decision instructions and execute actions.

3. An on-board driver health detection system, based on the on-board driver health detection method according to claim 1, characterized in that, Including vehicle-side systems; The vehicle-mounted system is used to collect driver health indicators to form perception data and upload it to the cloud system. The vehicle-mounted system receives decision instructions generated by the cloud system through analysis and decision-making, and executes corresponding reminders or interventions based on the decision instructions.

4. An on-board driver health detection system, based on the on-board driver health detection method according to claim 1, characterized in that, Including cloud systems; The cloud system is used to receive driver health indicators collected by the vehicle-side system to form perception data; The cloud system stores the received data; it then processes the data based on a personalized database and sends the decision instructions to the vehicle-side system.

Images