Grading early warning system based on driver health state

Abstract

The invention discloses a grading early warning system based on the health state of a driver, and the system is characterized in that the system comprises a data collection module, a data processing module, and a grading early warning module; wherein the data acquisition module is used for acquiring vehicle data, driver physiological feature data and environment data; the data processing module is used for performing feature extraction on the collected multi-source data; the grading early warning module carries out grading early warning on the driver according to the processed multi-source data, through multi-mode data fusion, higher anomaly detection accuracy is achieved, abnormal conditions can be recognized more accurately, missing detection and false detection are reduced, and the driving safety is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent automobile safety, in particular to a driver health state grading early warning system. BACKGROUND

[0002] In the field of intelligent automobile safety, with the rapid development of automatic driving technology and Internet of Vehicles technology, how to effectively improve the safety during driving and reduce traffic accidents caused by abnormal driver health state has become the focus of the industry. At present, although there are various driving safety systems based on biological feature monitoring in the market, these systems have obvious limitations and are difficult to meet the needs of complex driving environments.

[0003] In the prior art, many driving safety systems rely on single biological feature detection, such as only using heart rate monitoring to evaluate the health state of the driver. However, single biological feature detection has a high false positive rate (> 15%), which is mainly because physiological signals are easily disturbed by the motion state of the driver, emotional fluctuations and environmental factors. For example, the heart rate may be accelerated due to the nervousness of the driver or the jolt during vehicle driving, but this does not necessarily mean that the driver is in an unsafe state. Therefore, it is difficult to accurately distinguish between physiological abnormalities and motion interference only by heart rate monitoring, resulting in frequent false positives of the system and reducing the driving experience and safety. SUMMARY

[0004] The present application provides a driver health state grading early warning system, which realizes higher abnormal detection accuracy through multi-modal data fusion, can more accurately identify abnormal conditions, reduces missed detection and false detection, and improves driving safety.

[0005] The present application provides a driver health state grading early warning system, which includes a data acquisition module, a data processing module and a grading early warning module. The data acquisition module is used to acquire vehicle data, driver physiological feature data and environmental data. The data processing module is used to extract features from the collected multi-source data. The grading early warning module performs grading early warning on the driver according to the processed multi-source data.

[0006] Preferably, the grading early warning module identifies the response of the driver after early warning by introducing a load outlier index, and the identification method is as follows: S101, acquiring multi-source data through the data acquisition module, the multi-source data including vehicle data, driver physiological feature data and environmental data; S102, performing grading early warning on the driver and calculating the load outlier index according to the collected multi-source data, and identifying the response of the driver after early warning through the load outlier index.

[0007] Preferably, the system further comprises a posture recognition module and a health prediction model construction module, the posture recognition module collects posture data of the driver, and the health prediction model construction module is configured to construct a health prediction model.

[0008] Preferably, the hierarchical early warning module adopts a hierarchical early warning method comprising the following steps: S201, the posture recognition module collects posture data, calculates a fatigue index based on the posture data, identifies a posture state, and divides a response level according to the posture state; S202, a health prediction model is constructed based on the obtained posture data and multi-source data, and a future health state of the driver is predicted based on the health prediction model.

[0009] Preferably, the formula for calculating the load outlier index is: wherein, is the load outlier index, n represents the number of physiological parameters participating in the calculation, is the weight of each physiological parameter, is the value of the i-th physiological parameter measured at the current time, is the historical mean value of the i-th physiological parameter, and ∑ represents a covariance matrix describing the correlation between the physiological parameters, is the inverse matrix of the covariance.

[0010] Preferably, the formula for calculating the fatigue index is: wherein, is the fatigue index, is a seat cushion pressure distribution uniformity index, is a trunk micro-motion frequency change index, and F is a heart rate fluctuation index, , and are weight coefficients of the seat cushion pressure distribution uniformity index, the trunk micro-motion frequency change index, and the heart rate fluctuation index, respectively, wherein, + .

[0011] Preferably, the data processing module is integrated with a clock chip, adds a time stamp to each collected data, and detects abnormal data.

[0012] One or more technical solutions provided in the present application have at least the following technical effects or advantages: through multi-modal data fusion, higher accuracy of abnormality detection is achieved, abnormal situations can be more accurately identified, missed detection and false detection are reduced, driving safety is improved, precise evaluation and timely intervention of the driver's suitability for the post are realized, and driving safety is improved and the risk of traffic accidents is reduced. Through the risk prediction model, early warning of the driver health risk can be realized, potential health problems are found in advance, more response time is obtained for the driver, a detailed three-level response execution system is set according to the fatigue index, the posture abnormal condition and the health risk score, each level has a clear trigger condition and execution action, the driving safety of the driver is comprehensively ensured, and traffic accidents caused by physical discomfort or fatigue are reduced. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 a block diagram of a driver health state grading early warning system of the present application; Figure 2 a flowchart of data processing of the present application; Figure 3 a flowchart of response identification after driver early warning of the present application; Figure 4 a flowchart of a grading early warning method of the grading early warning module of the present application. DETAILED DESCRIPTION

[0014] In order to facilitate the understanding of the present application, the present application will be described more fully below with reference to the accompanying drawings; the preferred embodiments of the present application are shown in the drawings, however, the present application can be realized in many different forms and is not limited to the embodiments described herein; on the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0015] It should be noted that the terms "vertical", "horizontal", "upper", "lower", "left", "right" and similar expressions used herein are only for the purpose of illustration and do not represent the only implementation.

[0016] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs; the terms used herein in the specification of the present application are only for the purpose of describing the specific embodiments and are not intended to limit the present application; the term "and / or" used herein includes any and all combinations of one or more related listed items.

[0017] Embodiment one: Figure 1 is a block diagram of a driver health state grading early warning system of the present application, comprising: the system comprises a data acquisition module, a data processing module and a grading early warning module; the data acquisition module is used for acquiring multi-source data, the data processing module is used for extracting features from the acquired multi-source data, the grading early warning module grades and early warns the driver according to the processed multi-source data, the response of the driver after early warning is identified by introducing a load outlier index, such as Figure 3The recognition method is shown as follows: S101, collecting multi-source data through a data acquisition module, wherein the multi-source data includes vehicle data, driver physiological feature data and environmental data; Further, the triaxial acceleration sensor is installed in the middle of the vehicle chassis, the range of the triaxial acceleration sensor is set to ±8g, the sampling frequency is set to 100Hz, the steering angle sensor is installed on the steering column below the steering wheel to measure the rotation angle of the steering wheel, the brake pressure data and the vehicle speed data are collected by the pressure sensor and the heat exchange speed sensor, the data collected by the data acquisition module is transmitted to the data processing module through the CAN FD bus (5Mbps), the clock chip is integrated in the data processing module, the clock chip is used to add a time stamp to each collected data, so that the multi-source data is consistent in time, the sliding window standard deviation analysis method is used to detect abnormal data, the window size and the standard deviation threshold are set to filter the data, when the standard deviation of the data exceeds the set standard deviation threshold, the data is considered to be abnormal data and is excluded; the monitoring of the driver physiological feature data is divided into contact monitoring and non-contact monitoring, the 6-point capacitive matrix array sensor (FDC2214) is embedded in the 3-point or 9-point position of the steering wheel for contact monitoring, the size of the grip force is reflected by measuring the change of the capacitance value, the grip force threshold is set, when the measured capacitance value is less than the set grip force threshold, the grip force is considered to disappear, the duration of each grip force disappearance is recorded, the feature reflects the degree of distraction and fatigue state of the driver, for example: the grip force of the driver frequently disappears for a period of time during driving, indicating that the driver is in a state of fatigue or distraction, the non-contact monitoring uses the infrared camera (OV2740) to collect the driver's face image and uses the dual-wavelength PPG sensor to detect the heart rate, the wavelength of the infrared camera (OV2740) is set to 940nm near-infrared light, the frame rate is set to 60fps, which can track the facial expression and motion change of the driver in real time, the time-of-flight (ToF) technology is used for 3D imaging, the facial feature point detection algorithm (Adaboost algorithm) is used to locate the feature points of the driver's face, including eyes, eyebrows, nose and mouth, according to the located eye feature points, the image region where the eyes are located is extracted, the gray mean feature value of the eye region is calculated, the feature value threshold is set, when the feature value is less than the feature value threshold, the eyes are closed; when the feature value is greater than the feature threshold, the eyes are open, the labeled eye open and closed image samples and the extracted eye region features are input into the trained support vector machine (SVM) model, the model outputs the eye state judgment result, the time window is set, the eye closure events of the driver in the time window are counted, the proportion of the eye closure time of the driver in the time window to the total time is calculated, the formula is: wherein, a proportion of a time in which the driver's eyes are closed in a time window accounts for a total time, a total time in which the eyes are closed, T represents a total time of the time window, a fatigue threshold is set, when the calculated proportion is greater than the fatigue threshold, it is determined that the driver is in a fatigue state; the non-contact monitoring of the dual-wavelength PPG sensor does not interfere with the driver and can penetrate clothes and thin obstacles; environmental data such as light intensity, temperature and humidity, and rainfall are measured using the sensor, and the collected data is also transmitted to the data processing module for preprocessing.

[0018] S102, according to the collected multi-source data, the driver is given a hierarchical early warning and a load outlier index is calculated, and the response of the driver after the early warning is identified through the load outlier index; Further, according to the size of the real-time monitored steering wheel internal capacitance value and the real-time speed of the vehicle, a first level grip force time threshold and a first level speed threshold are set, when the duration of grip force disappearance is greater than the first level grip force time threshold and the vehicle speed is greater than the first level speed threshold, a first level early warning is triggered, after the local micro control unit (MCU) receives the trigger signal, the vibration device is immediately started, the vibration device is LRA linear motor, the LRA linear motor is installed inside the seat to produce three zone vibration, the vibration device vibrates according to the preset frequency and intensity, to remind the driver to pay attention to the abnormal state of the steering wheel, the MCU sends instructions to the instrument panel at the same time, and the warning light on the instrument panel is lit, the warning light is red; According to the calculation, the proportion of the time when the driver's eyes are closed in the total time in the time window is obtained, the heart rate variability (HRV) data of the driver is obtained through the vehicle physiological monitoring equipment (smart watch), when the proportion of the time when the driver's eyes are closed in the total time is greater than the set fatigue threshold, and the HRV is lower than the set threshold, a second level early warning is triggered, after the domain controller receives the trigger signal, the vibration device is immediately started, instructions are sent to the air conditioning system, and the temperature and air volume of the air conditioning are adjusted according to the preset algorithm, for example, if the environmental temperature is high and the driver is in a fatigue state, the air conditioner will automatically reduce the temperature and increase the air volume, so as to create a more comfortable driving environment and relieve the fatigue of the driver, the domain controller communicates and controls the double flash to start at the same time, obtains the current position of the vehicle and the information of the surrounding rest area, then, through the voice prompt and the display of the navigation interface, the nearest rest area is recommended for the driver, and the best driving route is planned to guide the driver to rest; When the vehicle vital sign monitoring equipment (medical grade sensor) monitors the heart rate, respiration and other physiological parameters of the driver in real time, according to the medical data, the normal range of heart rate and the threshold of respiratory frequency are set, when the heart rate is not in the set normal range and the respiratory frequency is less than the threshold of respiratory frequency, a third level early warning is triggered, the domain controller sends L3 level automatic driving to the electronic stability program (ESP) system of the vehicle through 5G-V2X (vehicle to everything) technology, realizes parking by the side and controls the vehicle to start, after the ESP system receives the instruction, the braking system and the power system of the vehicle are controlled quickly, automatic emergency braking is realized, the vehicle stops as soon as possible and safely, at the same time, the domain controller contacts the emergency rescue center to send SOS signal through 5G-V2X technology, sends the real-time position of the vehicle, the vital sign information of the driver and the key data such as the fault condition of the vehicle to the rescue center, the rescue center can respond quickly according to these information, arrange rescue personnel and vehicles to go to the scene for rescue.

[0019] According to the collected multi-source data, the load outlier index is calculated, the formula is: , wherein, represents the load outlier index, n represents the number of physiological parameters participating in the calculation, The weight of each physiological parameter is determined according to experimental data, The value of the i-th physiological parameter measured at the current time, The historical mean value of the i-th physiological parameter, ∑ represents the covariance matrix, which describes the correlation between each physiological parameter, The inverse matrix of the covariance, the load outlier index is calculated to judge the driver's state, when the PLI value is greater than the preset load threshold, it indicates that the driver's load is large, in this case, the driver is in a state of tension, fatigue or inattention, etc. In this case, the driver's emergency response ability will be affected. The time point at which the driver steps on the brake pedal is measured by a sensor, and the time point at which the vehicle starts to decelerate is obtained by the vehicle control system. The difference between the two time points is the brake response time of the driver. This time difference reflects the reaction speed of the driver from discovering an emergency to taking braking measures. A brake time threshold is set. If the brake response time of the driver is less than the brake time threshold, it means that the driver can quickly respond in an emergency and has good reaction speed and emergency handling ability. On the contrary, if the brake response time is greater than the brake time threshold, the driver's reaction ability and driving state need to be concerned. Because the driver is tired, distracted or for other reasons, the reaction is slow, and the driver needs to be reminded to pay attention to safety or make appropriate adjustments. The steering wheel sensor is used to monitor the rotation angle of the steering wheel in real time. When the vehicle encounters obstacles or needs to avoid obstacles during driving, the angle of the steering wheel turned by the driver is recorded. A steering correction amplitude range is set. If the steering correction amplitude of the driver when avoiding obstacles is within the preset steering correction amplitude range, it means that the operation is accurate and the driver can effectively control the driving direction of the vehicle, which indicates that the driver can remain calm when facing unexpected situations, accurately judge the driving trajectory of the vehicle, and take appropriate steering operation. If the steering correction amplitude is too large or too small, it means that the driver's operation is not accurate or there are tension, panic and other situations.

[0020] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages: through multi-modal data fusion, higher anomaly detection accuracy is achieved, abnormal situations can be more accurately identified, missed detection and false detection are reduced, driving safety is improved, accurate evaluation and timely intervention of the driver's suitability for the job are realized, and driving safety is improved and traffic accident risk is reduced.

[0021] Embodiment two: based on the monitoring of real-time data in embodiment one, this embodiment improves the accuracy of health status recognition by warning the health status and identifying the posture, reduces traffic accidents caused by physical discomfort or fatigue, as shown in Figure 2 and Figure 4 .

[0022] S201, collect posture data, calculate fatigue index based on posture data, identify posture state, and divide response level according to posture state; Further, the posture data includes pressure distribution data, trunk micro-motion frequency data, and heart rate data. A 64-point pressure-sensitive matrix is installed in the seat cushion area, a capacitive fit sensor is integrated in the seat back area, and a millimeter wave radar is installed at a suitable position in the vehicle for monitoring the driver's trunk micro-motion frequency. The 64-point pressure-sensitive matrix continuously detects the pressure distribution of each point in the cushion area, the capacitive fit sensor real-time senses the fit degree of the human body and the back, the millimeter wave radar monitors the micro-motion frequency of the driver's trunk, records the data, and identifies the disappearance of grip strength. The fatigue index is calculated based on the obtained posture data, and the formula is: wherein, is the fatigue index, which is usually set in the range of 0-1, and the larger the value, the higher the fatigue degree of the driver, is the seat cushion pressure distribution uniformity index, wherein, is the standard deviation of the pressure values of all pressure-sensitive points, is the average value of the pressure values of all pressure-sensitive points, The value of is in the range of 0-1, and the closer the value is to 1, the more uniform the pressure distribution, the more stable the driver's sitting posture, and the lower the fatigue degree; on the contrary, the closer the value is to 0, the more uneven the pressure distribution, and the driver may be in a fatigue or uncomfortable sitting posture, is the trunk micro-motion frequency change index, and F is the heart rate fluctuation index, and F are also obtained by standard deviation ratio and average value, , and are the weight coefficients of the seat cushion pressure distribution uniformity index, the trunk micro-motion frequency change index, and the heart rate fluctuation index, respectively, wherein, ​According to the calculated fatigue index, fatigue levels are classified, and light threshold and moderate threshold are set according to the collected posture data. When the fatigue index is greater than or equal to zero and less than the light threshold, the driver is in a light fatigue state. At this time, the system will display prompt information through the in-vehicle display screen, such as please pay attention to proper rest and keep alert driving, reminding the driver to pay attention to his own state. When the fatigue index is greater than or equal to the light threshold and less than the moderate threshold, the driver is in a moderate fatigue state. In addition to displaying prompt information, the system will also trigger the voice warning function and issue a more urgent voice reminder, such as you are in a moderate fatigue state, please find a safe place to park and rest as soon as possible. When the fatigue index is greater than or equal to the moderate threshold and less than or equal to one, the driver is in a severe fatigue state. The system will immediately take emergency measures, such as automatically turning on the double flasher, slowly reducing the vehicle speed, and issuing strong warning information through voice and display screen at the same time, such as severe fatigue driving, the system is taking emergency measures, please stop and rest immediately to maximize driving safety.

[0023] The sensor data features under different abnormal postures are collected through sensors, and the collected abnormal posture data is feature extracted, for example: for the epilepsy seizure mode, the extracted features may include trunk micro-motion frequency at 4-7Hz, abnormal gravity center offset of seat cushion area pressure distribution, abnormal fit of backrest area, detection of both hands off the steering wheel by the steering wheel capacitance sensor and abnormal heart rate fluctuation, etc. For the fatigue posture, the extracted features may include reduced trunk micro-motion frequency, uneven seat cushion area pressure distribution and gradually forward gravity center, reduced fit of backrest area, long-term slight hands off the steering wheel or reduced grip strength, and slow heart rate. The extracted features of each abnormal posture are stored in a database to form an abnormal mode library, and each mode is assigned a unique identifier. At the same time, the feature weights of each mode are recorded, and a weighted similarity matching algorithm is used to calculate the matching degree of real-time features and each mode in the abnormal mode library, the formula is: wherein, is the matching degree of real-time data and abnormal mode, the value range is between [0, 1], the larger the value, the higher the matching degree, n is the number of feature weights of abnormal mode, the weight of the jth feature, is the real-time feature and the mode feature The similarity between the calculated matching degree and the preset matching degree threshold is compared, and it is judged whether the real-time posture matches an abnormal mode. If the matching degree is greater than or equal to the matching degree threshold, it is considered that the posture of the driver matches the abnormal mode and may be in an abnormal state; if the matching degree is less than the matching degree threshold, it is considered that the posture of the driver is normal. The determination of the threshold can be determined through a large number of experiments and data analysis to ensure the accuracy and reliability of the judgment. According to the posture state division response level, when the system detects an abnormal posture for 5 seconds, it is a first-level response, and the in-vehicle voice system inquires: Do you need help? At the same time, the state and response of the driver are continuously monitored; if the driver does not respond after the first-level response voice inquiry, and the heart rate is out of the normal range, it is a second-level response, and the system automatically opens the vehicle window to 30% opening to improve the air circulation in the vehicle and further attract the attention of the driver; through the sensor and identification to long-time non-response, a third-level response is started, and double flashing is started to alert surrounding vehicles, the vehicle keeps lane driving and gradually stops on the roadside, and automatically calls for rescue, sends the vehicle position, driver state and other information to the preset rescue center, and waits for rescue.

[0024] S202, constructing a health prediction model according to the obtained posture data and multi-source data, and predicting the future health state of the driver according to the health prediction model; Specifically, the features of the posture data, vehicle data, driver physiological feature data and environmental data are extracted. For physiological features, indicators such as standard deviation and root mean square difference of heart rate are extracted to reflect the change of heart rate. For vehicle data, indicators such as standard deviation of steering wheel rotation, change rate of accelerator and brake pedal depression force are extracted. For posture data, pressure distribution and adhesion degree are extracted. For environmental data, the comfort index of the vehicle environment is extracted. The data set of the obtained historical data is divided into a training set, a validation set and a test set, and the proportion is usually 7:1:2. The training set is used for model training, the validation set is used for adjusting the hyperparameters of the model, and the test set is used for evaluating the generalization ability of the model. The random forest algorithm is used as the health prediction model, and the grid search algorithm is used to optimize the hyperparameters of the random forest model, including the number of decision trees, the maximum depth, the minimum leaf node sample number, etc. Through cross-validation on the validation set, the hyperparameter combination that makes the model performance optimal is selected to construct the health prediction model. The real-time acquired posture data, vehicle data, driver physiological feature data and environmental data are input into the trained health prediction model, and the health prediction model outputs a health risk probability. The health risk probability is used to calculate the health score of the driver, and the formula is: wherein, is the health risk score, n is the number of features, is the importance weight of the i-th feature, To predict the health risk probability corresponding to the i-th feature output by the model, the driver's health risk is divided into three levels: low risk, medium risk, and high risk, based on the size of the health risk score. The specific threshold for this division can be determined based on the actual application scenario and expert experience. For example: Low risk, with a health risk score in the range [0, 0.3), indicates that the driver is in a state of fatigue or potential health risk, but has not yet reached the level requiring immediate intervention; Medium risk, with a health risk score in the range [0.3, 0.7), indicates that the driver's health risk is relatively high and requires certain measures for mitigation and intervention; High risk, with a health risk score in the range [0.7, 1], indicates that the driver's health condition is under serious threat and an emergency response mechanism needs to be activated immediately.

[0025] The technical solutions described in the above embodiments of this application have at least the following technical effects or advantages: Through the risk prediction model, early warning of driver health risks can be achieved, potential health problems can be detected in advance, and drivers can have more time to respond. A detailed three-level response execution system is set up based on fatigue index, abnormal posture and health risk score. Each level has clear triggering conditions and execution actions, which comprehensively protects the driver's driving safety and reduces traffic accidents caused by physical discomfort or fatigue.

[0026] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A driver health status-based graded early warning system, characterized in that, include: The system comprises a data acquisition module, a data processing module, and a tiered early warning module. The data acquisition module collects vehicle data, driver physiological characteristic data, and environmental data. The data processing module extracts features from the collected multi-source data. The tiered early warning module provides tiered early warnings to drivers based on the processed multi-source data.

2. The driver health status-based graded early warning system as described in claim 1, characterized in that, The graded early warning module identifies the driver's response after receiving an early warning by introducing a load outlier index. The identification method is as follows: S101, collect multi-source data through the data acquisition module, the multi-source data including vehicle data, driver physiological characteristic data and environmental data; S102, based on the collected multi-source data, classifies and issues warnings to drivers and calculates the load outlier index, and identifies the driver's response after the warning through the load outlier index.

3. The driver health status-based graded early warning system as described in claim 1, characterized in that, It also includes a posture recognition module and a health prediction model building module. The posture recognition module collects the driver's posture data, and the health prediction model building module is used to build a health prediction model.

4. The driver health status-based graded early warning system as described in claim 3, characterized in that, The hierarchical early warning module uses the following hierarchical early warning method: S201, The posture recognition module collects posture data, calculates the fatigue index based on the posture data, identifies the posture state, and classifies the response level according to the posture state. S202, construct a health prediction model based on the obtained attitude data and multi-source data, and predict the driver's future health status based on the health prediction model.

5. The driver health status-based graded early warning system as described in claim 2, characterized in that, The formula for calculating the load outlier index is: ,in, This represents the outlier index, where n represents the number of physiological parameters used in the calculation. This represents the weight of each physiological parameter. This represents the value of the i-th physiological parameter measured at the current moment. Let represent the historical mean of the i-th physiological parameter, and ∑ represent the covariance matrix, which describes the correlation between the various physiological parameters. This represents the inverse matrix of the covariance.

6. The driver health status-based graded early warning system as described in claim 4, characterized in that, The formula for calculating the fatigue index is: ,in, The fatigue index. As an indicator of the uniformity of pressure distribution in the seat cushion, F represents the change in trunk micro-motion frequency, and F represents the heart rate variability index. , and These are the weighting coefficients for the cushion pressure distribution uniformity index, the trunk micro-motion frequency change index, and the heart rate fluctuation index, respectively. + .

7. The driver health status-based graded early warning system as described in claim 1, characterized in that, The data processing module integrates a clock chip to add a timestamp to each collected data and to detect abnormal data.

Images