A driver monitoring system optimization method and system based on expected functional safety

Abstract

The application relates to a driver monitoring system optimization method and system based on expected function safety. The method comprises the following steps: selecting a driver monitoring system to be tested; constructing a simulation driving scene; a driver performs visual subtasks and auditory subtasks in the simulation driving scene and is monitored by the driver monitoring system; based on the expected function safety theory, a test index of the driver monitoring system is constructed, the test index comprising exposure, severity and controllability; based on the Bayesian theory, the exposure index is quantified; through analysis of a hazard event of the driver monitoring system, the severity index is quantified; based on the driver cognition theory, the controllability index is quantified; the safety level of the driver monitoring system is given in combination with the exposure, the severity and the controllability, and the system is optimized. Compared with the prior art, the application can cope with risks caused by potential hazard behaviors in non-failure conditions, and provides a reference for function development and safety design of the driver monitoring system.

Description

Technical Field

[0001] This invention relates to the field of driver monitoring system testing, and in particular to a method and system for optimizing a driver monitoring system based on expected functional safety. Background Technology

[0002] DMS (Driver Monitor System) is an information technology system that monitors a driver's fatigue and dangerous driving behavior around the clock. When the system detects driver fatigue, yawning, squinting, or other erroneous driving behaviors, it analyzes these behaviors promptly and provides voice and light alerts to warn the driver and help correct these errors.

[0003] Currently, mainstream driver monitoring systems primarily rely on camera-based visual detection, while also incorporating techniques for indirect estimation of vehicle driving status. These systems are continuously exploring the application of advanced bio-mechanical sensors in the driver field. However, due to the complexity of the automotive traffic environment, the detection and recognition accuracy of current driver monitoring systems is insufficient to meet the needs of autonomous driving development. Integrating driver monitoring systems into the electronic and electrical architecture of intelligent vehicles may lead to safety hazards due to limitations in perception and algorithmic defects.

[0004] Ensuring the safety of driver monitoring systems is of paramount importance. Currently, the industry uses the ISO26262 standard to classify and analyze automotive safety issues caused by electronic and electrical system failures and to define their hazard levels. This can effectively avoid or control random hardware failures or systemic failures that affect safety objectives.

[0005] However, for electronic and electrical systems such as driver monitoring systems, which are significantly affected by the dynamic internal and external environment of the vehicle, potential risks and hazards not covered by the ISO 26262 standard may arise. These include insufficient system algorithm performance, uncertainty of driver error, sudden scene changes, or sensor instability—all related to the safety of the intended functionality (SOTIF). SOTIF research complements the ISO 26262 functional safety standard, further addressing unreasonable risks arising from potential hazardous behaviors under non-failure conditions.

[0006] In summary, existing technologies lack research on the expected functional safety of DMS products, and testing of DMS usually requires real-vehicle testing, which has problems such as high safety risks and long time consumption. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the existing technology by providing an optimization method and system for driver monitoring systems based on expected functional safety. This invention quantifies the expected functional safety of driver monitoring systems (DMS) based on driver cognitive theory and Bayesian probability models, thereby providing a reference for the functional development and safety design of DMS products.

[0008] The objective of this invention can be achieved through the following technical solutions:

[0009] An optimization method for a driver monitoring system based on expected functional safety includes the following steps:

[0010] Select the driver monitoring system to be tested;

[0011] Construct simulated driving scenarios;

[0012] The driver performs visual and auditory sub-tasks in the simulated driving scenario, and the driver monitoring system monitors the data to obtain monitoring data.

[0013] Based on the expected functional safety theory, test indicators are constructed for testing the driver monitoring system, and the monitoring data is applied to the calculation of the test indicators. The test indicators include exposure, severity, and controllability.

[0014] The exposure index is quantified based on Bayesian theory;

[0015] The severity index is quantified by analyzing hazard events of the driver monitoring system;

[0016] Based on driver cognitive theory, the controllability index is quantified.

[0017] Based on the exposure, severity, and controllability, a test model for the driver monitoring system is constructed.

[0018] Output the test results of the test model on the driver monitoring system, i.e., the safety level of the driver monitoring system;

[0019] Based on the test results, the driver monitoring system was optimized.

[0020] Furthermore, the quantification of the exposure index based on Bayesian theory includes the following steps:

[0021] For drivers performing visual and auditory sub-tasks in simulated driving scenarios, calculate the number of hazardous events n that occur within the simulation work cycle T due to insufficient expected functional safety of the driver monitoring system;

[0022] Based on Bayesian theory, the probability of m hazardous events occurring within time r is calculated, thus obtaining the exposure probability function;

[0023] The expression for the Bayesian theory is as follows:

[0024]

[0025] In the formula P(B i () is event B i The prior probability of occurrence, P(B) i A) is event B given that event A has occurred. i The posterior probability of occurrence.

[0026] Furthermore, the severity index is quantified by analyzing hazard events in the driver monitoring system, and its expression is as follows:

[0027]

[0028] Where S represents the severity of a hazard event caused by the driver monitoring system; in a safe environment, a DMS malfunction indicates that the driver monitoring system misses or misdetects events under safe driving conditions, but this will not lead to a serious hazard event; in a dangerous environment, a DMS malfunction indicates that a serious hazard event will occur due to a DMS miss or misdetection under dangerous driving conditions.

[0029] Furthermore, based on driver cognitive theory, controllability is measured by the time and accuracy required for the driver to return from the current sub-task to the driving task. The controllability index is quantified through the following steps:

[0030] The information that the driver obtains from the surrounding environment or the actions he performs are statistically analyzed to form an extracted information block;

[0031] Calculate the accuracy P of extracting information block i i :

[0032]

[0033] Where C represents the noise of extracted information block i, typically set to 0.8, and A i To extract the activation value of information block i, S p To extract the production strength of information block i;

[0034] Calculate the reaction time T1 required to extract information block i:

[0035]

[0036] T1 is used to evaluate the driver's reaction time when performing actions. In the formula, B is a delay factor, which varies depending on the model and is usually taken as 0.35.

[0037] The formula for calculating driver controllability is:

[0038]

[0039] Where k is the correction coefficient.

[0040] Furthermore, combining the exposure, severity, and controllability, a test model for the driver monitoring system is constructed, and the expression of the test model is:

[0041]

[0042] Where n is the number of hazardous events caused by insufficient expected functional safety of the driver monitoring system within the work cycle T, and S is the number of such events. i Let represent the severity of hazard event i, and let a, b, and c be the weights corresponding to exposure, severity, and controllability, respectively, and a+b+c=1.

[0043] An optimized driver monitoring system based on expected functional safety includes a driving scenario simulation module, a driving simulator, a driver monitoring module, a display module, a test and evaluation module, and a host computer.

[0044] The display module is connected to the driving scenario simulation module and the driver monitoring module respectively, and the host computer is connected to the driver monitoring module, the driving scenario simulation module and the test and evaluation module respectively.

[0045] The driving scenario simulation module is used to construct a simulated driving scenario; the driver performs visual and auditory sub-tasks in the simulated driving scenario through a driving simulator.

[0046] The driver monitoring module includes a driver monitoring system, which is used to monitor the driver's performance of visual and auditory sub-tasks.

[0047] Furthermore, the driver monitoring system includes a camera, a heart rate sensor, and an embedded platform;

[0048] The camera is used to collect image data of the driver, and the heart rate sensor is used to collect heart rate data of the driver. The embedded platform acquires the image data and heart rate data.

[0049] Based on the image data and heart rate data, the embedded platform calculates driver status-related information, including the driver's visual attention orientation and heart rate. The embedded platform also transmits the driver status-related information to the host computer.

[0050] Furthermore, the host computer includes a processing and computing unit, which performs quantitative calculations of controllability and severity based on the driving state-related information.

[0051] Furthermore, the test evaluation module includes a test model for the driver monitoring system, the expression of which is:

[0052]

[0053] Where n represents the number of hazardous events caused by insufficient expected functional safety of the driver monitoring system within the work cycle T, and P i P represents the precision of extracting information block i. C S represents the degree of controllability. i The severity of hazard event i is represented by a, b, and c, which are the weights corresponding to exposure, severity, and controllability, respectively.

[0054] The larger the calculated eSOTIF value, the higher the risk level of the driver monitoring system, i.e., the lower the safety level. Finally, the test results of the test model on the driver monitoring system are output.

[0055] Furthermore, the accuracy P of the information block i i The expression is:

[0056]

[0057] Where C represents the noise of extracted information block i, typically set to 0.8, and A i To extract the activation value of information block i, S p To extract the production strength of information block i;

[0058] The formula for calculating driver controllability is:

[0059]

[0060] Where k is the correction coefficient.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] 1. Unlike current mainstream vehicle safety issues, this invention tests the driver monitoring system from the perspective of expected functional safety, which can address unreasonable risks caused by potential harmful behaviors in non-failure situations.

[0063] 2. This invention uses a Bayesian probability model to quantify exposure, which can be further extended to predict the working performance of the driver monitoring system throughout its entire life cycle, solving problems such as long simulation time.

[0064] 3. This invention considers the driver's controllability over hazardous events from the perspective of the driver's cognitive model. Combined with cognitive psychology theories (perception and cognition, memory and learning, reasoning, judgment and problem-solving), it can better simulate human performance and make the testing mechanism for driver monitoring systems more reasonable. Attached Figure Description

[0065] Figure 1 This is a schematic diagram of the system structure of the present invention;

[0066] Figure 2 This is a flowchart of the exposure calculation process of the present invention;

[0067] Figure 3 This is a flowchart of the controllability calculation process of the present invention. Detailed Implementation

[0068] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0069] like Figure 1 As shown, this is an optimized driver monitoring system based on expected functional safety, including a driving scenario simulation module, a driving simulator, a driver monitoring module, a display module, a test and evaluation module, and a host computer;

[0070] The driving scenario simulation module includes a simulated vehicle and a real-time simulation processor;

[0071] The host computer builds a virtual driving environment for the driver through the driving scenario simulation module. The driver controls the simulated vehicle through the driving simulator to perform visual and auditory sub-tasks. The real-time simulation processor sends the simulation time and traffic flow conditions to the host computer and displays information such as the driving speed and accelerator / brake pedal opening of the simulated vehicle on the display module.

[0072] The display module is connected to the driving scenario simulation module and the driver monitoring module respectively, and the host computer is connected to the driver monitoring module, the driving scenario simulation module and the test and evaluation module respectively.

[0073] The driving scenario simulation module is used to construct simulated driving scenarios; the driver performs visual and auditory sub-tasks in the simulated driving scenario through the driving simulator;

[0074] The driver monitoring module includes a driver monitoring system, which is used to monitor the driver's performance of visual and auditory sub-tasks.

[0075] The driver monitoring system includes cameras, heart rate sensors, and an embedded platform;

[0076] The camera is used to collect image data of the driver, and the heart rate sensor is used to collect heart rate data of the driver. The embedded platform acquires the image data and heart rate data.

[0077] The embedded platform calculates driver status information based on image data and heart rate data. The driver status information includes the driver's visual attention orientation and heart rate. At the same time, the embedded platform transmits the driver status information to the host computer.

[0078] The host computer includes a processing and computing unit, which performs quantitative calculations on controllability and severity based on driving status-related information.

[0079] The test evaluation module includes a test model for the driver monitoring system. The expression for the test model is as follows:

[0080]

[0081] Where n represents the number of hazardous events caused by insufficient expected functional safety of the driver monitoring system within the work cycle T, and P i P represents the precision of extracting information block i. C S represents the degree of controllability. i The severity of hazard event i is represented by a, b, and c, which are the weights corresponding to exposure, severity, and controllability, respectively.

[0082] The larger the calculated eSOTIF value, the higher the level of danger of the driver monitoring system. Finally, the test results of the test model on the driver monitoring system are output.

[0083] Based on this system, a method for optimizing a driver monitoring system based on expected functional safety can be implemented, which includes the following steps:

[0084] S1. Select the driver monitoring system to be tested;

[0085] S2. Construct a simulated driving scenario;

[0086] S3. The driver performs visual and auditory sub-tasks in a simulated driving scenario, and the driver monitoring system monitors the data to obtain monitoring data.

[0087] S4. Based on the expected functional safety theory, test indicators are constructed for testing driver monitoring systems. This embodiment proposes three test indicators: exposure, severity, and controllability, taking into account the expected functional safety problems that may occur in DMS (blind spotting; insufficient sensors; driver misuse, etc.).

[0088] S5. Based on Bayesian theory, the exposure index is quantified;

[0089] S6. Quantify the severity index by analyzing the hazard events of the driver monitoring system;

[0090] S7. Based on driver cognitive theory, controllability indicators are quantified;

[0091] S8. Combining exposure, severity, and controllability, construct a test model for the driver monitoring system;

[0092] S9. Output the test results of the test model on the driver monitoring system, that is, the safety level of the driver monitoring system;

[0093] S10. Based on the test results, optimize the driver monitoring system.

[0094] like Figure 2 The diagram shows the calculation process for exposure. In step S5, based on Bayesian theory, the exposure index is quantified, including the following steps:

[0095] For drivers performing visual and auditory sub-tasks in simulated driving scenarios, experiments were conducted within a specified scenario to calculate the number of hazardous events n that occurred within the simulation work cycle T due to insufficient expected functional safety of the driver monitoring system.

[0096] Specifically, within a simulation cycle time T, the total number of times the driver performs visual and auditory sub-tasks is N. When the driver's eyes are turned away from the center of the road or his hands are placed outside the steering wheel, it is judged as a hazard event. The number of hazard events finally detected by the driver monitoring system is n, and both are used as inputs to the Bayesian model.

[0097] Based on Bayesian theory, the probability of m hazardous events occurring within time r is calculated, thus obtaining the exposure probability function;

[0098] The expression for Bayesian theory is as follows:

[0099]

[0100] In the formula P(B i () is event B i The prior probability of occurrence, P(B) i A) is event B given that event A has occurred. i The posterior probability of occurrence.

[0101] In step S6, the severity index is quantified by analyzing the hazard events of the driver monitoring system. The consequences of severity can be divided into two scenarios: In a safe driving environment, if the driver monitoring system misses or falsely detects events, it will not lead to serious hazard events. However, unnecessary warnings caused by false detections will affect the driver's driving experience to some extent. In a dangerous driving environment, if a distracted driver fails to receive timely warnings due to missed or false detections by the DMS, it will cause serious consequences, even leading to traffic accidents. Taking all factors into consideration, the severity of hazard events of the driver monitoring system is defined as follows:

[0102]

[0103] In step S7, the expected functional safety controllability involves the driver's subjectivity. The process from the driver acquiring information about a hazardous event from the current state to avoiding its occurrence—that is, the driver's perception-cognition-decision-control—conforms to the ACT-R cognitive process, so controllability can be measured using the driver's cognitive process. The ACT-R cognitive framework distinguishes between declarative and procedural knowledge, based on a rational analysis of the cognitive process. ACT-R cognitive theory includes declarative knowledge that reflects facts and procedural knowledge about how to perform cognitive activities, defining two types of knowledge: blocks and production rules. Blocks represent declarative knowledge, and rules represent procedural knowledge about how to do things, which are executed to produce actions. Based on driver cognitive theory, controllability is measured by the time and accuracy required for the driver to return from the current sub-task to the driving task. The controllability index is quantified by the following steps:

[0104] The information that the driver obtains from the surrounding environment or the actions he performs are statistically analyzed to form an extracted information block;

[0105] Calculate the accuracy P of extracting information block i i :

[0106]

[0107] Where C represents the noise of extracted information block i, typically set to 0.8, and A i To extract the activation value of information block i, S p To extract the production strength of information block i;

[0108] Calculate the reaction time T1 required to extract information block i:

[0109]

[0110] T1 is used to evaluate the driver's reaction time when performing actions. In the formula, B is a delay factor, which varies depending on the model and is usually taken as 0.35.

[0111] The formula for calculating driver controllability is:

[0112]

[0113] Where k is the correction coefficient.

[0114] By combining exposure, severity, and controllability, a test model for the driver monitoring system is constructed. The expression of the test model is as follows:

[0115]

[0116] Where n is the number of hazardous events caused by insufficient expected functional safety of the driver monitoring system within the work cycle T, and S is the number of such events. i Let represent the severity of hazard event i, and let a, b, and c be the weights corresponding to exposure, severity, and controllability, respectively, and a+b+c=1.

[0117] In a preferred embodiment, the driver monitoring system in this example uses an IR camera, a heart rate sensor with PPG (photoplethysmography) capability, and an NVIDIA Jetson AGX Xavier embedded platform. The IR camera and PPG sensor collect image data and heart rate data, which are then processed by Xavier. The system outputs data such as the driver's eye focus and heart rate to the host computer. The time interval t between the driver's current task and their return to the center of the road is used as the input for calculating controllability. Figure 3 As shown.

[0118] In specific implementation, the testing process of this invention is as follows:

[0119] 1) Set up a virtual simulation environment and configure parameters such as simulation time;

[0120] 2) Design different driver visual and auditory sub-tasks, and control the vehicle's steering wheel angle, accelerator pedal, and brake pedal through the interface of the driving simulator connected to the simulation software.

[0121] 3) The exposure level is calculated by the number of hazards that occur during the simulation period;

[0122] 4) The DMS device collects driver data and transmits it as a status input to the processing and computing unit in the host computer for controllability and severity calculation.

[0123] 5) Based on the calculation results of exposure, controllability, and severity, output the test results for the driver monitoring system;

[0124] 6) After completing the current driver test, switch to different drivers and conduct tests in the same scenario and with the same DMS equipment to obtain multiple sets of test data, so as to achieve a more comprehensive evaluation of the performance of the driver monitoring system.

[0125] 7) Based on the test results, make corresponding optimizations to the driver monitoring system. In specific implementation, the optimization plan can be formulated by professionals in this field according to the requirements.

[0126] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for optimizing a driver monitoring system based on intended functional safety, characterized in that, Includes the following steps: Select the driver monitoring system to be tested; Construct simulated driving scenarios; The driver performs visual and auditory sub-tasks in the simulated driving scenario, and the driver monitoring system monitors the data to obtain monitoring data. Based on the expected functional safety theory, test indicators are constructed for testing the driver monitoring system, and the monitoring data is applied to the calculation of the test indicators. The test indicators include exposure, severity, and controllability. The exposure index is quantified based on Bayesian theory; The severity index is quantified by analyzing hazard events of the driver monitoring system; The severity index is quantified by analyzing hazardous events in the driver monitoring system, and its expression is as follows: Where S represents the severity of a hazardous event caused by the driver monitoring system; in a safe environment, the DMS malfunction indicates that under safe driving conditions, the driver monitoring system may miss or misdetect the system, but this will not lead to a serious hazardous event; in a dangerous environment, the DMS malfunction indicates that under dangerous driving conditions, a serious hazardous event may occur due to a missed or misdetected system by the DMS. Based on driver cognitive theory, controllability is measured by the time and accuracy required for a driver to return from the current sub-task to the driving task. The controllability index is quantified through the following steps: The information that the driver obtains from the surrounding environment or the actions he performs are statistically analyzed to form an extracted information block; Calculate the accuracy of extracting information block i : Where C represents the noise of extracted information block i. To extract the activation value of information block i, To extract the production strength of information block i; Calculate the reaction time required to extract information block i. : Used to evaluate the driver's reaction time when performing actions, where B is the delay factor; The formula for calculating driver controllability is: Where k is the correction coefficient; Based on the exposure, severity, and controllability, a test model for the driver monitoring system is constructed, and the expression of the test model is: Where n represents the number of hazardous events caused by insufficient expected functional safety of the driver monitoring system within the work cycle T. Indicates the severity of harmful event i. a, b, c These are the weights corresponding to exposure, severity, and controllability, respectively. ; Based on driver cognitive theory, the controllability index is quantified. Based on the exposure, severity, and controllability, a test model for the driver monitoring system is constructed. Output the test results of the test model on the driver monitoring system; Based on the test results, the driver monitoring system was optimized.

2. The driver monitoring system optimization method based on expected functional safety according to claim 1, characterized in that, The quantification of the exposure index based on Bayesian theory includes the following steps: For drivers performing visual and auditory sub-tasks in simulated driving scenarios, calculate the number of hazardous events n that occur within the simulation work cycle T due to insufficient expected functional safety of the driver monitoring system; Based on Bayesian theory, the probability of m hazardous events occurring within time r is calculated, thus obtaining the exposure probability function; The expression for the Bayesian theory is as follows: In the formula For the event The prior probability of occurrence, In the event Events occurring under certain conditions The posterior probability of occurrence.

3. A driver monitoring system optimization system based on expected functional safety, used in the driver monitoring system optimization method based on expected functional safety as described in any one of claims 1 to 2, characterized in that, It includes a driving scenario simulation module, a driving simulator, a driver monitoring module, a display module, a testing and evaluation module, and a host computer; The display module is connected to the driving scenario simulation module and the driver monitoring module respectively, and the host computer is connected to the driver monitoring module, the driving scenario simulation module and the test and evaluation module respectively. The driving scenario simulation module is used to construct a simulated driving scenario; the driver performs visual and auditory sub-tasks in the simulated driving scenario through a driving simulator. The driver monitoring module includes a driver monitoring system, which is used to monitor the driver's performance of visual and auditory sub-tasks.

4. The driver monitoring system optimization system based on expected functional safety according to claim 3, characterized in that, The driver monitoring system includes a camera, a heart rate sensor, and an embedded platform; The camera is used to collect image data of the driver, and the heart rate sensor is used to collect heart rate data of the driver. The embedded platform acquires the image data and heart rate data. Based on the image data and heart rate data, the embedded platform calculates driver status-related information, including the driver's visual attention orientation and heart rate. The embedded platform also transmits the driver status-related information to the host computer.

5. The driver monitoring system optimization system based on expected functional safety according to claim 4, characterized in that, The host computer includes a processing and calculation unit, which performs quantitative calculations on the controllability and severity based on the driving state-related information.

Images