Evaluation method, system and equipment of driver attention monitoring system

Abstract

The invention discloses an evaluation method, system and device for a driver attention monitoring system, and relates to the technical field of intelligent network connection automobiles. The method comprises the steps that when a first specified driver drives a to-be-tested vehicle, the fatigue magnitude of the first specified driver and fatigue warning information sent by a fatigue monitoring system are acquired, and a first effectiveness evaluation result of the fatigue monitoring system is determined according to the fatigue magnitude and the fatigue warning information; when a second specified driver drives the to-be-tested vehicle, acquiring a driving video associated with the second specified driver, a test vehicle speed interval corresponding to the driving video and a plurality of distraction warning information sent by the distraction monitoring system to perform system effectiveness evaluation, and determining a second effectiveness evaluation result of the distraction monitoring system; and according to the first effectiveness evaluation result and the second effectiveness evaluation result, determining a target evaluation result of the driver attention monitoring system of the to-be-tested vehicle. According to the method, the requirement of manual operation can be reduced, and the evaluation efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent connected vehicles, and in particular, to an evaluation method, system and device for a driver attention monitoring system. Background Art

[0002] With the increasing annual export volume of automobiles in China, more and more automobiles are going abroad and being exported to all over the world. At present, the internal system components of exported automobiles need to meet the requirements of the exporting country for vehicle access. At present, the effectiveness verification of the driver attention detection system depends on manual operation, and professional personnel are required to collect and analyze data. This not only takes time and effort, but is also easily affected by subjective factors, resulting in inconsistencies and deviations in evaluation results. Therefore, how to provide an accurate evaluation of the effectiveness of the driver attention monitoring system is an urgent problem to be solved at present. Summary of the Invention

[0003] In view of this, the present application provides an evaluation method, system and device for a driver attention monitoring system, mainly aiming to solve the technical problem that the effectiveness verification of the driver attention detection system depends on manual operation, and professional personnel are required to collect and analyze data. This not only takes time and effort, but is also easily affected by subjective factors, resulting in inconsistencies and deviations in evaluation results.

[0004] According to a first aspect of the present invention, there is provided an evaluation method for a driver attention monitoring system, the method comprising:

[0005] When a first designated driver drives a vehicle to be tested, obtain the fatigue level of the first designated driver and the fatigue warning information issued by the fatigue monitoring system, and determine a first effectiveness evaluation result of the fatigue monitoring system according to the fatigue level and the fatigue warning information, where the fatigue level is uploaded by the first designated driver based on the Karolinska Sleepiness Scale (KSS) scoring system;

[0006] Establish a vehicle coordinate system based on a three-dimensional H-point device and a coordinate measuring machine, and determine the coordinate interval of the distraction area according to the vehicle coordinate system, and determine the coordinates of a plurality of distraction points according to the coordinate interval of the distraction area;

[0007] When a second designated driver drives the vehicle to be tested, obtain the driving video associated with the second designated driver, the corresponding test vehicle speed interval of the driving video, and a plurality of distraction warning information issued by the distraction monitoring system for system effectiveness evaluation, and determine a second effectiveness evaluation result of the distraction monitoring system, where the driving video includes the distraction video of the second designated driver's gaze falling on each of the distraction points;

[0008] Based on the first validity evaluation result and the second validity evaluation result, determine the target evaluation result of the driver attention monitoring system of the vehicle to be tested.

[0009] Optionally, the obtaining the fatigue level of the first designated driver and the fatigue warning information sent by the fatigue monitoring system, and determining the first validity evaluation result of the fatigue monitoring system according to the fatigue level and the fatigue warning information includes:

[0010] For the current test round, determine the first designated driver associated with the current test round;

[0011] When the fatigue level self-evaluated by the first designated driver reaches the first preset level, at intervals of a preset duration, based on the Karolinska Sleepiness Scale (KSS) scoring system, obtain the fatigue level of the first designated driver for detection;

[0012] When it is detected that the fatigue level reaches the second preset fatigue level, if the fatigue warning information sent by the fatigue monitoring system is received, generate a fatigue test result for indicating that the test type of the current test round is a true valid type;

[0013] Replace the first designated driver to drive the vehicle to be tested again for a fatigue test until the number of fatigue test results of the specified type reaches a preset quantity threshold, and complete the test of the fatigue monitoring system, where the specified type is a true valid type or a false negative type;

[0014] According to the fatigue test results of each round of testing, determine the index values corresponding to each preset evaluation item, compare each index value with the preset index threshold associated with the corresponding preset evaluation item, and determine the first validity evaluation result according to the comparison result. The preset evaluation index items include, but are not limited to, the average sensitivity and the lower limit of the 90% confidence interval of the sensitivity result. The first validity evaluation result includes an evaluation result for indicating that the system is effective or an evaluation result for indicating that the system is ineffective.

[0015] Optionally, after it is detected that the fatigue score reaches the preset fatigue level, the method further includes:

[0016] If the fatigue warning information sent by the fatigue monitoring system is not received, set the test type of the current test round to a false invalid type and conduct an additional test;

[0017] During the test time interval of the additional test, if the fatigue level of the first designated driver does not reach the second preset fatigue level, mark the test type of the current test round as abnormal data and generate a fatigue test result for indicating that the test type of the current test round is a true invalid type.

[0018] If the fatigue level of the first designated driver reaches the preset fatigue level, a fatigue test result is generated to indicate that the test type of the current test round is a false negative type.

[0019] Optionally, establishing a vehicle coordinate system based on the three-dimensional H-point device and the coordinate measuring machine, determining the coordinate interval of the distraction area according to the vehicle coordinate system, and determining the coordinates of multiple distraction points according to the coordinate interval of the distraction area, including:

[0020] Dividing the visual area of the vehicle to be tested according to the preset division standard and the vehicle coordinate system to obtain the area coordinates of the distraction area, the distraction area includes a first distraction area, a second distraction area and a third distraction area, the first distraction area is any area outside two vertical planes in the vehicle to be tested, the two vertical planes intersect at the eye reference point, the second distraction area is 10° around the windshield and window areas, the third distraction area is any area below the plane 30° downward from the eye reference point, and the eye reference point is located directly above the H-point and the distance from the H-point satisfies a preset distance value;

[0021] Determining the coordinates of multiple distraction points according to the preset division standard and the coordinate interval of the distraction area, the distraction points include the left knee point of the driver, the right knee point of the driver, the crotch point of the driver, the foot point of the co-pilot, the surface point of the co-pilot seat, the glove box point of the co-pilot, the left air outlet point of the driver's side air conditioner, the right air outlet point of the driver's side air conditioner, the instrument panel point, the steering wheel point, the shifter point, the air conditioner control area point, the information entertainment display screen and the center console point.

[0022] Optionally, obtaining the driving video associated with the second designated driver, the corresponding test vehicle speed interval of the driving video, and multiple distraction warning messages sent by the distraction monitoring system for system effectiveness evaluation, and determining the second effectiveness evaluation result of the distraction monitoring system, including:

[0023] For the current test round, determining the second designated driver associated with the current test round, obtaining the driving video associated with the second designated driver, the corresponding test vehicle speed interval of the driving video, and multiple distraction warning messages sent by the distraction monitoring system;

[0024] Identifying each segment of distraction video recorded in the driving video, determining the first time point when the gaze of the second designated driver falls on each distraction point, and determining the distraction point indicated by each distraction warning message and the second time point when each distraction warning message is sent;

[0025] Calculate the time difference between the second time point associated with each of the distraction points and the corresponding first time point, compare the time difference with the time interval threshold associated with the test vehicle speed range, and determine the point test result corresponding to each distraction point;

[0026] If any of the point test results indicates a failed test, generate a distraction test result indicating a failed test; otherwise, generate a distraction test result indicating a passed test;

[0027] Replace the second designated driver to drive the vehicle to be tested again for the distraction test until the number of distraction test results reaches the preset quantity threshold, and complete the test of the distraction monitoring system;

[0028] When all the distraction test results indicate a passed test, generate a second effectiveness evaluation result indicating that the system is effective;

[0029] When any of the distraction test results indicates a failed test, generate a second effectiveness evaluation result indicating that the system is ineffective.

[0030] Optionally, the comparing the time difference with the time interval threshold associated with the test vehicle speed range in the test data to determine the point test result corresponding to each distraction point includes:

[0031] If the comparison result indicates that the time difference is less than or equal to the corresponding time interval threshold, generate a point test result indicating that the distraction point test is passed;

[0032] If the comparison result indicates that the time difference is greater than the corresponding time interval threshold, generate a point test result indicating that the distraction point test is failed.

[0033] Optionally, the determining the target evaluation result of the driver attention monitoring system according to the first effectiveness evaluation result and the second effectiveness evaluation result includes:

[0034] If both the first effectiveness evaluation result and the second effectiveness evaluation result indicate that the system is effective, generate a target evaluation result indicating that the driver attention monitoring system is effective;

[0035] If the first effectiveness evaluation result and / or the second effectiveness evaluation result indicates that the system is ineffective, generate a target evaluation result indicating that the driver attention monitoring system is ineffective.

[0036] Optionally, the method further includes:

[0037] For the test of the fatigue monitoring system, the fatigue warning signal, vehicle speed, the fatigue level self-evaluated by the driver clicking the KSS scoring record system, the alarm interface of the fatigue monitoring system, and the video result of the behavior monitoring of the first designated driver are collected and visualized.

[0038] For the test of the distraction monitoring system, the distraction actions of the second designated driver, the real-time vehicle speed, the distraction warning signal of the distraction monitoring system, and the alarm interface of the distraction monitoring system are collected and visualized.

[0039] According to the second aspect of the present invention, there is provided an evaluation system for a driver attention monitoring system, the system comprising:

[0040] A first evaluation module, configured to, when there is a first designated driver driving the vehicle to be tested, obtain the fatigue level of the first designated driver and the fatigue warning information issued by the fatigue monitoring system, and determine a first effectiveness evaluation result of the fatigue monitoring system according to the fatigue level and the fatigue warning information, where the fatigue level is uploaded by the first designated driver according to the Karolinska Sleepiness Scale KSS scoring system;

[0041] A setting module, configured to establish a vehicle coordinate system based on a three-dimensional H-point device and a three-coordinate measuring instrument, determine the coordinate interval of the distraction area according to the vehicle coordinate system, and determine the coordinates of multiple distraction points according to the coordinate interval of the distraction area;

[0042] A second evaluation module, configured to, when there is a second designated driver driving the vehicle to be tested, obtain the driving video associated with the second designated driver, the corresponding test vehicle speed interval of the driving video, and multiple distraction warning information issued by the distraction monitoring system for system effectiveness evaluation, and determine a second effectiveness evaluation result of the distraction monitoring system, where the driving video includes the distraction video of the second designated driver's gaze falling on each of the distraction points;

[0043] A determination module, configured to determine a target evaluation result of the driver attention monitoring system of the vehicle to be tested according to the first effectiveness evaluation result and the second effectiveness evaluation result.

[0044] According to the third aspect of the present invention, there is provided a computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, where when the processor executes the program, the above-mentioned evaluation method of the driver attention monitoring system is implemented.

[0045] An evaluation method, system and device for a driver attention monitoring system provided by the present invention. When a first designated driver drives a vehicle to be tested, the fatigue level of the first designated driver and the fatigue warning information sent by the fatigue monitoring system are obtained. Based on the fatigue level and the fatigue warning information, the first effectiveness evaluation result of the fatigue monitoring system is determined. A vehicle coordinate system is established based on a three-dimensional H-point device and a coordinate measuring machine, and the coordinate interval of the distraction area is determined according to the vehicle coordinate system, and the coordinates of multiple distraction points are determined according to the coordinate interval of the distraction area. When a second designated driver drives the vehicle to be tested, the driving video associated with the second designated driver, the corresponding test vehicle speed interval of the driving video, and multiple distraction warning information sent by the distraction monitoring system are obtained for system effectiveness evaluation, and the second effectiveness evaluation result of the distraction monitoring system is determined. According to the first effectiveness evaluation result and the second effectiveness evaluation result, the target evaluation result of the driver attention monitoring system of the vehicle to be tested is determined. In the embodiments of the present application, the driver attention monitoring system includes a fatigue monitoring system and a distraction monitoring system. For the effectiveness evaluation of the fatigue detection system, the evaluation system analyzes whether the fatigue monitoring system can issue fatigue warning information matching the fatigue level by using the self-assessed fatigue level of the driver, and then judges the effectiveness of the fatigue monitoring system. It reduces the need for manual observation and recording of driver behavior and solves the problem of dependence on manual operation. For the effectiveness judgment of the distraction monitoring system, the present application establishes a vehicle coordinate system based on a three-dimensional H-point device and a coordinate measuring machine, divides the distraction area according to the test requirements and sets distraction points, which improves the accuracy of monitoring, reduces the evaluation deviation caused by inaccurate area division, and solves the problem of subjective factor influence. Through the above methods, the need for manual operation can be reduced, and the efficiency and accuracy of evaluation can be improved. At the same time, it reduces the inconsistency and deviation caused by subjective factors, and improves the reliability and effectiveness of the evaluation results.

[0046] The above description is only an overview of the technical solution of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the specific embodiments of the present application are specifically exemplified below. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The drawings described herein are used to provide a further understanding of the present invention, and constitute a part of the present application. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0048] Figure 1 It shows a schematic flow chart of an evaluation method for a driver attention monitoring system provided by an embodiment of the present invention;

[0049] Figure 2Shows a schematic diagram of the installation of the auxiliary brake in an evaluation method of a driver attention monitoring system provided by an embodiment of the present invention;

[0050] Figure 3 Shows a schematic diagram of the KSS scoring system in an evaluation method of a driver attention monitoring system provided by an embodiment of the present invention;

[0051] Figure 4 Shows a schematic diagram of the visualization interface in an evaluation method of a driver attention monitoring system provided by an embodiment of the present invention;

[0052] Figure 5 Shows a schematic diagram of the three - coordinate measuring instrument in an evaluation method of a driver attention monitoring system provided by an embodiment of the present invention;

[0053] Figure 6 Shows a schematic diagram of the distraction area and distraction points in an evaluation method of a driver attention monitoring system provided by an embodiment of the present invention;

[0054] Figure 7 Shows a schematic diagram of the installation of the camera in an evaluation method of a driver attention monitoring system provided by an embodiment of the present invention;

[0055] Figure 8 Shows a schematic diagram of the visualization interface in an evaluation method of a driver attention monitoring system provided by an embodiment of the present invention;

[0056] Figure 9 Shows a schematic diagram of the second effectiveness evaluation result in an evaluation method of a driver attention monitoring system provided by an embodiment of the present invention;

[0057] Figure 10 Shows a schematic diagram of the structure of an evaluation system of a driver attention monitoring system provided by an embodiment of the present invention;

[0058] Figure 11 Shows a schematic diagram of the device structure of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0059] The present invention will be described in detail below with reference to the drawings and in conjunction with the embodiments. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0060] In the embodiments of the present application, the vehicle to be tested is a vehicle equipped with a driver attention monitoring system, i.e., a DMS (Driver Monitoring System) system. The brand and model of the vehicle to be tested are not specifically limited in the present application. The DMS system is an intelligent vehicle safety technology mainly used to monitor the behavior and physiological state of the driver to improve driving safety. The DMS includes multiple subsystems such as a fatigue monitoring system, a distraction monitoring system, and a dangerous behavior detection system, etc. In the embodiments of the present application, only the effectiveness of the fatigue monitoring system and the distraction monitoring system is evaluated, and then the target evaluation result of the DMS system is determined according to the first effectiveness evaluation result of the fatigue monitoring system and the second effectiveness evaluation result of the distraction monitoring system.

[0061] Among them, the fatigue monitoring system can monitor the fatigue state of the driver, such as behaviors like closing eyes and yawning, and judge whether the driver is fatigued by analyzing indicators such as eye closure time and blink frequency. The distraction monitoring system can detect the line-of-sight deviation and head posture of the driver to determine whether the driver is distracted, for example, whether the driver is looking at the phone, talking to passengers, or doing other things unrelated to driving.

[0062] Currently, the internal system components of exported cars need to meet the requirements of the exporting country regarding vehicle access. Take Europe as an example. Cars exported to Europe, from the whole vehicle to parts and components, need to meet the vehicle access requirements of the European Union. The European Union has clear mandatory requirements (EU) 2021 / 1341 and (EU) 2023 / 2590 for the DMS system in the vehicle ADAS system.

[0063] Regarding driver fatigue, (EU) 2021 / 1341 requires the DMS system to monitor the fatigue level of the driver, and at the same time, the system monitoring result needs to be consistent with the driver's subjective self-evaluation of the fatigue level during driving. Currently, for the subjective evaluation of driver fatigue testing, it all adopts the form of manual question-and-answer records, lacking automated test recording equipment.

[0064] Regarding driver distraction, (EU) 2023 / 2590 requires the division and confirmation of the vehicle distraction areas and points, and at the same time, the confirmation of the distraction time during vehicle driving is recorded. Ensure that the DMS system does not give false alarms or miss alarms. Currently, there is a lack of effective tools for dividing vehicle distraction areas in China, and it is impossible to confirm the vehicle distraction areas and distraction points. At the same time, there is a lack of an effective calculation method for the confirmation of distraction time during vehicle driving.

[0065] In some embodiments, real vehicle experiments with real people participating can be carried out on a closed site to ensure the safety and controllability of the test.

[0066] In some embodiments, real-vehicle experiments with human participation can be carried out on open roads to reflect different situations that may be encountered in actual driving.

[0067] To evaluate the effectiveness of the driver attention monitoring system, an embodiment of the present application provides an evaluation method for the driver attention monitoring system, as Figure 1 shown, the method includes:

[0068] 101. When there is a first designated driver driving the vehicle to be tested, obtain the fatigue level of the first designated driver and the fatigue warning information issued by the fatigue monitoring system, and determine the first effectiveness evaluation result of the fatigue monitoring system according to the fatigue level and the fatigue warning information.

[0069] In the embodiment of the present application, for the effectiveness evaluation of the fatigue monitoring system, the driver fatigue is defined by the Karolinska Sleepiness Scale, and a subjective evaluation system for driver fatigue levels is developed for the performance test of the vehicle DMS system (for the fatigue monitoring system).

[0070] The test is a real-vehicle test with human participation on a closed site or an open road. Two safety officers are arranged for the test. One intervenes in the test vehicle speed through the auxiliary brake pedal, and the other safety officer can intervene in the steering of the vehicle's steering wheel to ensure the test safety. All first designated drivers participating in the test meet the target user demographic characteristics of the test vehicle and hold a valid driver's license for driving the test vehicle. All first designated drivers participating in the test should not have participated in the development of the fatigue driving system. Before testing and verifying the performance of the vehicle DMS (fatigue monitoring) system, the first designated drivers participating in the test should receive training on the KSS (Karolinska Sleepiness Scale). Understand the fatigue level they are in during driving. The sleepiness levels (Karolinska Sleepiness Scale) referred to in the fatigue monitoring system effectiveness test are shown in Table 1 below:

[0071] Table 1. Karolinska Sleepiness Scale

[0072]

[0073] Further, prepare the test equipment. The hardware equipment includes an auxiliary brake pedal, a k-Pad, and two cameras. Under the condition of not affecting normal driving, as Figure 2As shown, install the secondary brake pedal, and the position of the secondary brake pedal should be convenient for the co-driver safety officer to step on. The connection line between the main driver's and co-driver's brake pedals should be stable without shaking, and the connection line should ensure no interference with normal driving. One of the two cameras is used for the whole process recording of the driver's facial expressions, and the other is used for recording the visual alarm interface of the fatigue monitoring system. The k-Pad is used to obtain the fatigue level input by the first designated driver. The KSS scoring system conducts KSS evaluation in the form of color differentiation. The first designated driver understands their own fatigue status and clicks on the color of the KSS scoring system screen to confirm their own fatigue level. During the test, paper labels of KSS levels 1-4 will be pasted on the steering wheel, and KSS levels 5-9 will be displayed in the k-Pad developed by CMVR. The k-Pad is placed in the cab, as Figure 3 shown, for the first designated driver to self-evaluate their KSS level and scoring record after reaching level 6. Once the KSS level of the first designated driver participating in the test reaches 5 or above, they will be required to enter a self-evaluation through the k-Pad. Once their KSS level reaches 6, a 5-minute measurement interval will be adopted, and every 5 minutes the k-Pad will light up once to remind the first designated driver participating in the test to upload the current self-evaluation level based on accessing the KSS scoring system through the k-Pad.

[0074] Furthermore, in order to visualize the data during the test process, facilitate on-site comparison by the test personnel, judge the quality of the driver attention monitoring system, and at the same time eliminate the need for the test personnel to manually record, making the data of the entire test process highly integrated and visual and easy to export, the embodiment of the present application also provides a visualization interface, as Figure 4 shown. For the test of the fatigue monitoring system, a new engineering interface is developed in the driver fatigue level evaluation system in the evaluation system. Through this interface, the collected fatigue warning signals, vehicle speed, and the fatigue level self-evaluated by the driver clicking on the KSS scoring record system, the alarm interface of the fatigue monitoring system, and the video results of the behavior monitoring of the first designated driver are visualized, facilitating the test engineer to evaluate the performance of the fatigue monitoring system.

[0075] The minimum number of drivers participating in the test is 10. For the current test round, first, it is necessary to determine the first designated driver associated with the current test round. The first designated driver takes a seat, and the current KSS level is measured and recorded by the tester in the corresponding form. The tester announces the start of the test, and the first designated driver drives the test vehicle to start the test. When the fatigue level self-evaluated by the first designated driver reaches the first preset level (reaching the fatigue level of KSS level 6 or above), report the status to the safety officer, and the safety officer records the time point and KSS level at this time.

[0076] Starting from the first time the first designated driver reaches the first preset level, that is, reaches a fatigue level of KSS level 6 or above, during the subsequent test period, at every preset time interval, such as 5 minutes, the driver accesses the KSS scoring system through the k-Pad to upload their own fatigue level. It should be noted that the fatigue level uploaded at each subjective measurement point is a self-assessment for the previous 5 minutes. The tester observes the driver's scoring situation. Once the tester discovers that the tested system issues an alarm (mild fatigue reminders are not included in the alarm reminders), the time point when the alarm appears is recorded in the table of the previous KSS level assessment.

[0077] When the previous or next score of the driver participating in the test is a KSS level of 7 or higher, any warning from the fatigue monitoring system should be regarded as a truly valid event.

[0078] Furthermore, when it is detected that the fatigue level reaches the second preset fatigue level (KSS level 8), if a fatigue warning message is received from the fatigue monitoring system, a fatigue test result indicating that the test type of the current test round is a true valid type (TP) is generated, and this round of testing is ended. If a fatigue warning message is not received from the fatigue monitoring system, the test type of the current test round is set to a false invalid type, and an additional test is conducted. Specifically, during the test time interval of the additional test, if the fatigue level of the first designated driver does not reach the second preset fatigue level, the test type of the current test round is marked as abnormal data. It should be noted that the abnormal data is only recorded. A fatigue test result indicating that the test type of the current test round is a true invalid type is generated, and this round of testing is ended. If the fatigue level of the first designated driver reaches the preset fatigue level (higher than or equal to KSS level 8), a fatigue test result indicating that the test type of the current test round is a false negative type (FN) is generated, and this round of testing is ended. For example, the scoring sequence can be 7-8-8, 7-9-9, or 7-9-8. Among them, the test types also include true invalid types and false invalid types.

[0079] Furthermore, replace the first designated driver and repeat the fatigue test on the vehicle to be tested until the number of fatigue test results of the specified type reaches the preset quantity threshold, and the test of the fatigue monitoring system is completed. The specified type is the true valid type or the false negative type. That is, when the number of test results TP or FN is greater than 10 events, the test of the vehicle to be tested is terminated.

[0080] Further, after all the first designated drivers have completed driving, based on the fatigue test results of each round of testing, determine the index values corresponding to each preset evaluation item, compare each index value with the preset index threshold associated with the corresponding preset evaluation item, and determine the first effectiveness evaluation result according to the comparison result. The preset evaluation index items include, but are not limited to, the average sensitivity and the lower limit of the 90% confidence interval of the sensitivity result. The first effectiveness evaluation result includes an evaluation result indicating that the system is effective or an evaluation result indicating that the system is ineffective. Specifically, after all the first designated drivers have completed driving, use the following formula 1 to calculate the single-sample sensitivity of the first designated drivers participating in the test.

[0081]

[0082] Among them, n(TP) is the sample size of the fatigue test results associated with a single first designated driver indicating that the test type is the true effective type (TP). n(FN) is the sample size of the fatigue test results associated with a single first designated driver indicating that the test type is the false negative type (FN). Next, use the following formula 2 to calculate the average sensitivity of all the first designated drivers participating in the test.

[0083]

[0084] Among them, Number of participants is the total number of samples. Next, use the following formula 3 to calculate the standard deviation of sensitivity.

[0085]

[0086] Next, use the following formula 4 to calculate the lower limit of the 90% confidence interval of the sensitivity result.

[0087]

[0088] In the embodiment of the present application, for the closed-site test, if the average sensitivity of the drivers participating in the test is higher than 40% and the lower limit of the 90% confidence interval of the sensitivity result is higher than 20%, the test is qualified. For the open-road test, if the average sensitivity of the drivers participating in the test is higher than 37.5% and the lower limit of the 90% confidence interval of the sensitivity result is higher than 17.5%, the test is qualified.

[0089] 102. Establish a vehicle coordinate system based on a three-dimensional H-point device and a coordinate measuring machine, and determine the coordinate interval of the distraction area according to the vehicle coordinate system, and determine the coordinates of multiple distraction points according to the coordinate interval of the distraction area.

[0090] In the embodiments of the present application, the distraction monitoring system of the vehicle to be tested can be evaluated before or after evaluating the fatigue monitoring system of the vehicle to be tested. In the evaluation test preparation stage, it is necessary to train the second designated driver participating in the test. The training content includes: (1) The ADDW system architecture, including the hardware and the basic algorithm of how the system monitors driver distraction. The test engineer will indicate to the driver the location of the driver status monitoring (DSM) camera; (2) The nominal situation and the non-nominal situations related to the driver declared by the manufacturer. If non-nominal situations are declared, what postures or actions should the driver take to create such situations; (3) The difference between long-term distraction and intermittent distraction (VATS). The driver should be informed that long-term distraction will be used in this test, and intermittent distraction and body movements other than head postures should be avoided; (4) The difference between lizard-type distraction and owl-type distraction. The driver's distraction movements to each fixation point should be as natural as possible, such as lizard-type, or owl-type, or a combination of both, depending on the position of the measured fixation point; (5) The test vehicle speed and all fixation points to be tested; (6) The driver should confirm that the DSM camera can be seen at the adjusted seat position to ensure that the DSM camera can monitor the driver's status and there is no occlusion caused by vehicle interior parts (such as the steering wheel).

[0091] Further, according to the preset division criteria, the distraction area of the vehicle to be tested should be divided and distraction points should be set before the test starts. Specifically, first, the tire pressure of the vehicle is inflated to the standard tire pressure, the vehicle is placed on a dry and flat ground, and the seat, backrest angle, and steering wheel are adjusted to the initial design positions. Next, a three-dimensional H-point device is installed on the driver's seat. This device is used to confirm the actual H-point coordinates of the vehicle. Further, install a Figure 5 coordinate measuring instrument as shown to establish a vehicle coordinate system, and check whether the position of the three-dimensional H-point device is accurate. If it is not accurate, it should be adjusted.

[0092] Further, according to the preset division criteria and the vehicle coordinate system, the visual area of the vehicle to be tested is divided to obtain the area coordinates of the distraction area. The distraction area includes a first distraction area, a second distraction area, and a third distraction area. Among them, the first distraction area is any area outside two vertical planes in the vehicle to be tested, and the two vertical planes intersect at the eye reference point. The second distraction area is 10° around the windshield and window areas. The third distraction area is any area below the plane 30° downward from the eye reference point. It should be noted that the eye reference point is located directly above the H point, and the distance from the H point satisfies the preset distance value of 635 mm. As Figure 6As shown in the figure, the distraction areas are marked with yellow, blue, and red tape to mark the boundaries of each area. The distraction points include the left knee point a of the driver, the right knee point b of the driver, the crotch point c of the driver, the foot point d of the co-driver, the surface point e of the co-driver's seat, the glove box point f of the co-driver, the left air outlet point g of the driver, the right air outlet point h of the driver, the instrument panel point i, the steering wheel point j, the shifter point k, the air conditioning control area point l, and the information entertainment display and center console point m. During actual operation, the vehicle interior image can be captured by a laser line projector or a camera, and the distraction areas can be identified and marked using image processing algorithms for division.

[0093] 103. When a second designated driver drives the vehicle under test, obtain the driving video associated with the second designated driver, the corresponding test vehicle speed range of the driving video, and multiple distraction warning messages issued by the distraction monitoring system for system effectiveness evaluation, and determine the second effectiveness evaluation result of the distraction monitoring system.

[0094] In the embodiment of the present application, a test is carried out using an evaluation system. The system includes a microphone, three cameras, and a set of GPS speed measurement units. The acoustic warning signal is picked up by the microphone arranged on the instrument panel. The three cameras include two measurement cameras and an auxiliary camera. As Figure 7 shown, the first measurement camera is arranged at each fixation point, focusing on the driver's eyes, and is used to determine the start time when the driver gazes at the fixation point. This camera has an infrared illumination function and can effectively monitor the driver's gaze direction even during night tests. The second measurement camera focuses on the visual signal display area. The auxiliary camera is arranged in front of the driver without blocking the field of view required for the driver's safe driving, and is used to monitor the start time when the driver's gaze direction moves away from the non-distraction area.

[0095] The acoustic warning signal collected by the microphone, the video signals collected by the three cameras, and the vehicle speed signal collected by the GPS module will be synchronized in the data acquisition system of the evaluation system and further data analysis will be carried out. In order to visualize the data during the test process, facilitate on-site comparison by the test personnel, judge the quality of the driver attention monitoring system, and at the same time eliminate the need for the test personnel to manually record, making the data of the entire test process highly integrated and visualized and convenient for export, the embodiment of the present application also provides a visualization interface. As Figure 8 shown, for the test of the distraction monitoring system, the distraction actions of the second designated driver, the real-time vehicle speed, the distraction warning signal of the distraction monitoring system, and the alarm interface of the distraction monitoring system are visualized. During the test process, the OBS software is used to record the computer screen, and the frame rate of the recording is set above 25 FPS.

[0096] The ADDW verification test is a real-vehicle test with real people participating on a closed site or an open road. A safety officer is arranged for the test to intervene in the test vehicle speed through the secondary brake pedal to ensure the test safety. For the current test round, determine the second designated driver associated with the current test round and the test data of the current test round. The test data includes the test data in the first vehicle speed range and the test data in the second vehicle speed range. It should be noted that the first vehicle speed range is the speed range of 20 to 35 Km / h, and the second vehicle speed range is the speed range of 50 to 65 Km / h. Each of the two speed ranges is tested at least once, with a total of 14 fixation points, that is, each driver has to be tested 28 times. Before the test starts, the driver should drive the vehicle for more than 1 minute. And after each point is tested, wait for more than 15 seconds before testing the next point.

[0097] For the current test round, after the test, use video distraction software to analyze the driving video frame by frame to determine the first time point when the second designated driver's gaze falls on each distraction point, and to determine the distraction point indicated by each of the distraction warning messages in the test data and the second time point when each of the distraction warning messages is issued. Calculate the time difference between the second time point associated with each distraction point and the corresponding first time point, and compare the time difference with the time interval threshold associated with the test vehicle speed range in the test data to determine the point test result corresponding to each distraction point. For example, as Figure 8 shown, find the starting moment of driver distraction. The first moment when the driver's gaze falls on the fixation distraction point (the fixation distraction point is the driver's left knee). Find the time positioning of this frame through the video software and record it as T1. Find the starting moment of ADDW alarm. The first moment when the ADDW audio alarm feature appears. Find the time positioning of this frame through the video software and record it as T2. The time difference obtained by T2 - T1 is the distraction alarm time T0. When testing in the speed range of 20 to 35 Km / h, T0 should not be greater than 6.5S; when testing in the speed range of 55 to 65 Km / h, T0 should not be greater than 4S. All fixation point tests meeting the requirements are considered qualified. If there is any point test result indicating unqualified, generate a distraction test result indicating unqualified. Otherwise, generate a distraction test result indicating qualified. Replace the second designated driver and drive the vehicle to be tested again for the distraction test until the number of distraction test results reaches the preset number threshold to complete the test of the distraction monitoring system. When all distraction test results indicate qualified, generate a second effectiveness evaluation result indicating the system is effective. When there is any distraction test result indicating unqualified, generate a second effectiveness evaluation result indicating the system is ineffective. Specifically, as Figure 9 shown is the test record of a driver passing all points after participating in the test.

[0098] 104. Determine the target evaluation result of the driver attention monitoring system of the vehicle to be tested according to the first effectiveness evaluation result and the second effectiveness evaluation result.

[0099] If both the first effectiveness evaluation result and the second effectiveness evaluation result indicate that the system is effective, generate a target evaluation result for indicating that the driver attention monitoring system is effective. If the first effectiveness evaluation result and / or the second effectiveness evaluation result indicates that the system is ineffective, generate a target evaluation result for indicating that the driver attention monitoring system is ineffective. Such an evaluation method ensures the accuracy and reliability of the driver attention monitoring system and provides technical support for ensuring road traffic safety.

[0100] For the evaluation method of the driver attention monitoring system provided in this embodiment, when a first designated driver drives the vehicle to be tested, obtain the fatigue level of the first designated driver and the fatigue warning information sent by the fatigue monitoring system, and determine the first effectiveness evaluation result of the fatigue monitoring system according to the fatigue level and the fatigue warning information. Establish a vehicle coordinate system based on the three-dimensional H-point device and the coordinate measuring machine, and determine the coordinate interval of the distraction area according to the vehicle coordinate system, and determine the coordinates of multiple distraction points according to the coordinate interval of the distraction area. When a second designated driver drives the vehicle to be tested, obtain the driving video associated with the second designated driver, the corresponding test vehicle speed interval of the driving video, and multiple distraction warning information sent by the distraction monitoring system for system effectiveness evaluation, and determine the second effectiveness evaluation result of the distraction monitoring system. Determine the target evaluation result of the driver attention monitoring system of the vehicle to be tested according to the first effectiveness evaluation result and the second effectiveness evaluation result. In the embodiment of the present application, the driver attention monitoring system includes a fatigue monitoring system and a distraction monitoring system. For the effectiveness evaluation of the fatigue detection system, the evaluation system uses the driver's self-assessed fatigue level to analyze whether the fatigue monitoring system can issue a fatigue warning message that matches the fatigue level, and then judges the effectiveness of the fatigue monitoring system. It reduces the need for manual observation and recording of driver behavior and solves the problem of dependence on manual operation. For the effectiveness judgment of the distraction monitoring system, the present application establishes a vehicle coordinate system based on the three-dimensional H-point device and the coordinate measuring machine, divides the distraction area according to the test requirements and sets distraction points, which improves the accuracy of monitoring, reduces the evaluation deviation caused by inaccurate area division, and solves the problem of subjective factor influence. Through the above methods, the need for manual operation can be reduced, the efficiency and accuracy of evaluation can be improved. At the same time, it reduces the inconsistency and deviation caused by subjective factors, and improves the reliability and effectiveness of the evaluation result.

[0101] Further, as Figure 1 a specific implementation of the method shown, this embodiment provides an evaluation system for a driver attention monitoring system, as Figure 10 shown, the system includes:

[0102] The first evaluation module is used to, when a first designated driver drives the vehicle to be tested, obtain the fatigue level of the first designated driver and the fatigue warning information sent by the fatigue monitoring system, and determine the first effectiveness evaluation result of the fatigue monitoring system according to the fatigue level and the fatigue warning information. The fatigue level is uploaded by the first designated driver according to the Karolinska Sleepiness Scale (KSS) scoring system;

[0103] The setting module is used to establish a vehicle coordinate system based on a three-dimensional H-point device and a three-coordinate measuring instrument, determine the coordinate interval of the distraction area according to the vehicle coordinate system, and determine the coordinates of multiple distraction points according to the coordinate interval of the distraction area;

[0104] The second evaluation module is used to, when a second designated driver drives the vehicle to be tested, obtain the driving video associated with the second designated driver, the corresponding test vehicle speed interval of the driving video, and multiple distraction warning information sent by the distraction monitoring system for system effectiveness evaluation, and determine the second effectiveness evaluation result of the distraction monitoring system. The driving video includes the distraction video of the second designated driver's gaze falling on each distraction point;

[0105] The determination module is used to determine the target evaluation result of the driver attention monitoring system of the vehicle to be tested according to the first effectiveness evaluation result and the second effectiveness evaluation result.

[0106] It should be noted that for other corresponding descriptions of each functional unit involved in the evaluation system of a driver attention monitoring system provided in this embodiment, reference can be made to the corresponding description in Figure 1 and will not be elaborated here.

[0107] Based on the method as shown in Figure 1 Accordingly, this embodiment further provides a storage medium on which a computer program is stored. When the program is executed by a processor, it implements the evaluation method of the driver attention monitoring system as shown in Figure 1 above.

[0108] Based on such an understanding, the technical solution of this application can be embodied in the form of a software product. The software product to be recognized can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various implementation scenarios of this application.

[0109] Based on the method as shown in Figure 1 above, and Figure 10An evaluation system embodiment of the driver attention monitoring system shown. To achieve the above object, refer to Figure 11 This embodiment also provides an entity device for evaluating a driver attention monitoring system, which can specifically be a personal computer, a server, a smart phone, a tablet computer, a smart watch, or other network devices, etc. This device includes a communication bus, a processor, a memory, and a communication interface, and may also include an input / output interface and a display device. Among them, each functional unit can complete mutual communication through the bus. The memory stores a computer program, and the processor is used to execute the program stored on the memory to execute the evaluation method of the driver attention monitoring system in the above embodiment.

[0110] Optionally, the entity device may further include a user interface, a network interface, a camera, a Radio Frequency (RF) circuit, sensors, an audio circuit, a WI-FI module, etc. The user interface may include a display screen (Display), an input unit such as a keyboard (Keyboard), etc. Optionally, the user interface may further include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.

[0111] Those skilled in the art can understand that the structure of the entity device for the evaluation method of the driver attention monitoring system provided in this embodiment does not limit the entity device, and it may include more or fewer components, or combine certain components, or have different component arrangements.

[0112] The storage medium may further include an operating system and a network communication module. The operating system is a program for managing the hardware and software resources to be recognized of the above entity device, and supports the operation of the information processing program and other software and / or programs to be recognized. The network communication module is used to implement the communication between the components inside the storage medium, and the communication between other hardware and software in the information processing entity device.

[0113] Those skilled in the art can understand that the drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the drawings are not necessarily essential for implementing this application. Those skilled in the art can understand that the modules in the device in the implementation scenario can be distributed in the device in the implementation scenario according to the description of the implementation scenario, or can be correspondingly changed to be located in one or more devices different from this implementation scenario. The modules in the above implementation scenario can be combined into one module, or further split into multiple sub-modules.

[0114] The above serial numbers of this application are only for description and do not represent the advantages or disadvantages of the implementation scenarios. The above disclosure is only several specific implementation scenarios of this application. However, this application is not limited thereto, and any changes that can be conceived by those skilled in the art should fall within the protection scope of this application.

Claims

1. A method for evaluating a driver attention monitoring system, characterized in that: The driver attention monitoring system includes a fatigue monitoring system and a distraction monitoring system. The method is applicable to an evaluation system and includes: When there is a first designated driver driving the vehicle to be tested, obtaining the fatigue level of the first designated driver and the fatigue warning information issued by the fatigue monitoring system, and determining a first effectiveness evaluation result of the fatigue monitoring system according to the fatigue level and the fatigue warning information, wherein the fatigue level is uploaded by the first designated driver based on the Karolinska Sleepiness Scale (KSS) scoring system; Establishing a whole vehicle coordinate system based on the three-dimensional H-point device and the three-coordinate measuring machine, and determining the coordinate interval of the distraction area according to the whole vehicle coordinate system, and determining the coordinates of multiple distraction points according to the coordinate interval of the distraction area; When there is a second designated driver driving the vehicle to be tested, obtaining a driving video associated with the second designated driver, a test vehicle speed interval corresponding to the driving video, and a plurality of distraction warning messages issued by the distraction monitoring system to perform a system effectiveness evaluation, and determining a second effectiveness evaluation result of the distraction monitoring system, wherein the driving video includes a distraction video in which the second designated driver's eyes fall on each of the distraction points; A target evaluation result of the driver attention monitoring system of the vehicle to be tested is determined according to the first effectiveness evaluation result and the second effectiveness evaluation result.

2. The method according to claim 1, characterized in that The step of obtaining the fatigue level of the first designated driver and the fatigue warning information issued by the fatigue monitoring system, and determining a first effectiveness evaluation result of the fatigue monitoring system according to the fatigue level and the fatigue warning information, includes: For the current test round, determining a first designated driver associated with the current test round; When the fatigue level of the first designated driver's self-assessment reaches a first preset level, the fatigue level of the first designated driver is obtained for detection based on the Karolinska Sleepiness Scale (KSS) scoring system at every preset time interval; When it is detected that the fatigue level reaches a second preset fatigue level, if a fatigue warning message issued by the fatigue monitoring system is received, a fatigue test result indicating that the test type of the current test round is a true and valid type is generated; Replacing the first designated driver to drive the vehicle to be tested again to perform fatigue testing, until the number of fatigue test results of a designated type reaches a preset number threshold, thereby completing the test of the fatigue monitoring system, wherein the designated type is a true valid type or a false negative type; According to the fatigue test results of each round of testing, the index value corresponding to each preset evaluation item is determined, and the first effectiveness evaluation result is determined according to the index value. The preset evaluation index items include but are not limited to the average sensitivity and the lower limit of the 90% confidence interval of the sensitivity result. The first effectiveness evaluation result includes an evaluation result indicating that the system is effective or an evaluation result indicating that the system is invalid.

3. The method according to claim 2, characterized in that After detecting that the fatigue score reaches a preset fatigue level, the method further includes: If the fatigue warning information sent by the fatigue monitoring system is not received, the test type of the current test round is set to a false invalid type, and an additional test is performed; During the test time interval of the additional test, if the fatigue level of the first designated driver does not reach the second preset fatigue level, marking the test type of the current test round as abnormal data, and generating a fatigue test result indicating that the test type of the current test round is a true invalid type; If the fatigue level of the first designated driver reaches the preset fatigue level, a fatigue test result indicating that the test type of the current test round is a false negative type is generated.

4. The method according to claim 1, characterized in that The method of establishing a vehicle coordinate system based on the three-dimensional H-point device and the three-coordinate measuring machine, determining the coordinate interval of the distraction area according to the vehicle coordinate system, and determining the coordinates of multiple distraction points according to the coordinate interval of the distraction area includes: According to the preset division standard and the whole vehicle coordinate system, the visual area of ​​the vehicle to be tested is divided to obtain the regional coordinates of the distraction area, wherein the distraction area includes a first distraction area, a second distraction area and a third distraction area, wherein the first distraction area is any area of ​​the vehicle to be tested located outside two vertical planes, the two vertical planes intersecting with the eye reference point, the second distraction area is 10° around the windshield and window area, and the third distraction area is any area below the plane 30° downward from the eye reference point, wherein the eye reference point is located directly above point H, and the distance between the eye reference point and point H meets the preset distance value; According to the preset division standard and the coordinate range of the distraction area, the coordinates of multiple distraction points are determined, and the distraction points include the driver's left knee point, the driver's right knee point, the driver's crotch point, the front passenger's footstep point, the front passenger's seat surface point, the front passenger's glove box point, the driver's left air-conditioning outlet point, the driver's right air-conditioning outlet point, the instrument panel point, the steering wheel point, the shifter point, the air-conditioning control area point, the infotainment display screen and the center console point.

5. The method according to claim 4, characterized in that The obtaining of the driving video associated with the second designated driver, the test vehicle speed range corresponding to the driving video, and the plurality of distraction warning messages issued by the distraction monitoring system to perform system effectiveness evaluation, and determining a second effectiveness evaluation result of the distraction monitoring system includes: For the current test round, determining a second designated driver associated with the current test round, obtaining a driving video associated with the second designated driver, a test vehicle speed range corresponding to the driving video, and a plurality of distraction warning messages issued by the distraction monitoring system; Identify each distraction video recorded in the driving video, determine a first time point when the second designated driver's gaze falls on each distraction point, and determine a distraction point indicated by each distraction warning message and a second time point when each distraction warning message is issued; Calculating the time difference between the second time point associated with each of the distraction points and the corresponding first time point, comparing the time difference with the time interval threshold associated with the test vehicle speed interval, and determining the point test result corresponding to each distraction point; If any point test result indicates that the test fails, a distraction test result indicating that the test fails is generated; otherwise, a distraction test result indicating that the test passes is generated; Replacing the second designated driver with the vehicle to be tested to perform the distraction test again, until the number of distraction test results reaches a preset number threshold, thereby completing the test of the distraction monitoring system; When all the distraction test results indicate that the test is qualified, generating a second effectiveness evaluation result for indicating that the system is effective; When any distraction test result indicates that the test is unqualified, a second effectiveness evaluation result indicating that the system is invalid is generated.

6. The method according to any one of claim 5, characterized in that: The step of comparing the time difference with a time interval threshold associated with a test vehicle speed interval in the test data to determine a point test result corresponding to each distraction point includes: If the comparison result indicates that the time difference is less than or equal to the corresponding time interval threshold, generating a point test result indicating that the distraction point test is qualified; If the comparison result indicates that the time difference is higher than the corresponding time interval threshold, a point test result indicating that the distraction point test is unqualified is generated.

7. The method according to claim 1, characterized in that Determining a target evaluation result of the driver attention monitoring system according to the first effectiveness evaluation result and the second effectiveness evaluation result includes: If both the first effectiveness evaluation result and the second effectiveness evaluation result indicate that the system is effective, generating a target evaluation result for indicating that the driver attention monitoring system is effective; If the first effectiveness evaluation result and / or the second effectiveness evaluation result indicates that the system is invalid, a target evaluation result is generated for indicating that the driver attention monitoring system is invalid.

8. The method according to claim 1, characterized in that The method further comprises: For the test of the fatigue monitoring system, the fatigue warning signal, vehicle speed, the fatigue level of the driver's self-assessment by clicking on the KSS scoring recording system, the alarm interface of the fatigue monitoring system, and the behavior monitoring video results of the first designated driver are collected and visualized; For the test of the distraction monitoring system, the distraction action of the second designated driver, the real-time vehicle speed, the distraction warning signal of the distraction monitoring system are collected, and the alarm interface of the distraction monitoring system is visualized.

9. An evaluation system for a driver attention monitoring system, characterized in that: The system comprises: A first evaluation module is used for obtaining a fatigue level of the first designated driver and fatigue warning information issued by the fatigue monitoring system when there is a first designated driver driving the vehicle to be tested, and determining a first effectiveness evaluation result of the fatigue monitoring system according to the fatigue level and the fatigue warning information, wherein the fatigue level is uploaded by the first designated driver according to the Karolinska Sleepiness Scale (KSS) scoring system; A setting module, used for establishing a whole vehicle coordinate system based on a three-dimensional H-point device and a three-coordinate measuring machine, and determining a coordinate interval of a distraction area according to the whole vehicle coordinate system, and determining coordinates of a plurality of distraction points according to the coordinate interval of the distraction area; a second evaluation module, for obtaining a driving video associated with the second designated driver, a test vehicle speed interval corresponding to the driving video, and a plurality of distraction warning messages issued by the distraction monitoring system to perform a system effectiveness evaluation and determine a second effectiveness evaluation result of the distraction monitoring system when there is a second designated driver driving the vehicle to be tested, the driving video including a distraction video in which the second designated driver's eyes fall on each of the distraction points; A determination module is used to determine a target evaluation result of the driver attention monitoring system of the vehicle to be tested according to the first effectiveness evaluation result and the second effectiveness evaluation result.

10. A device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images