False braking evaluation method for automatic driving system

Abstract

The invention relates to the technical field of automobile automatic driving and intelligent network connection, in particular to a false braking evaluation method for an automatic driving system, which comprises the following steps of: S1, constructing an induction factor system for false braking of the automatic driving system, and analyzing a sensitive incidence relation of induction factors of each dimension; s2, constructing an induction factor gain model; s3, constructing a test working condition; the test working conditions comprise a single test working condition for testing by a single induction factor and a composite test working condition for testing by a plurality of induction factors; s4, on the basis of the sensing gain characteristics of the induction factor gain model, respectively determining sensing gain compensation coefficients of different sensing schemes under each test working condition; s5, performing harmfulness analysis on each test working condition, and endowing a working condition hazard score; s6, determining an actual test score and a false braking recovery compensation score of each single working condition; and S7, constructing an automatic driving system false braking test evaluation model.

Description

Technical Field

[0001] This invention relates to the fields of autonomous driving and intelligent connected vehicle technology, specifically to a method for evaluating the false braking of an autonomous driving system, and more particularly to a method for screening key test scenarios and evaluating the entire process by constructing an inducing factor gain model. Background Technology

[0002] Currently, the methods for evaluating the false braking of autonomous driving systems include simulation testing, closed-site testing, open road testing, and virtual-real combination testing. However, they are all essentially operational scenario testing, which tests the false triggering of an autonomous driving system through various operational scenarios to further evaluate its related performance. There are two main shortcomings at present.

[0003] First, existing methods are mostly based on discrete, typical scenarios for testing, lacking systematic analysis and modeling of deep, multi-dimensional inducing factors (such as the coupling effects of weather and infrastructure) of mis-braking events. This results in insufficient exploration of key test scenarios and low testing efficiency and coverage. Second, at the evaluation level, existing methods focus primarily on whether mis-braking has occurred, limiting themselves to "trigger evaluation" while neglecting the system's self-correction capability after mis-braking, i.e., "recovery evaluation." This fails to achieve a comprehensive and holistic assessment of the autonomous driving system's ability to withstand mis-braking risks. Summary of the Invention

[0004] To address the aforementioned issues, this invention provides a method for evaluating the mis-braking of autonomous driving systems, further improving the systematicness, comprehensiveness, and accuracy of such evaluation methods. Through a multi-dimensional evaluation system, it achieves scientific screening and efficient extraction of massive test conditions, ensuring focus on key scenarios. Furthermore, by introducing full-process evaluation indicators, it achieves the completeness and accuracy of single-test evaluations.

[0005] The technical solution adopted in this invention is to provide a method for evaluating the false braking of an autonomous driving system, comprising the following steps:

[0006] S1. Construct a system of inducing factors for erroneous braking in autonomous driving systems and analyze the sensitive correlation between inducing factors in each dimension;

[0007] S2. Based on the sensitive correlation between various inducing factors, construct an inducing factor gain model;

[0008] S3. Based on the inducing factor gain model, construct test conditions; test conditions include single test conditions containing a single inducing factor and composite test conditions containing multiple inducing factors.

[0009] S4. Based on the sensing gain characteristics of the inducing factor gain model, determine the sensing gain compensation coefficients for different sensing schemes under each test condition.

[0010] S5. Conduct a hazard analysis on each test condition and assign a hazard score to each condition;

[0011] S6. Determine the actual test score and the compensation score for accidental braking recovery for each single working condition;

[0012] S7. Construct a test and evaluation model for the erroneous braking of an autonomous driving system. The formula is as follows:

[0013] Formula 1

[0014] In Equation 1, S represents the evaluation score for the erroneous braking test of the autonomous driving system; i represents the i-th test condition; n represents the total number of test conditions; D i G represents the hazard score for the i-th test condition; i K represents the gain score for the i-th test condition; i T represents the perception gain compensation coefficient for the i-th test condition of the autonomous driving system; i R represents the actual test score for the i-th test condition; i This represents the compensation score for erroneous braking recovery in the i-th test condition.

[0015] The inducing factors in step S1 include weather, infrastructure, aerial interference, and vehicle status.

[0016] Step S3 also includes classifying the test conditions and assigning weights to each classification.

[0017] In the test conditions, the gain relationship between the various factors ranges from weak to strong, and the levels range from low to high, with the corresponding gain scores increasing sequentially.

[0018] The perception gain characteristics in step S4 include gains based on vision, millimeter-wave radar, and lidar perception.

[0019] The hazard score in step S5 increases as the hazard level increases.

[0020] The method for determining the actual test score for a single working condition in step S6 includes: calculating on a single test basis, scoring for passing the test and not scoring for failing the test; or scoring based on the probability of passing the test multiple times in the same working condition.

[0021] The compensation score for accidental braking recovery in step S6 is the compensation score obtained by the autonomous driving system after accidental braking occurs and when the braking is released within a preset time.

[0022] It also includes an optimization method for the test evaluation model of the autonomous driving system's mis-braking, comprising the following steps:

[0023] a. Calculate the pressure hazard factor for each test condition, using the following formula.

[0024] Formula 2

[0025] In Equation 2, Indicates the pressure hazard factor. This represents the hazard score for the i-th test condition. This represents the gain score for the i-th test condition. This represents the perception gain compensation coefficient for the autonomous driving system under the i-th test condition.

[0026] b. Sort the test conditions in order according to the pressure hazard coefficient;

[0027] c. Determine the target coverage constant A under test resource constraints, using the following formula.

[0028] Formula 3

[0029] In Equation 3, This represents the pressure hazard coefficient for the i-th test condition. This represents the test coverage constant;

[0030] d. Determine the actual range of test conditions that need to be tested based on the test coverage constant.

[0031] The beneficial effects of this invention are:

[0032] 1. By introducing the operating condition gain, perception gain compensation, and hazard score of the test conditions, a pressure hazard coefficient of the test conditions is constructed. An evaluation system for the test conditions is established from three dimensions: the correlation of operating condition elements, the correlation characteristics between the operating condition and the autonomous driving system under test, and the degree of danger of the operating condition itself, so as to achieve effective mining and extraction of key test conditions.

[0033] 2. By adding a compensation score for accidental braking recovery, the evaluation of the actual test process is improved. This takes into account the performance of the autonomous driving system throughout the entire working cycle, enabling a full-process evaluation of its accidental braking behavior and improving the completeness and accuracy of single-condition test evaluation. Attached Figure Description

[0034] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0035] like Figure 1 As shown, the present invention provides a method for evaluating the false braking of an autonomous driving system, comprising the following steps:

[0036] S1. Construct a system of inducing factors for erroneous braking in autonomous driving systems. Through analysis of autonomous driving perception principles, accident analysis, and expert consultation, analyze the sensitive correlations of inducing factors across various dimensions. Inducing factors include weather, infrastructure, aerial interference, and vehicle status. An example of the inducing factor system is shown in Table 1.

[0037] Table 1 Inducing Factor System

[0038] Serial Number Dimension elements 1 weather Including but not limited to rainfall, snowfall, hail, fog, dust storms, and light; 2 Road facilities and conditions (roadside) Including but not limited to guardrails, barriers, anti-glare and noise reduction devices, trash cans, roadblocks, and water-filled barriers on both sides of the road; 3 Road facilities and conditions (road surface and road surface) Including but not limited to manhole covers, water accumulation, snow accumulation, tree groves, speed bumps, overpasses, and height restriction barriers; 4 aerial interference Including but not limited to flying plastic bags, leaves, scraps of paper, and birds; 5 Vehicle status Including but not limited to loads and sensor stains;

[0039] S2. Based on the sensitive correlation between various inducing factors, construct an inducing factor gain model; examples of sensitive correlation are shown in Table 2:

[0040] Table 2 Sensitivity Associations with Inducing Factors

[0041] Serial Number Dimension elements Sensitive relationships 1 weather Including but not limited to rainfall, snowfall, hail, fog, dust storms, and light; 2 Road facilities and conditions (roadside) Including but not limited to guardrails, barriers, anti-glare and noise reduction devices, trash cans, roadblocks, and water-filled barriers on both sides of the road; Road alignment 3 Road facilities and conditions (road surface and road surface) Including but not limited to manhole covers, water accumulation, snow accumulation, tree foliage, speed bumps, and other overpasses and height restriction poles; Road longitudinal slope vehicle load 4 aerial interference Including but not limited to flying plastic bags, leaves, scraps of paper, birds, etc.; Lighting conditions 5 Vehicle status Including but not limited to loads and sensor stains;

[0042] S3. Based on the gain model of inducing factors, construct test conditions; test conditions include single test conditions containing a single inducing factor and composite test conditions containing multiple inducing factors; examples of gain models are shown in Table 3:

[0043] Table 3 Gain Model

[0044] Gain Relationship weather Road facilities and conditions (roadside) Road facilities and conditions (road surface and road surface) aerial features Vehicle status weather / Weak gain Weak gain strong gain Weak gain Road facilities and conditions (roadside) Weak gain / / Weak gain Weak gain Road facilities and conditions (road surface and road surface) Weak gain / / / strong gain aerial features strong gain Weak gain Weak gain Weak gain Weak gain Vehicle status Weak gain Weak gain strong gain Weak gain /

[0045] The test conditions are classified and assigned weights. The gain relationship between the factors in each test condition increases from weak to strong, and the classification increases from low to high, with the corresponding gain scores for each condition increasing sequentially. For example, the gain scores for the test conditions increase sequentially from single-inducing-factor test condition, first-level gain test condition, second-level gain test condition, to third-level gain test condition.

[0046] S4. Based on the inducing factor gain model, determine the perception gain compensation coefficients for different perception schemes under each test condition; the perception gain characteristics include the gains generated by vision, millimeter-wave radar, and lidar perception.

[0047] Perceived gain characteristics refer to further analysis of the gain characteristics under gain conditions:

[0048] For example, in scenario 1, there is a strong gain relationship between aerial interference (flying plastic bags, leaves, paper scraps, birds, etc.) and weather conditions (lighting conditions), which together constitute the operating condition. This strong gain relationship is primarily based on the influence of visual sensor occlusion, meaning that the impact of this operating condition is more severe on visual perception systems, resulting in greater perception compensation for visual perception systems and less for perception systems such as lidar and millimeter-scale radar.

[0049] For example, in case 2, there is a strong gain relationship between road infrastructure (manhole covers, etc.) and vehicle status (load), which together constitute the operating condition. This strong gain relationship primarily stems from the vehicle load affecting the vehicle's attitude and thus the sensor angle. Ultimately, due to changes in the radar pitch angle, misidentification and braking of manhole covers, etc., occur. In other words, the impact of this operating condition is more severe on the radar system, requiring greater perception compensation for radar sensing solutions, while providing less compensation for visual perception solutions.

[0050] S5. Conduct a hazard analysis on each test condition and assign a hazard score to each condition; the hazard score increases as the hazard level increases.

[0051] The hazard score for each working condition refers to the degree of danger of each working condition. It is a comprehensive assessment based on the severity of the accident that may be caused by mis-braking in that working condition, such as collision speed and risk of personal injury, and assigns a corresponding hazard score Di, which is quantified according to industry safety standards or expert scoring methods.

[0052] S6. Determine the actual test score and the compensation score for accidental braking recovery for each individual operating condition. The method for determining the actual test score for a single operating condition includes calculating on a single test basis, awarding points for passing and not awarding points for failing, or awarding points based on the probability of passing multiple tests under the same operating condition. The compensation score for accidental braking recovery is the preset compensation score that the automatic driving system obtains when it releases the brakes within a preset time after an accidental braking occurs.

[0053] The actual test score for a single operating condition is determined using the following two methods:

[0054] 1. Scoring is based on a single test pass. For example, passing a test under a certain working condition earns 1 point, while failing the test earns 0 points.

[0055] 2. Scoring is based on the probability of passing multiple tests under the same working conditions; for example, if 5 tests are conducted and all 5 pass results in 1 point, and 1 pass results in 0.2 points.

[0056] The compensation score for accidental braking recovery is the compensation score obtained by the autonomous driving system when the brake is released within a preset time after accidental braking occurs. For example, a compensation score of 0.3 points is obtained if the brake is released within 0.1 seconds of accidental braking, and 0 points are obtained if the brake is not engaged or is released after 0.1 seconds.

[0057] S7. Construct a test and evaluation model for the erroneous braking of an autonomous driving system. The formula is as follows:

[0058] Formula 1

[0059] In Equation 1, S represents the evaluation score for the erroneous braking test of the autonomous driving system; i represents the i-th test condition; n represents the total number of test conditions; D i G represents the hazard score for the i-th test condition;i K represents the gain score for the i-th test condition; i T represents the perception gain compensation coefficient for the i-th test condition of the autonomous driving system; i R represents the actual test score for the i-th test condition; i This represents the compensation score for erroneous braking recovery in the i-th test condition.

[0060] This invention also includes an optimization method for the test and evaluation model of a false braking system in an autonomous driving system, further optimizing the above model. While the above evaluation methods are accurate and comprehensive, they inevitably suffer from drawbacks such as long testing cycles and high costs. To address the need for faster and lower-cost testing, a rapid optimization method for the model is proposed, including the following steps:

[0061] a. Calculate the pressure hazard factor for each test condition, using the following formula.

[0062] Formula 2

[0063] In Equation 2, Indicates the pressure hazard factor. This represents the hazard score for the i-th test condition. This represents the gain score for the i-th test condition. This represents the perception gain compensation coefficient for the autonomous driving system under the i-th test condition.

[0064] b. Sort the test conditions in order according to the pressure hazard coefficient;

[0065] c. Determine the acceptable test coverage constant A, where 0 < A ≤ 1, using the following formula.

[0066] Formula 3

[0067] In Equation 3, This represents the pressure hazard coefficient for the i-th test condition. This represents the test coverage constant. The constant A is determined by setting the percentage of cumulative pressure hazard coefficients that are expected to be covered based on the resource limits of test time and test cost. For example, A is set such that the cumulative pressure hazard coefficients of the final selected test case set account for 80% of the cumulative values ​​of all test cases.

[0068] d. Based on the test coverage constant, determine the range of test conditions that actually need to be tested in the first to the i-th test conditions.

[0069] For example, for eight operating conditions, the pressure hazard factors are 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, and 0.8, respectively. The acceptable test coverage constant A is set at 50%.

[0070]

[0071]

[0072] The test range is then the three working conditions corresponding to pressure hazard factors of 0.8, 0.7, and 0.6.

Claims

1. A method for evaluating false braking in an autonomous driving system, characterized in that: Includes the following steps, S1. Construct a system of inducing factors for erroneous braking in autonomous driving systems and analyze the sensitive correlation between inducing factors in each dimension; S2. Based on the sensitive correlation between various inducing factors, construct an inducing factor gain model; S3. Based on the inducing factor gain model, construct test conditions; test conditions include single test conditions containing a single inducing factor and composite test conditions containing multiple inducing factors. S4. Based on the sensing gain characteristics of the inducing factor gain model, determine the sensing gain compensation coefficients for different sensing schemes under each test condition. S5. Conduct a hazard analysis on each test condition and assign a hazard score to each condition; S6. Determine the actual test score and the compensation score for accidental braking recovery for each single working condition; S7. Construct a test and evaluation model for the erroneous braking of an autonomous driving system. The formula is as follows: Formula 1 In Equation 1, S represents the evaluation score for the erroneous braking test of the autonomous driving system; i represents the i-th test condition; n represents the total number of test conditions; D i G represents the hazard score for the i-th test condition; i K represents the gain score for the i-th test condition; i T represents the perception gain compensation coefficient for the i-th test condition of the autonomous driving system; i This represents the actual test score for the i-th test condition; R i This represents the compensation score for erroneous braking recovery in the i-th test condition.

2. The method for evaluating false braking in an autonomous driving system according to claim 1, characterized in that: The inducing factors in step S1 include weather, infrastructure, aerial interference, and vehicle status.

3. The method for evaluating false braking in an autonomous driving system according to claim 1, characterized in that: Step S3 also includes classifying the test conditions and assigning weights to each classification.

4. The method for evaluating false braking in an autonomous driving system according to claim 3, characterized in that: In the test conditions, the gain relationship between the various factors ranges from weak to strong, and the levels range from low to high, with the corresponding gain scores increasing sequentially.

5. The method for evaluating false braking in an autonomous driving system according to claim 1, characterized in that: The perception gain characteristics in step S4 include gains based on vision, millimeter-wave radar, and lidar perception.

6. The method for evaluating false braking in an autonomous driving system according to claim 1, characterized in that: The hazard score in step S5 increases as the hazard level increases.

7. The method for evaluating false braking in an autonomous driving system according to claim 1, characterized in that: The method for determining the actual test score for a single working condition in step S6 includes: calculating on a single test basis, scoring for passing the test and not scoring for failing the test; or scoring based on the probability of passing the test multiple times in the same working condition.

8. The method for evaluating false braking in an autonomous driving system according to claim 1, characterized in that: The compensation score for accidental braking recovery in step S6 is the compensation score obtained by the autonomous driving system after accidental braking occurs and when the braking is released within a preset time.

9. A method for evaluating false braking in an autonomous driving system according to any one of claims 1-8, characterized in that: It also includes optimization methods for the test and evaluation model of automatic driving system mis-braking. Includes the following steps, a. Calculate the pressure hazard factor for each test condition, using the following formula. Formula 2 In Equation 2, Indicates the pressure hazard factor. This represents the hazard score for the i-th test condition. This represents the gain score for the i-th test condition. This represents the perception gain compensation coefficient for the autonomous driving system under the i-th test condition. b. Sort the test conditions in order according to the pressure hazard coefficient; c. Determine the acceptable test coverage constant A, using the following formula. Formula 3 In Equation 3, This represents the pressure hazard coefficient for the i-th test condition. This represents the test coverage constant; d. Determine the actual range of test conditions that need to be tested based on the test coverage constant.

Images