Method for evaluating acceptance level of perception delay error of automatic driving system

Abstract

The invention relates to an automatic driving system perception delay error acceptability level evaluation method. The method comprises the following steps: determining a perception delay error generation process of a tested system and corresponding scene state parameters; determining a complete collision avoidance process of the tested system based on the scene state parameters of the sensing delay error occurrence process and the emergency collision avoidance capability of the tested system; determining the collision avoidance process of the mature human driver in the same scene as a comparison reference by referring to the mature human driver emergency collision avoidance model; calculating a perception delay error level judgment index, and determining whether a perception delay error level basic judgment inequality is established or not; according to the perception delay error level judgment index, a perception delay error acceptable level judgment coordinate system is established, and the perception delay error acceptable level of the tested system is judged in combination with a perception delay error level basic judgment inequality. Compared with the prior art, the method can clearly sense a delay error acceptable level judgment principle from the whole vehicle level of the automatic driving system.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving perception test and evaluation, and particularly relates to a method for evaluating the acceptable level of perception delay errors in an autonomous driving system. Background Art

[0002] As the "eyes" of intelligent connected vehicles, the perception system provides true external information for the vehicle, and its robustness and reliability are directly related to the safety of the vehicle during driving. However, the perception system is easily affected by various triggering conditions such as bad weather conditions and complex traffic environments, resulting in perception errors and its inability to achieve the expected function, leading to problems of Safety of the Intended Functionality (SOTIF), such as changes in environmental light, sensor occlusion, obstacles during vehicle driving, etc. Since it is inevitable for perception system errors to occur, it is particularly important to explore whether the risks brought by perception system errors are acceptable, which is a key step in promoting the iteration, popularization, and implementation of the perception system.

[0003] Although current research on individual perception modules or algorithms under different triggering conditions has been relatively in-depth, there are still challenges in how to conduct test and evaluation of the acceptable level of perception system errors at the system level. Traditional test methods mostly rely on scenario-based grid experiments and evaluate their performance by comparing with the driving performance of human drivers in the same scenario. However, such methods do not clarify the acceptable principles of the driving performance of autonomous driving systems and cannot distinguish and evaluate the level of decline in driving performance caused by perception errors, making it difficult for experimental results to objectively reflect the acceptable level of perception system errors.

[0004] Therefore, there is an urgent need for a determination method to evaluate the acceptable level of perception system errors at the system level, in order to clarify the determination principles of the acceptable level of perception errors and provide a scientific basis for error attribution analysis in the development and testing of perception systems. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for evaluating the acceptable level of perception delay errors in an autonomous driving system to overcome the defects of the above-mentioned existing technologies.

[0006] The purpose of the present invention can be achieved through the following technical solutions: A method for evaluating the acceptable level of perception delay errors in an autonomous driving system, comprising the following steps:

[0007] S1. Clarify the occurrence process of perception delay errors in the system under test and the corresponding scenario state parameters;

[0008] S2. Determine the complete collision avoidance process of the system under test based on the scenario state parameters of the occurrence process of perception delay errors and the emergency collision avoidance ability of the system under test;

[0009] S3. Refer to the emergency collision avoidance model of mature human drivers, and determine the collision avoidance process of mature human drivers in the same scenario as the comparison benchmark.

[0010] S4. Based on the collision avoidance process of mature human drivers and the complete collision avoidance process of the system under test, calculate the determination index of the perception delay error level, and determine whether the basic determination inequality of the perception delay error level holds.

[0011] S5. According to the determination index of the perception delay error level, establish a determination coordinate system for the acceptable level of the perception delay error. Combine the basic determination inequality of the perception delay error level to determine the acceptable level of the perception delay error of the system under test.

[0012] Furthermore, the step S1 includes the following steps:

[0013] S11. Analyze the triggering conditions and triggering mechanisms that cause perception delay errors in the scenario.

[0014] S12. Determine the potential risk factors in the scenario corresponding to the occurrence of perception delay errors. The potential risk factors include target traffic elements that require the system under test to identify and perform collision avoidance behaviors in a timely manner.

[0015] S13. Simplify the scenario based on the interaction relationship between the system under test and the potential risk factors in the scenario, and retain the key scenario elements.

[0016] S14. Determine the initial moment T0 and the risk detection moment T1 when the potential risk factors in the specific scenario can be perceived, and the scenario state parameters at the corresponding moments.

[0017] Furthermore, the potential risk factors in the step S12 are located in front of the vehicle under test, and the speed in the driving direction of the vehicle under test is lower than that of the vehicle under test.

[0018] The key scenario elements in the step S13 include the braking of the vehicle in front, the cutting in of the vehicle in front, and the cutting out of the vehicle in front and encountering a stationary vehicle in front.

[0019] The scenario state parameters in the step S14 include the initial distance D0 between the system under test and the potential risk factors, and the driving speeds of the system under test and the target elements.

[0020] Furthermore, the step S2 includes the following steps:

[0021] S21. Determine the collision avoidance strategy based on the specific scenario or the actual driving decision result of the system under test.

[0022] S22. Based on the process of the occurrence of perception delay errors, assume that the system under test travels at a constant speed without performing collision avoidance behaviors, and calculate the perception delay error duration T of the system under test. err, that is, the interval duration from the moment when the perceived risk can be sensed to the moment when the risk is detected:

[0023] T err = T1 - T0

[0024] S23. Calculate the system's limit collision avoidance distance D for the corresponding scenario based on the scenario state parameters at the risk detection moment and the system's limit collision avoidance ability of the system under test avo_col .

[0025] Furthermore, the collision avoidance strategy in step S21 includes emergency braking collision avoidance and emergency steering collision avoidance.

[0026] Furthermore, step S3 includes the following steps:

[0027] S31. Based on the collision avoidance strategy of the system under test, select a mature human driver's emergency collision avoidance model with the same trigger conditions as the benchmark;

[0028] S32. Determine the risk perception duration T of the mature human driver model react , that is, the duration from the moment when the risk can be perceived by the mature human driver to the moment when the collision avoidance decision response is made;

[0029] S33. Calculate the limit collision avoidance distance D of the mature human driver in the same scenario based on the scenario state parameters at the decision response moment and the mature human driver collision avoidance model parameters avo_col_human .

[0030] Furthermore, the mature human driver's emergency collision avoidance model in step S31 includes a mature human driver's emergency braking model and an emergency steering model.

[0031] Furthermore, step S4 includes the following steps:

[0032] S41. For the braking performance of the system under test, calculate the judgment indexes of the perception delay error level, including the perception error duration ratio R1 and the limit braking distance ratio R2. Specifically:

[0033]

[0034] Among them, R1 is the perception error duration ratio, which represents the time ratio of the collision avoidance response delay caused by the perception system's failure to sense the target element (possible risk) in time; R2 is the limit collision avoidance distance ratio, which represents the ratio of the distance required to achieve complete collision avoidance in the initial distance of the scenario; the perception error duration T erris the duration from the moment when the risk can be perceived to the moment when the perception system stably perceives; D0 is the initial distance of the scenario, that is, the initial distance between the host vehicle and the target element at the moment when the risk can be perceived; TTC0 is the TTC value at the moment when the risk can be perceived, which is calculated based on the initial distance D0 of the scenario and the relative speed between the measured vehicle and the target element; D avo_col is the extreme collision avoidance distance, which is determined based on the collision avoidance ability of the measured vehicle in a specific scenario;

[0035] S42. Determine whether the measured system can avoid a collision through the following inequality, that is, the basic determination of the perception delay error level:

[0036] R1 ≤ 1 - R2

[0037] When the inequality holds, it means that after the perception delay error occurs, the measured system still maintains a sufficient distance from the vehicle in front to be able to perform an emergency collision avoidance. Therefore, the perception delay error level is generally acceptable;

[0038] S43. For the braking performance of the benchmark mature human driver model, calculate the determination index of the perception delay error level, including the proportion of the risk perception duration R 1_human and the proportion of the extreme braking distance R 2_human , specifically:

[0039]

[0040] Among them, R 1_human is the proportion of the perception duration of the mature human driver risk model, R 2_human is the proportion of the extreme collision avoidance distance of the mature human driver model, T react is the risk perception duration of the mature human driver model, D avo_col_human is the extreme collision avoidance distance of the mature human driver model.

[0041] Furthermore, the step S5 includes the following steps:

[0042] S51. Based on the perception delay error acceptable determination formula, construct a perception delay error acceptable level determination coordinate system, where the horizontal axis is the proportion of the extreme collision avoidance distance R2, and the vertical axis is the proportion of the perception error duration R1;

[0043] S52. Determine the perception delay error acceptable marking line R1 = 1 - R2 in the coordinate system, and judge the collision avoidance situation through the position relationship between the coordinate point and this marking line. When the coordinate point is located in the lower part of the oblique line, it means the collision avoidance is successful, and when it is located in the upper part, it means the collision avoidance fails;

[0044] S53. The coordinate point corresponding to the braking performance of the mature human driver model (R 2_human , R 1_human)Plotted on the determination coordinate system, and the coordinate axes are divided into different regions based on the position of the coordinate points of mature human drivers;

[0045] S54. Plot the corresponding coordinate point (R2, R1) of the braking performance of the system under test on the determination coordinate system, and determine the acceptable level of perception delay error based on the region where it is located.

[0046] Furthermore, the effective region of the acceptable level determination coordinate system in step S51 is the region delimited by R1 ∈ [0, 1] and R2 ∈ [0, 1]. When R1 > 1, it indicates that the perception system has not detected the target element at all during the process, belonging to the perception failure region; when R2 > 1, it indicates that in the corresponding initial parameters of the scenario, due to the collision avoidance ability of the vehicle, collision avoidance cannot be completed, belonging to the non-collision-avoidable scenario region.

[0047] Furthermore, the specific process of step S53 is as follows:

[0048] When the human driver does not have a collision, the corresponding benchmark coordinate point of the mature human driver is located in the part below the acceptable mark of perception delay error. At this time, the part above the mark is the unacceptable region of perception delay error. Based on the ordinate of the benchmark coordinate point of the mature human driver, the lower region is divided into the acceptable region of perception delay error and the region where the perception delay error is acceptable and the response is better than that of humans;

[0049] When the human driver has a collision, the corresponding benchmark coordinate point of the mature human driver is located in the part above the acceptable mark of perception delay error. At this time, the part below the mark is the acceptable region of perception delay error. Based on the ordinate of the benchmark coordinate point of the mature human driver, the region above the mark is divided into the upper part of the unacceptable region of perception and the lower part of the tolerable region of perception delay error.

[0050] Furthermore, the specific process of step S54 is as follows:

[0051] If the human driver does not have a collision in the same scenario, when the coordinate point of the system under test is located in the acceptable region of perception delay error, it indicates that the system under test has successfully avoided a collision, but the perception response is slower than that of the human driver, and the perception delay error is acceptable; when the coordinate point is located in the region where the perception delay error is acceptable and the response is better than that of humans, it indicates that the system under test has successfully avoided a collision and the perception response is better than that of humans, and the perception delay error is acceptable; when the coordinate point is located in the unacceptable region of perception delay error, it indicates that the system under test has failed to avoid a collision, and the perception delay error is unacceptable.

[0052] When a collision occurs to a human driver in the same scenario, if the coordinate point of the system under test is within the acceptable region of perception delay error, it indicates that the system under test has successfully avoided the collision, with braking performance better than that of the human driver and acceptable perception delay error; if the coordinate point is within the unacceptable region of perception delay error, it indicates that the system under test has failed to avoid the collision and its perception response is slower than that of the human driver, with unacceptable perception delay error; if the coordinate point is within the tolerable region of perception delay error, it indicates that the system under test has failed to avoid the collision, but its perception response is better than that of the human driver, with tolerable perception delay error.

[0053] Compared with the prior art, the present invention has the following advantages:

[0054] The present invention first determines the complete collision avoidance process of the system under test based on the scenario state parameters during the occurrence process of perception delay error and the emergency collision avoidance ability of the system under test; then, with reference to the emergency collision avoidance model of a mature human driver, determines the collision avoidance process of a mature human driver in the same scenario as the comparison benchmark; then, based on the collision avoidance process of the mature human driver and the complete collision avoidance process of the system under test, calculates the determination index of the perception delay error level and determines whether the basic determination inequality of the perception delay error level holds; finally, based on the determination index of the perception delay error level, establishes a determination coordinate system for the acceptable level of perception delay error, and combines it with the basic determination inequality of the perception delay error level to determine the acceptable level of perception delay error of the system under test. Thus, a determination scheme for the acceptable level of perception delay error of an autonomous driving perception system is realized, which can, starting from the vehicle level of the autonomous driving system, clarify the determination principle of the acceptable level of perception delay error, calculate the determination coordinate points of the acceptable level of perception delay error corresponding to the complete collision avoidance performances of the human driver and the system under test by benchmarking with the mature human driver model, and realize the graphical determination of the acceptable level of perception delay error of the perception system.

[0055] Based on the GAMAB (Globally at least as good) principle, the present invention takes the mature human driver model as the benchmark for the acceptable level of perception delay error, and based on the occurrence process of perception delay error and the collision avoidance performance of the system under test under different triggering conditions, and by benchmarking with the collision avoidance performance of the mature human driver model, realizes the determination of the acceptable level of perception delay error at the vehicle level of the system, filling the blank of test evaluation in related fields; at the same time, the present invention can serve different stages such as the design and research and development, test verification of the autonomous driving system, and the attribution analysis after the occurrence of an autonomous driving accident, and has an important promoting effect and practical value on the iterative upgrade of the perception system of high-level intelligent networked vehicles.

[0056] The present invention first proposes a graphical determination method based on a coordinate system for determining the acceptable level of perception delay errors. Its core is to obtain key parameters in the logical scenario, calculate the determination index of the perception delay error level and form coordinate points. By comparing the relative positions of the human benchmark and the coordinate points corresponding to the braking process of the system under test, the graphical determination of the acceptable level of perception system errors can be achieved.

[0057] The present invention has good applicability and ease of use. The key parameters of the scenario used in the present invention (including the initial distance between the system under test and potential risk factors, the driving speed of the system under test and the target element, etc.) are easy to obtain. In addition, based on the collision avoidance processes of mature human drivers and the system under test, the determination index of the perception delay error level (including the proportion of the risk perception duration and the proportion of the extreme braking distance) can be conveniently and quickly calculated, which is applicable to most emergency collision avoidance scenarios. Brief Description of the Drawings

[0058] Figure 1 is a schematic flow chart of the method of the present invention;

[0059] Figure 2 is a schematic diagram of the determination principle of the acceptable level of perception delay errors in the embodiment;

[0060] Figure 3 is the specific test scenario conditions in the embodiment;

[0061] Figure 4 is the rendering of the simulation reproduction of the test scenario in the embodiment;

[0062] Figure 5 is a schematic diagram of the complete braking process of the system under test in the embodiment;

[0063] Figure 6 is a schematic diagram of the key parameters of the scenario where the leading vehicle cuts out and encounters a stationary leading vehicle in the embodiment;

[0064] Figure 7 is the analysis result diagram of the acceptable level of perception delay errors of the collision avoidance case in the embodiment;

[0065] Figure 8 is the analysis result diagram of the acceptable level of perception delay errors of the collision case in the embodiment; Detailed Embodiment

[0066] The present invention will be described in detail below with reference to the drawings and specific embodiments.

[0067] Embodiment

[0068] As Figure 1 shown, a method for evaluating the acceptable level of perception delay errors of an autonomous driving system includes the following steps:

[0069] S1. Clearly define the process of the perception delay error occurring in the system under test and the corresponding scenario state parameters;

[0070] S2. Based on the scenario state parameters of the process of the perception delay error occurring and the emergency collision avoidance ability of the system under test, determine the complete collision avoidance process of the system under test;

[0071] S3. Refer to the emergency collision avoidance model of a mature human driver, and determine the collision avoidance process of a mature human driver in the same scenario as the comparison benchmark;

[0072] S4. Based on the collision avoidance process of a mature human driver and the complete collision avoidance process of the system under test, calculate the determination index of the perception delay error level, and determine whether the basic determination inequality of the perception delay error level holds;

[0073] S5. According to the determination index of the perception delay error level, establish a determination coordinate system for the acceptable level of the perception delay error, and combine it with the basic determination inequality of the perception delay error level to determine the acceptable level of the perception delay error of the system under test.

[0074] In this embodiment, the above scheme is applied. Based on the GAMAB (Globally at least as good) principle, the mature human driver model is used as the benchmark for the acceptable level of the perception delay error, ensuring that the overall response performance of the system under test under the triggering conditions should not be lower than that of a mature human driver. The overall determination principle is as Figure 2 shown, including the following steps:

[0075] Step 1: Clearly define the process of the perception delay error occurring in the system under test and analyze the corresponding specific scenarios;

[0076] Step 2: Based on the scenario state parameters of the process of the perception delay error occurring and the emergency collision avoidance ability of the system under test, determine the complete collision avoidance process of the system under test;

[0077] Step 3: Select a benchmark mature human driver emergency collision avoidance model, and determine the collision avoidance process of a mature human driver in the same scenario as the comparison benchmark;

[0078] Step 4: Based on the collision avoidance processes of a mature human driver and the system under test, calculate the determination index of the perception delay error level, and determine whether the basic determination inequality of the perception delay error level holds;

[0079] Step 5: Determine the acceptable level of the perception delay error of the system under test through the determination coordinate system of the acceptable level of the perception delay error.

[0080] The applicable perception error of the above method is limited to: the system under test has a delay in perceiving potential risk factors due to specific triggering conditions.

[0081] Among them, Step 1 includes the following steps:

[0082] Step 11: Analyze the triggering conditions and their triggering mechanisms that cause perception delay errors in the scenario.

[0083] Step 12: Determine the potential risk factors in the scenario corresponding to the occurrence of perception delay errors, where the potential risk factors include, but are not limited to, target traffic elements that require the system under test to identify and take collision avoidance actions in a timely manner.

[0084] Step 13: Simplify the scenario based on the interaction relationship between the system under test and the potential risk factors in the scenario, and retain the key scenario elements, including, but not limited to, the slow driving of the vehicle in front, the cutting-in of the vehicle in front, the cutting-out of the vehicle in front and encountering a stationary vehicle in front, etc.

[0085] Step 14: Determine the initial moment T0 and the risk detection moment T1 at which the potential risk factors in the specific scenario can be perceived, and the scenario state parameters at the corresponding moments, including the initial distance D0 between the system under test and the potential risk factors, the driving speeds of the system under test and the potential risk factors, etc.

[0086] Among them, the applicable preconditions of the above method are that the vehicle is located on a straight one-way two-lane road, the potential risk factors are in front of the vehicle under test, and the speed in the driving direction of the vehicle under test is lower than that of the vehicle under test. The potential risk factors may include vehicles, pedestrians, two-wheel vehicles, and stationary obstacles, etc.

[0087] Step 2 includes the following steps:

[0088] Step 21: Determine the collision avoidance strategy based on the specific scenario or the actual driving decision result of the system under test, including emergency braking collision avoidance and emergency steering collision avoidance.

[0089] Calculate the perception error duration T of the system under test based on the occurrence process of the perception delay error err , that is, the interval duration from the moment T0 when the risk can be perceived to the risk detection moment T1. It is assumed that the system under test travels at a constant speed without taking collision avoidance measures during T err . The calculation formula is as follows:

[0090] T err = T1 - T0

[0091] Calculate the system's limit collision avoidance distance D under the corresponding scenario based on the scenario state parameters at the risk detection moment and the system's limit collision avoidance ability avo_col .

[0092] Step 3 includes the following steps:

[0093] Step 31: Based on the collision avoidance strategy of the system under test, select the mature human driver's emergency collision avoidance models under the same triggering conditions, including the mature human driver's emergency braking model and emergency steering model.

[0094] Step 32: Determine the risk perception duration T of the mature human driver model react , that is, the duration from the moment when the risk can be perceived by the mature human driver to the moment when a collision avoidance decision response is made

[0095] Step 33: Calculate the extreme collision avoidance distance D of the mature human driver in the same scenario based on the scene state parameters at the decision response moment and the parameters of the mature human driver collision avoidance model avo_col_human .

[0096] Step 4 includes the following steps:

[0097] Step 41: For the braking performance of the system under test, calculate the judgment indexes of the perception delay error level: the proportion of the perception error duration R1 and the proportion of the extreme braking distance R2. The index calculation formulas are as follows:

[0098]

[0099] Among them, R1 is the proportion of the perception error duration, representing the proportion of the time of the collision avoidance response delay caused by the perception system failing to perceive the target element (possible risk) in time; R2 is the proportion of the extreme collision avoidance distance, representing the proportion of the distance required to achieve complete collision avoidance in the initial distance of the scene; D0 is the initial distance of the scene, that is, the initial distance between the host vehicle and the target element at the moment when the risk can be perceived; the perception delay error duration T err is the duration from the moment when the risk can be perceived to the moment when the perception system stably perceives; TTC0 is the TTC value at the moment when the risk can be perceived, calculated based on the initial distance D0 of the scene and the relative speed between the vehicle under test and the target element; D avo_col is the extreme collision avoidance distance, determined based on the collision avoidance ability of the vehicle under test in a specific scenario

[0100] Step 42: Determine whether the system under test can avoid collision through the following inequality, that is, the basic judgment of the perception delay error level:

[0101] R1 ≤ 1 - R2

[0102] When the inequality holds, it means that after the perception delay error occurs, the system under test still maintains a sufficient distance from the vehicle in front to perform emergency collision avoidance, so the perception delay error level is generally acceptable

[0103] Step 43: For the braking performance of the benchmark mature human driver model, calculate the judgment indexes of the perception delay error level: the proportion of the risk perception duration R 1_human and the proportion of the extreme braking distance R 2_human . The index calculation formulas are as follows:

[0104]

[0105] Among them, R1_human The proportion of the perception duration in the risk model of a mature human driver, R 2_human The proportion of the ultimate collision avoidance distance in the mature human driver model, T react The risk perception duration of the mature human driver model, D avo_col_human The ultimate collision avoidance distance of the mature human driver model.

[0106] The specific content of step 5 is as follows:

[0107] Step 51: Based on the perception delay error level determination index, construct a perception delay error acceptable level determination coordinate system, with the horizontal axis being the proportion of the ultimate collision avoidance distance R2 and the vertical axis being the proportion of the perception error duration R1.

[0108] Among them, the effective area in the acceptable level determination coordinate system is the area framed by R1 ∈ [0, 1] and R2 ∈ [0, 1]. When R1 > 1, it indicates that the perception system fails to detect the target element during the process, belonging to the perception failure area; when R2 > 1, it indicates that limited by the collision avoidance ability of the vehicle under the initial parameters of the corresponding scenario, collision avoidance cannot be completed, belonging to the non-collision avoidance area.

[0109] Step 52: Determine the acceptable marking line of the perception delay error in the coordinate system as R1 = 1 - R2, and judge the collision avoidance situation based on the position relationship between the coordinate point and this marking line. When the coordinate point is below the diagonal line, it indicates successful collision avoidance; when it is above, it indicates failed collision avoidance.

[0110] Step 53: Plot the coordinate point (R 2_human , R 1_human ) corresponding to the braking performance of the mature human driver model on the determination coordinate system, and divide the coordinate axes into different regions based on the position of the coordinate point of the mature human driver.

[0111] Specifically, the division method of different acceptable level regions is as follows:

[0112] When no collision occurs to the human driver, the corresponding benchmark coordinate point of the mature human driver is below the acceptable marking line of the perception delay error. At this time, the upper part of the marking line is the unacceptable region of the perception delay error. Further, based on the ordinate of the benchmark coordinate point of the mature human driver, the lower region is divided into an acceptable region of the perception delay error and a region where the perception delay error is acceptable and the response is better than that of humans.

[0113] When a collision occurs to the human driver, the corresponding benchmark coordinate point of the mature human driver is in the upper part of the acceptable marking line of the perception delay error. At this time, the lower part of the marking line is the acceptable region of the perception delay error. Further, based on the ordinate of the benchmark coordinate point of the mature human driver, the upper region is divided into an upper part of the perception unacceptable region and a lower part of the tolerable region of the perception delay error.

[0114] Step 54: Plot the coordinate points (R2, R1) corresponding to the braking performance of the system under test on the judgment coordinate system, and determine the acceptable level of perception delay error based on the region where they are located.

[0115] Specifically, the determination methods for different levels of perception delay error are as follows:

[0116] No collision occurred to the human driver in the same specific scenario: When the coordinate points of the system under test are located in the acceptable region of perception delay error, it indicates that the system under test successfully avoided a collision, but the perception response is slower than that of the human driver, and the perception delay error is acceptable; when the coordinate points are located in the region where the perception delay error is acceptable and the response is better than that of the human, it indicates that the system under test successfully avoided a collision and the perception response is better than that of the human, and the perception delay error is acceptable; when the coordinate points are located in the unacceptable region of perception delay error, it indicates that the system under test failed to avoid a collision, and the perception delay error is unacceptable.

[0117] Collision occurred to the human driver in the same scenario: When the coordinate points of the system under test are located in the acceptable region of perception delay error, it indicates that the system under test successfully avoided a collision, and the braking performance is better than that of the human driver, and the perception delay error is acceptable; when the coordinate points are located in the unacceptable region of perception delay error, it indicates that the system under test failed to avoid a collision and the perception response is slower than that of the human, and the perception delay error is unacceptable; when the coordinate points are located in the tolerable region of perception delay error, it indicates that the system under test failed to avoid a collision, but the perception response is better than that of the human driver, and the perception delay error is tolerable.

[0118] In this embodiment, the scenario where the leading vehicle cuts out and encounters a stationary leading vehicle is selected as the logical scenario for method verification, and the domain controller including the perception function is selected as the system under test, and the test is carried out on the hardware-in-the-loop test bench to determine the acceptable level of perception delay error of the system under test. Among them, the scenario includes clear perception trigger conditions, that is, the features of the target vehicle are partially blocked, and the system under test needs to detect in time after a part of the stationary leading vehicle is revealed and adopt a braking strategy to avoid a collision.

[0119] In the selection of specific test scenarios, referring to the parameter combinations of the leading vehicle cut-out scenario in the White Paper on Mature Driving Models for Intelligent Connected Vehicles, 20 groups of test scenarios as shown in Figure 3 are selected and reproduced using the simulation software 51-Simone as the input of the system under test, and the reproduction effect is as shown in Figure 4 .

[0120] Analyzing the specific test scenarios, the complete braking process of the system under test is as shown in Figure 5 . Among them, the moment when the leading vehicle is revealed corresponds to the risk perceivable moment T0, and the stable perception moment (AEB Flag response) corresponds to the risk detection moment T1.

[0121] At the moment when the system under test stably perceives, that is, the moment when the system determines the AEB intervention, obtain the state parameters of the corresponding scenario. Select the ideal AEB model based on TTC, and calculate the complete braking process of the system under test based on the scenario state parameters. Its braking trigger threshold is TTC = 1.5 s. When the TTC with the vehicle ahead in the scenario is less than 1.5 s, the model intervenes and applies a braking deceleration of 0.9 g.

[0122] Select the mature driver braking model in the "White Paper on Mature Driving Models for Intelligent Connected Vehicles" as the benchmark to construct the braking behavior of human drivers in the corresponding test scenarios. In the scenario where the vehicle ahead cuts out, the decision-making response time of human drivers is 1.29 s, the braking efficiency improvement time is 0.51 s, and it can be approximately a linear braking pressure build-up process, with a maximum braking deceleration of -7.93 m / s^2.

[0123] Through the hardware-in-the-loop bench test, obtain the key scenario state parameters included as Figure 6 Based on the specific scenario, transform the determination formula in step 4 into the following three formulas:

[0124] R1 ≤ 1 - R2

[0125]

[0126] In the formula, R2 is the proportion of the extreme braking distance, representing the proportion of the distance required to achieve full braking in the initial distance; D bra is the extreme braking distance, which is determined by theoretical calculation based on the braking model parameters and the initial state parameters of the scenario.

[0127] By calculating the coordinate points of the experimental results in 20 scenarios where the vehicle ahead cuts out and comparing them with the braking reference points of human drivers, the statistical results shown in Table 1 are obtained by summarization, including the number of case points in each acceptable level area.

[0128] Table 1 Summary of the acceptable levels of perception delay errors for test cases

[0129]

[0130] The results show that due to the ideal braking effect of the AEB model, in most experimental cases, the perception system errors can be accepted. In addition, since the algorithm is required to continuously and stably detect the target object when the domain controller under test outputs the AEB Flag response, no cases where the response is better than that of human drivers occur in the experimental cases.

[0131] This embodiment also selects two typical cases for specific analysis based on the coordinate system for determining the acceptable level of errors, as Figure 7 and Figure 8 shown. In Figure 7In the collision avoidance case shown, neither the human driver nor the system under test collided, so the coordinate points of both are below the acceptable line for perception delay errors. However, since its perception response is not as fast as that of the human driver, it is within the acceptable region for perception delay errors.

[0132] In Figure 8 In the collision case shown, the system under test collided while the human driver did not, so the coordinate point of the system under test is above the performance, belonging to the unacceptable region for perception delay errors. By analyzing the cause of the collision, it can be found that the proportion of the error duration of the perception system is too high, almost close to 1. When the AEB Flag is triggered, there is only 0.69 m from the stationary vehicle in front, and the vehicle does not have enough distance to complete braking.

Claims

1. An evaluation method for the acceptable level of perception delay error of an autonomous driving system, characterized in that It includes the following steps: S1. Define the occurrence process of the perception delay error of the system under test and the corresponding scenario state parameters; S2. Determine the complete collision avoidance process of the system under test based on the scenario state parameters of the occurrence process of the perception delay error and the emergency collision avoidance ability of the system under test; S3. Refer to the emergency collision avoidance model of a mature human driver to determine the collision avoidance process of a mature human driver in the same scenario as the comparison benchmark; S4. Calculate the judgment index of the perception delay error level based on the collision avoidance process of a mature human driver and the complete collision avoidance process of the system under test, and determine whether the basic judgment inequality of the perception delay error level holds; S5. According to the judgment index of the perception delay error level, establish a judgment coordinate system for the acceptable level of the perception delay error, and combine it with the basic judgment inequality of the perception delay error level to judge the acceptable level of the perception delay error of the system under test.

2. The method for evaluating the acceptable level of perception delay error of an autonomous driving system according to claim 1, wherein The step S1 includes the following steps: S11. Analyze the triggering conditions and triggering mechanisms that cause the perception delay error in the scenario; S12. Determine the potential risk factors in the scenario corresponding to the occurrence of the perception delay error. The potential risk factors include the target traffic elements that require the system under test to identify and perform collision avoidance behaviors in a timely manner; S13. Simplify the scenario based on the interaction relationship between the system under test and the potential risk factors in the scenario, and retain the key scenario elements; S14. Determine the initial moment T0 and the risk detection moment T1 when the potential risk factors in the specific scenario can be perceived, and the corresponding scenario state parameters.

3. A method for evaluating the acceptable level of perception delay error of an autonomous driving system according to claim 2, characterized in that, In the step S12, the potential risk factors are located in front of the vehicle under test, and the speed in the driving direction of the vehicle under test is lower than that of the vehicle under test; In the step S13, the key scenario elements include the braking of the vehicle in front, the cutting in of the vehicle in front, and the cutting out of the vehicle in front encountering a stationary vehicle in front; In the step S14, the scenario state parameters include the initial distance D0 between the system under test and the potential risk factors, and the driving speeds of the system under test and the target elements.

4. The method for evaluating the acceptable level of perception delay error of an autonomous driving system according to claim 3, wherein The step S2 includes the following steps: S21. Determine the collision avoidance strategy based on the specific scenario or the actual driving decision result of the system under test; S22. Based on the occurrence process of the perception delay error, assuming that the system under test is driving at a constant speed without performing collision avoidance behavior, calculate the perception delay error duration T of the system under test err , that is, the interval duration from the moment when the perceived risk can be perceived to the moment when the risk is detected T err = T1 - T0 S23. Calculate the system's ultimate collision avoidance distance D for the corresponding scenario based on the scenario state parameters at the risk detection moment and the ultimate collision avoidance ability of the system under test avo_col .

5. The method for evaluating the acceptable level of perception delay error of an autonomous driving system according to claim 4, characterized in that, The step S3 includes the following steps: S31. Select the emergency collision avoidance model of the benchmark mature human driver under the same triggering conditions based on the collision avoidance strategy of the system under test; S32. Determine the risk perception duration T of the mature human driver model react , that is, the duration from the moment when the risk can be perceived by the mature human driver to the moment when a collision avoidance decision response is made S33. Calculate the extreme collision avoidance distance D of a mature human driver in the same scenario based on the scenario state parameters at the decision response moment and the parameters of the mature human driver collision avoidance model avo_col_human .

6. The method for evaluating the acceptable level of perception delay error of an autonomous driving system according to claim 5, wherein The step S4 includes the following steps: S41. For the braking performance of the system under test, calculate the judgment index of the perception delay error level, including the proportion of the perception error duration R1 and the proportion of the extreme braking distance R2. Specifically: Among them, R1 is the proportion of the perception error duration, representing the proportion of the time of the collision avoidance response delay caused by the failure of the perception system to perceive the target element (possible risk) in time; R2 is the proportion of the limit collision avoidance distance, representing the proportion of the distance required to achieve complete collision avoidance in the initial distance of the scene; the perception error duration T err is the duration from the moment when the risk can be perceived to the moment when the perception system stably perceives; D0 is the initial distance of the scene, that is, the initial distance between the host vehicle and the target element at the moment when the risk can be perceived; TTC0 is the TTC value at the moment when the risk can be perceived, calculated based on the initial distance D0 of the scene and the relative speed between the measured vehicle and the target element; D avo_col is the limit collision avoidance distance, determined based on the collision avoidance ability of the measured vehicle in a specific scene; S42. Determine whether the system under test can avoid a collision through the following inequality, that is, the basic judgment of the perception delay error level: R1 ≤ 1 - R2 When the inequality holds, it means that after the perception delay error occurs, the system under test still maintains a sufficient distance from the vehicle in front to perform emergency collision avoidance. Therefore, the overall perception delay error level is acceptable; S43. Calculate the judgment index of the perception delay error level for the braking performance benchmarked against the mature human driver model, including the risk perception duration ratio R 1_human and the extreme braking distance ratio R 2_human , specifically as follows: Among them, R 1_human is the proportion of the perception duration in the mature human driver risk model, and R 2_human is the proportion of the ultimate collision avoidance distance in the mature human driver model. T react is the risk perception duration of the mature human driver model, and D avo_col_human is the ultimate collision avoidance distance of the mature human driver model.

7. A method for evaluating the acceptable level of perception delay error of an autonomous driving system according to claim 6, characterized in that, The step S5 includes the following steps: S51. Based on the acceptable judgment formula of the perception delay error, construct a judgment coordinate system for the acceptable level of the perception delay error, where the horizontal axis is the proportion of the extreme collision avoidance distance R2, and the vertical axis is the proportion of the perception error duration R1; S52. Determine the acceptable marking line of perception delay error in the coordinate system as R1 = 1 - R2, and judge the collision avoidance situation based on the position relationship between the coordinate point and this marking line. When the coordinate point is located in the lower part of the oblique line, it indicates successful collision avoidance; when it is located in the upper part, it indicates failed collision avoidance. S53. Plot the coordinate points (R 2_human , R 1_human ) corresponding to the braking performance of the mature human driver model on the determination coordinate system, and divide the coordinate axes into different regions based on the positions of the coordinate points of the mature human driver; S54. Plot the coordinate point (R2, R1) corresponding to the braking performance of the system under test on the determination coordinate system, and determine the acceptable level of perception delay error based on the area where it is located.

8. A method for evaluating the acceptable level of perception delay error of an autonomous driving system according to claim 7, characterized in that, In the step S51, the effective area of the acceptable level determination coordinate system is the area framed by R1 ∈ [0, 1] and R2 ∈ [0, 1]. When R1 > 1, it indicates that the perception system fails to detect the target element completely during the process, belonging to the perception failure area; when R2 > 1, it indicates that in the initial parameters of the corresponding scenario, due to the collision avoidance ability of the vehicle being limited, collision avoidance cannot be completed, belonging to the non-collision-avoidable scenario area.

9. The method for evaluating the acceptable level of perception delay error of an autonomous driving system according to claim 8, wherein The specific process of the step S53 is as follows: When there is no collision for the human driver, the corresponding benchmark coordinate point of the mature human driver is located in the lower part of the acceptable marking line of perception delay error. At this time, the upper part of the marking line is the unacceptable area of perception delay error. Based on the ordinate of the benchmark coordinate point of the mature human driver, the lower area is divided into the acceptable area of perception delay error and the area where the perception delay error is acceptable and the response is better than that of humans. When there is a collision for the human driver, the corresponding benchmark coordinate point of the mature human driver is located in the upper part of the acceptable marking line of perception delay error. At this time, the lower part of the marking line is the acceptable area of perception delay error. Based on the ordinate of the benchmark coordinate point of the mature human driver, the upper area of the marking line is divided into the upper part of the unacceptable area of perception and the lower part of the tolerable area of perception delay error.

10. A method for evaluating the acceptable level of perception delay error of an autonomous driving system according to claim 8, characterized in that The specific process of the step S54 is as follows: If there is no collision for the human driver in the same scenario, when the coordinate point of the system under test is located in the acceptable area of perception delay error, it indicates that the system under test has successfully avoided a collision, but the perception response is slower than that of the human driver, and the perception delay error is acceptable; when the coordinate point is located in the area where the perception delay error is acceptable and the response is better than that of humans, it indicates that the system under test has successfully avoided a collision and the perception response is better than that of humans, and the perception delay error is acceptable; when the coordinate point is located in the unacceptable area of perception delay error, it indicates that the system under test has failed to avoid a collision, and the perception delay error is unacceptable. If there is a collision for the human driver in the same scenario, when the coordinate point of the system under test is located in the acceptable area of perception delay error, it indicates that the system under test has successfully avoided a collision and the braking performance is better than that of the human driver, and the perception delay error is acceptable; when the coordinate point is located in the unacceptable area of perception delay error, it indicates that the system under test has failed to avoid a collision and the perception response is slower than that of the human driver, and the perception delay error is unacceptable; when the coordinate point is located in the tolerable area of perception delay error, it indicates that the system under test has failed to avoid a collision, but the perception response is better than that of the human driver, and the perception delay error is tolerable.

Images