A Method for Quantitative Evaluation of the Performance of an Autonomous Vehicle Perception System

Abstract

The present invention relates to a method for quantitatively evaluating the performance of an autonomous vehicle perception system, comprising the following steps: 1) obtaining functional scenarios and perception system performance requirements according to autonomous driving functions and designed operating domains; 2) collecting perception system evaluation indicators and constructing an evaluation system for the autonomous vehicle perception system; 3) analyzing the triggering conditions of the perception system and constructing specific test scenarios for the perception system; 4) testing and verifying the perception system in specific test scenarios to determine whether it meets the performance requirements, and performing iteration of the perception system until the performance requirements are met; 5) outputting the evaluation results of the perception system. Compared with the prior art, the present invention meets the evaluation requirements in the development process of the perception system, provides guidance for the development of the autonomous vehicle perception system for expected functional safety, has strong operability and is easy to operate.

Description

Technical Field

[0001] The present invention relates to the field of autonomous vehicle test and evaluation, and particularly to a method for quantitatively evaluating the performance of an autonomous vehicle perception system. Background Art

[0002] An autonomous driving perception system refers to a system that an autonomous driving system obtains information from the environment and extracts relevant knowledge. It uses the information obtained by on-vehicle sensors to interact with the control unit and decision-making unit of the autonomous driving system, and generally includes two parts: a perception process and a cognitive process. Due to the complexity and randomness of the operating scenarios of the autonomous driving system, it is required that the autonomous driving perception system integrates multiple sensors and complex software algorithms to meet the system design requirements. However, due to the performance limitations of multiple sensors and complex software algorithms, the autonomous driving system may fail to achieve the expected functions, that is, the expected functional safety.

[0003] During the development process of autonomous driving functions, specific performance requirements are put forward for the autonomous driving perception system. However, under certain extreme conditions, the performance of the perception system cannot be guaranteed, and there are triggering conditions in terms of the performance limitations of the perception system, resulting in the autonomous driving perception system failing to meet the set performance requirements. Summary of the Invention

[0004] The purpose of the present invention is to overcome the above-mentioned defects existing in the prior art and provide a method for quantitatively evaluating the performance of an autonomous driving vehicle perception system, which meets the evaluation requirements during the development process of the perception system, provides guidance for the development of an autonomous driving vehicle perception system oriented to expected functional safety, has strong operability and is easy to operate.

[0005] The purpose of the present invention can be achieved through the following technical solutions:

[0006] A method for quantitatively evaluating the performance of an autonomous driving vehicle perception system includes the following steps:

[0007] 1) According to the autonomous driving function and the designed operating domain, obtain the functional scenarios and the performance requirements of the perception system;

[0008] 2) Generate specific test scenarios for the autonomous driving vehicle perception system according to the functional scenarios;

[0009] 3) Run the autonomous driving vehicle perception system in the specific test scenarios, and determine whether the running result meets the performance requirements of the perception system. If so, execute step 4); otherwise, iterate the autonomous driving vehicle perception system and execute step 3);

[0010] 4) According to the running result, calculate the quantitative evaluation result of the autonomous driving vehicle perception system through the evaluation system;

[0011] The evaluation method proposed by the present invention can systematically evaluate and test the perception system that relies on complex sensors and algorithms, identify and iterate the perception system, improve the accuracy, reliability and certainty of the autonomous driving perception system in complex dynamic traffic scenarios, meet the expected functional safety requirements of autonomous driving, meet the evaluation requirements in the development process of the perception system, and provide guidance for the development of the autonomous driving perception system.

[0012] Further, the autonomous driving functions include multiple autonomous driving functions such as HWP, AVP, TJP, and NOP. The designed operating domain includes static entities, dynamic entities, and vehicle functions. In terms of static entities, it is required that the perception system can operate normally under defined road conditions, weather conditions, lighting conditions, etc. In terms of dynamic entities, it is required that the perception system correctly identifies and tracks various traffic participants. In terms of vehicle functions, it is required that the perception system operates normally under the condition that the activated vehicle speed and various sensors are error-free.

[0013] The functional scenarios include sub-functions of autonomous driving functions and their operating conditions.

[0014] The performance requirements of the perception system illustrate the types, quantities, and specific implemented functions of sensors. The performance requirements of the perception system ensure that the autonomous driving function can achieve its expected behavior in its operating scenario. The performance requirements of the perception system include the accuracy, stability, timeliness, comprehensiveness, tracking accuracy, and recognition parameter accuracy of identifying static and dynamic entities.

[0015] Further, the evaluation system is a hierarchical structure. Each layer includes several evaluation indicators. Except for the lowest layer, the evaluation indicators of each layer are obtained by weighted calculation of the secondary indicators subordinate to it.

[0016] Step 4) includes:

[0017] According to the operation results, a quantitative evaluation result of the autonomous driving vehicle perception system is obtained through weighted calculation of the evaluation indicators.

[0018] Further, the highest layer of the evaluation system includes the quantitative evaluation indicators of sensor performance and the quantitative evaluation indicators of perception system performance.

[0019] The secondary indicators subordinate to the quantitative evaluation indicators of sensor performance include the reliability, cost, and stability of sensors.

[0020] The secondary indicators subordinate to the quantitative evaluation indicators of perception system performance include the recognition rate, accuracy, and robustness of the perception system in specific scenarios.

[0021] Further, the secondary indicators subordinate to the reliability of sensors include the failure rate, robustness, and average service life of sensors.

[0022] The secondary indicators belonging to the cost of the sensor include the number of sensors and the unit price of the sensors;

[0023] The secondary indicators belonging to the stability of the sensor include the signal-to-noise ratio of the sensor, the electromagnetic interference stability, and the operating temperature range.

[0024] Further, the recognition rate, that is, the proportion of the perception system recognizing the operating scenario, and the secondary indicators belonging to the recognition rate of the perception system include the miss detection rate of the perception system;

[0025] The secondary indicators belonging to the accuracy of the perception system include the false detection rate and the detection accuracy of the perception system;

[0026] The secondary indicators belonging to the robustness of the perception system include the reporting delay of the perception system, the target following characteristics, and the maximum effective detection distance, and the target following characteristics include the multi-target tracking accuracy and the multi-target tracking accuracy.

[0027] Further, the step 2) includes the following steps:

[0028] 201) Obtain the trigger conditions of the perception system of the autonomous vehicle;

[0029] 202) Obtain the parameter range of the trigger conditions;

[0030] 203) Generate specific test scenarios according to the functional scenario, the trigger conditions, and the parameter range of the trigger conditions.

[0031] Further, the step 201) includes:

[0032] Construct a trigger source knowledge base, a trigger mechanism knowledge base, and a trigger effect knowledge base. The trigger source knowledge base stores the trigger source elements affecting the perception system in a tree structure, and the trigger mechanism knowledge base and the trigger effect knowledge base are established based on semantic guiding words;

[0033] Construct a trigger condition analysis matrix through the trigger source knowledge base, the trigger mechanism knowledge base, and the trigger effect knowledge base;

[0034] Obtain the trigger conditions through the trigger condition analysis matrix.

[0035] Further, obtain the parameter range of the trigger conditions according to the natural driving data.

[0036] Further, the step 203) includes:

[0037] Obtain the parameter range of the functional scenario according to the natural driving data. Within the parameter ranges of the functional scenario and the trigger conditions, combine the functional scenario and the trigger conditions with specific parameters to generate several overall logical scenarios;

[0038] Construct a scene exposure rate judgment matrix with the occurrence frequency of the trigger source as the input and the scene exposure rate level as the judgment result. According to the occurrence frequency of the trigger source in the natural driving data, determine the scene exposure rate level of the overall logical scene through the scene exposure rate judgment matrix;

[0039] Construct a scene severity judgment matrix with the severity of the accident as the input and the scene severity level as the judgment result. According to the severity of the accident in the natural driving data, determine the scene severity level of the overall logical scene through the scene severity judgment matrix;

[0040] Construct a scene risk level judgment matrix with the scene exposure rate level and the scene severity level as the input and the scene risk level as the output. According to the scene exposure rate level and the scene severity level, determine the scene risk level of the overall logical scene through the scene risk level judgment matrix;

[0041] Select the overall logical scene with the highest scene risk level as the specific test scene.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] The evaluation method proposed by the present invention can systematically evaluate and test the perception system that relies on complex sensors and algorithms to work, identify and iterate the perception system, improve the accuracy, reliability and certainty of the autonomous driving perception system in complex dynamic traffic scenarios, meet the requirements of the expected functional safety of autonomous driving, meet the evaluation needs in the development process of the perception system, provide guidance for the development of the autonomous driving vehicle perception system for expected functional safety, and has strong operability and is easy to operate. Description of the Drawings

[0044] Figure 1 It is a schematic flow chart of the performance quantitative evaluation method for the autonomous driving vehicle perception system;

[0045] Figure 2 It is a structural diagram for analyzing the performance limitations of the perception system;

[0046] Figure 3 It is a schematic flow chart for generating the specific test scene. Detailed Embodiment

[0047] The present invention will be described in detail below with reference to the drawings and specific embodiments. This embodiment is implemented on the premise of the technical solution of the present invention, and gives the detailed implementation manner and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0048] A performance quantitative evaluation method for an autonomous driving vehicle perception system, as Figure 1 , includes the following steps:

[0049] 1) Obtain the functional scenarios and the performance requirements of the perception system according to the autonomous driving function and the designed operating domain;

[0050] 2) Generate specific test scenarios for the perception system of the autonomous driving vehicle according to the functional scenarios;

[0051] 3) Run the perception system of the autonomous driving vehicle in the specific test scenarios, and determine whether the running result meets the performance requirements of the perception system. If so, execute step 4); otherwise, iterate the perception system of the autonomous driving vehicle and execute step 3);

[0052] 4) Calculate the quantitative evaluation result of the perception system of the autonomous driving vehicle through the evaluation system according to the running result;

[0053] The evaluation method proposed in this embodiment can systematically evaluate and test the perception system that relies on complex sensors and algorithms, identify and iterate the perception system, improve the accuracy, reliability and certainty of the autonomous driving perception system in complex dynamic traffic scenarios, and meet the expected functional safety requirements of autonomous driving.

[0054] Step 2) includes the following steps:

[0055] 201) Obtain the trigger conditions of the perception system of the autonomous driving vehicle;

[0056] 202) Obtain the parameter ranges of the trigger conditions;

[0057] 203) Generate specific test scenarios according to the functional scenarios, trigger conditions and the parameter ranges of the trigger conditions.

[0058] Step 201) includes:

[0059] Construct a trigger source knowledge base, a trigger mechanism knowledge base and a trigger effect knowledge base. The trigger source knowledge base stores the trigger source elements affecting the perception system in a tree structure, and the trigger mechanism knowledge base and the trigger effect knowledge base are established based on semantic guiding words;

[0060] Construct a trigger condition analysis matrix through the trigger source knowledge base, the trigger mechanism knowledge base and the trigger effect knowledge base;

[0061] According to the trigger condition analysis matrix, obtain the trigger conditions through the analysis of the limitations of the perception system.

[0062] Obtain the parameter ranges of the trigger conditions according to the natural driving data.

[0063] As Figure 3 , step 203) includes:

[0064] Obtain the parameter range of the functional scenario based on natural driving data. Within the parameter range of the functional scenario and the trigger condition, combine the functional scenario and the trigger condition with specific parameters to generate several overall logical scenarios;

[0065] Construct a scenario exposure rate judgment matrix with the occurrence frequency of the trigger source as the input and the scenario exposure rate level as the judgment result. The scenario exposure rate judgment matrix is shown in Table 1:

[0066] Table 1 Scenario Exposure Rate Judgment Matrix

[0067] E1 p≤0.05％ E2 0.05＜p≤1％ E3 (1％<p<10％) E4 (p>10％)

[0068] Determine the scenario exposure rate level of the overall logical scenario through the scenario exposure rate judgment matrix according to the occurrence frequency of the trigger source in the natural driving data;

[0069] Construct a scenario severity judgment matrix with the severity of the accident as the input and the scenario severity level as the judgment result. The scenario severity judgment matrix is shown in Table 2:

[0070] Table 2 Scenario Severity Judgment Matrix

[0071] S0 No injury S1 Mild or moderate injury S2 Serious but non-life-threatening injury S3 Life-threatening injury

[0072] Determine the scenario severity level of the overall logical scenario through the scenario severity judgment matrix according to the severity of the accident in the natural driving data;

[0073] Construct a scenario risk level judgment matrix with the scenario exposure rate level and the scenario severity level as the input and the scenario risk level as the output. The scenario risk level judgment matrix is shown in Table 3:

[0074] Table 3 Scenario Risk Level Judgment Matrix

[0075]

[0076] Determine the scenario risk level of the overall logical scenario through the scenario risk level judgment matrix according to the scenario exposure rate level and the scenario severity level;

[0077] Select the overall logical scenario with the highest scenario risk level as the specific test scenario.

[0078] The autonomous driving functions include multiple autonomous driving functions such as HWP, AVP, TJP, and NOP. The designed operating domain includes static entities, dynamic entities, and vehicle functions. In terms of static entities, it is required that the perception system can operate normally under defined road conditions, weather conditions, and lighting conditions, etc. In terms of dynamic entities, it is required that the perception system correctly identifies and tracks various traffic participants. In terms of vehicle functions, it is required that the perception system operates normally under the condition that the activated vehicle speed and various sensors are error-free;

[0079] The functional scenarios include sub - functions of the autonomous driving function and their operating conditions;

[0080] The performance requirements of the perception system specify the type, quantity, and specific implementation functions of sensors. The performance requirements of the perception system ensure that the autonomous driving function can achieve its expected behavior in its operating scenarios. The performance requirements of the perception system include the accuracy, stability, timeliness, comprehensiveness, tracking accuracy, and recognition parameter accuracy of identifying static and dynamic entities.

[0081] The evaluation system is hierarchical, with each layer including several evaluation indicators. Except for the lowest layer, the evaluation indicators of each layer are obtained by weighted calculation of the secondary indicators subordinate to it. Step 4) includes:

[0082] According to the operation results, the quantitative evaluation results of the perception system of the autonomous vehicle are obtained through weighted calculation by the evaluation system.

[0083] The highest layer of the evaluation system includes the quantitative evaluation indicators of sensor performance and the quantitative evaluation indicators of perception system performance;

[0084] The secondary indicators subordinate to the quantitative evaluation indicators of sensor performance include the reliability, cost, and stability of sensors; the secondary indicators subordinate to the quantitative evaluation indicators of perception system performance include the recognition rate, accuracy, and robustness of the perception system in specific scenarios; the secondary indicators subordinate to the reliability of sensors include the failure rate, robustness, and average service life of sensors; the secondary indicators subordinate to the cost of sensors include the number of sensors and the unit price of sensors; the secondary indicators subordinate to the stability of sensors include the signal - to - noise ratio, electromagnetic interference stability, and operating temperature range of sensors.

[0085] The recognition rate is the proportion of the perception system recognizing the operating scenario. The secondary indicators subordinate to the recognition rate of the perception system include the missed detection rate of the perception system; the secondary indicators subordinate to the accuracy of the perception system include the false detection rate and detection accuracy of the perception system; the secondary indicators subordinate to the robustness of the perception system include the reporting delay, target following characteristics, and maximum effective detection distance of the perception system. The target following characteristics include multi - target tracking accuracy and multi - target tracking accuracy.

[0086] Specific test scenarios can be reproduced on virtual simulation test tools, hardware - in - the - loop test tools, and whole - vehicle closed test sites to conduct tests and verify whether the perception system of the autonomous driving function meets its performance requirements.

[0087] The evaluation method proposed in this embodiment is used to evaluate the Navigate on Pilot (NOP) function (hereinafter referred to as the NOP function) of the equipment on an SAE L3-level autonomous vehicle. The vehicle is equipped with a lidar, four vision sensors, a set of Global Positioning System (GPS), and a high-precision map. The lidar is used to detect obstacles, motor vehicles, etc. The vision sensors are used to identify traffic signs, lane lines, etc. The GPS and high-precision map provide the vehicle's own positioning information. When the lane lines are clear, the vehicle will drive along the lane lines. If there are no lane lines or the lane lines are not clear, the vehicle will drive according to the path planned by the GPS and high-precision map. If there is a vehicle in front, it will drive at a set distance from the vehicle in front, and decide whether to change lanes and overtake according to whether the speed of the vehicle in front is lower than the expected speed for a long time.

[0088] The designed operating domain (ODD) of this autonomous vehicle is the highway section. The sub-functions and operating environment of the NOP function are defined as shown in Table 4:

[0089] Table 4 Definition Table of NOP Function Sub-functions

[0090]

[0091]

[0092] Taking the deceleration and obstacle avoidance sub-function as an example to describe the function scenario of NOP, including

[0093] Function Scenario 1: Within the speed range specified for the NOP function to be enabled, the vehicle is in the leftmost lane, a vehicle in front cuts in and the time to collision (TTC) with the vehicle in front < T0, and the vehicle outputs the corresponding deceleration to decelerate and avoid.

[0094] Function Scenario 2: Within the speed range specified for the NOP function to be enabled, the vehicle is in the leftmost lane, the vehicle in front decelerates suddenly and the time to collision (TTC) with the vehicle in front < T0, and the vehicle outputs the corresponding deceleration to decelerate and avoid.

[0095] In addition, during the development and design of the autonomous driving function and the definition of the operating domain, performance requirements for the perception system will be proposed. For example, it is required that the lidar sensor correctly identify obstacles, vehicles in front, and track the vehicle in front under limited conditions (medium rain, medium snow, and within limited light conditions), and the camera correctly identify lane lines and traffic signs within the ODD to ensure that the autonomous vehicle achieves the expected behavior in its deceleration and obstacle avoidance function scenario.

[0096] For the deceleration and obstacle avoidance of the NOP function, it mainly relies on the lidar sensor and the vision sensor. The evaluation system is shown in Table 5:

[0097] Table 5 Quantification Evaluation Index Table of Perception System Performance

[0098]

[0099]

[0100] According to the quantification evaluation index table of the perception system performance, the final quantification model of the perception system can be obtained. The calculation formula of the quantification model is as follows:

[0101] AS = I·ω

[0102] I = [P 1 , P 2 , …, P n1 , C 1 , C 2 , …, C n2

[0103] ω = [ω 1 , …, ω n1 , ω n1+1 , … ω n1+n2

[0104] Among them, AS is the quantification evaluation result of the perception system of the autonomous driving vehicle, I is the evaluation system, [P 1 , P 2 , …, P n1 is the bottom - layer index belonging to the quantification evaluation index at the system component level, [C 1 , C 2 , …, C n2 is the bottom - layer index belonging to the perception system performance evaluation index. n1 and n2 are respectively the quantities of the bottom - layer indexes belonging to the quantification evaluation index at the system component level and the perception system performance evaluation index. The higher the quantification evaluation result, the better the performance of the test result. The quantification evaluation result takes values from 1 to 10, and ω is the weight of each evaluation index, taking values from 0 to 1.

[0105] Starting from the physical structure and working principle of the lidar and vision sensors, as Figure 2 shown, a trigger condition knowledge base for the NOP function is formed, a trigger condition analysis matrix of the perception system is constructed, the trigger source elements constituting the trigger conditions of the perception system are identified, and based on the natural driving data, the parameters of the trigger source elements and the parameters in the functional scenarios are statistically analyzed. Combining with the designed operation domain of the autonomous driving function, the parameter ranges of the trigger source elements and the parameter ranges of the logical scenarios are determined. The generated partial trigger conditions and parameter ranges are shown in Table 6:

[0106] Table 6 Trigger Condition Knowledge Base

[0107]

[0108] ​​The remaining parameters in the logical scenario can be determined according to the designed operating domain and are not listed in the table.

[0109] The logical scenario combined with the NOP function and the triggering conditions forms an overall logic. The overall logical scenario is risk-assessed from three aspects: severity, exposure rate, and confounding degree. The scenario parameter combinations with high risks are selected as specific test scenarios. Therefore, the scenario parameters of high rainfall intensity, low visibility, low light intensity, and low collision time are selected to construct specific test scenarios.

[0110] The specific test in this embodiment is constructed on a closed vehicle test site. First, it is judged whether the vehicle can meet the set performance requirements and whether the vehicle can achieve the expected functions according to the experimental results. If the performance requirements cannot be met, the perception system needs to be iterated again. In this embodiment, when triggered by rainfall, the vehicle fails to decelerate and avoid the vehicle cutting in from the front reasonably and collides with the vehicle in front, so the perception system needs to be iterated. Under the other two triggering conditions, since the perception system meets the performance requirements, the quantitative evaluation results of the perception system performance of this autonomous driving function are output according to the test results.

[0111] First, calculate the quantitative evaluation results of the perception system performance according to the test results. The quantitative evaluation results of the perception system performance under foggy conditions are:

[0112] AS fog = I·ω = I f ·ω = 7.086

[0113] The quantitative evaluation results of the perception system performance under low light conditions are:

[0114] AS fog = I·ω = I f ·ω = 6.839

[0115] Based on the test results, improvement measures are proposed for the autonomous driving perception system. The evaluation results show that the lidar in this embodiment has relatively low scores in terms of detection accuracy, missed detection rate, and target following characteristics. Therefore, a lidar with a higher number of beams can be adopted, the internal algorithm of the lidar can be optimized, or the configuration structure of the perception system can be optimized. The improved autonomous driving function perception system is retested, and it is verified through experiments whether the results of the new perception system meet its performance requirements.

[0116] This embodiment proposes a method for quantitatively evaluating the performance of an autonomous driving vehicle perception system, which can systematically evaluate and test the perception system that relies on complex sensors and algorithms, identify and iterate the perception system, improve the accuracy, reliability, and certainty of the autonomous driving perception system in complex dynamic traffic scenarios, meet the safety requirements of the expected functions of autonomous driving, meet the evaluation requirements in the development process of the perception system, provide guidance for the development of the autonomous driving vehicle perception system for expected functional safety, and has strong operability and is easy to operate.

[0117] The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and variations based on the concept of the present invention without creative efforts. Therefore, all technical solutions that can be obtained by those skilled in the art in this technical field based on the concept of the present invention through logical analysis, reasoning, or limited experiments on the basis of the prior art shall fall within the protection scope determined by the claims.

Claims

1. A method for quantitatively evaluating the performance of an autonomous vehicle perception system, characterized in that, it includes the following steps: 1) According to the autonomous driving function and the designed operating domain, obtain the functional scenarios and the performance requirements of the perception system; 2) Generate specific test scenarios for the autonomous vehicle perception system according to the functional scenarios; 3) Run the autonomous vehicle perception system in the specific test scenarios, and judge whether the running results meet the performance requirements of the perception system. If so, execute step 4); otherwise, iterate the autonomous vehicle perception system and execute step 3); 4) According to the running results, calculate the quantitative evaluation results of the autonomous vehicle perception system through the evaluation system, The step 2) includes the following steps: 201) Obtain the triggering conditions of the autonomous vehicle perception system; 202) Obtain the parameter ranges of the triggering conditions; 203) Generate specific test scenarios according to the functional scenarios, triggering conditions and the parameter ranges of the triggering conditions, The step 201) includes: Construct a triggering source knowledge base, a triggering mechanism knowledge base and a triggering effect knowledge base. The triggering source knowledge base stores the triggering source elements affecting the perception system in a tree structure. The triggering mechanism knowledge base and the triggering effect knowledge base are established based on semantic guiding words; Construct a triggering condition analysis matrix through the triggering source knowledge base, the triggering mechanism knowledge base and the triggering effect knowledge base; Obtain the triggering conditions through the triggering condition analysis matrix, The step 202) includes: Obtain the parameter ranges of the triggering conditions according to the natural driving data, The step 203) includes: Obtain the parameter ranges of the functional scenarios according to the natural driving data. Within the parameter ranges of the functional scenarios and the triggering conditions, combine the functional scenarios and triggering conditions with specific parameters to generate several overall logical scenarios; Construct a scenario exposure rate judgment matrix with the occurrence frequency of the triggering source as the input and the scenario exposure rate level as the judgment result. According to the occurrence frequency of the triggering source in the natural driving data, determine the scenario exposure rate level of the overall logical scenario through the scenario exposure rate judgment matrix; Construct a scenario severity judgment matrix with the severity of the accident as the input and the scenario severity level as the judgment result. According to the severity of the accident in the natural driving data, determine the scenario severity level of the overall logical scenario through the scenario severity judgment matrix; Construct a scenario risk level judgment matrix with the scenario exposure rate level and the scenario severity level as the input and the scenario risk level as the output. According to the scenario exposure rate level and the scenario severity level, determine the scenario risk level of the overall logical scenario through the scenario risk level judgment matrix; Select the overall logical scenario with the highest scenario risk level as the specific test scenario.

2. The method for quantitatively evaluating the performance of an autonomous vehicle perception system according to claim 1, characterized in that, the functional scenarios include the sub-functions of the autonomous driving function and their operating conditions, and the performance requirements of the perception system include the recognition accuracy of static and dynamic entities, stability, timeliness, comprehensiveness, tracking accuracy and recognition parameter accuracy.

3. The method for quantitatively evaluating the performance of an autonomous vehicle perception system according to claim 1, characterized in that, The evaluation system described is a hierarchical structure, with each layer including several evaluation indicators. Except for the lowest layer, the evaluation indicators of each layer are obtained by weighted calculation of the secondary indicators subordinate to it. The said step 4) includes: According to the operation results, the quantitative evaluation result of the perception system of the autonomous vehicle is obtained through weighted calculation of the evaluation indicators.

4. A method for quantitatively evaluating the performance of a perception system of an autonomous vehicle according to claim 3, characterized in that the highest layer of the evaluation system described includes the quantitative evaluation indicators of sensor performance and the quantitative evaluation indicators of perception system performance; The secondary indicators subordinate to the quantitative evaluation indicators of sensor performance include the reliability, cost, and stability of the sensor; The secondary indicators subordinate to the quantitative evaluation indicators of perception system performance include the recognition rate, accuracy, and robustness of the perception system under specific scenarios.

5. A method for quantitatively evaluating the performance of a perception system of an autonomous vehicle according to claim 4, characterized in that The secondary indicators subordinate to the reliability of the sensor include the failure rate, robustness, and average service life of the sensor; The secondary indicators subordinate to the cost of the sensor include the number of sensors and the unit price of the sensors; The secondary indicators subordinate to the stability of the sensor include the signal-to-noise ratio, electromagnetic interference stability, and operating temperature range of the sensor.

6. A method for quantitatively evaluating the performance of a perception system of an autonomous vehicle according to claim 4, characterized in that The secondary indicators subordinate to the recognition rate of the perception system include the missed detection rate of the perception system; The secondary indicators subordinate to the accuracy of the perception system include the false detection rate and detection accuracy of the perception system; The secondary indicators subordinate to the robustness of the perception system include the reporting delay, target following characteristics, and maximum effective detection distance of the perception system.

Images