A method and device for evaluating automatic driving based on safety evaluation model

Abstract

The present invention discloses an automatic driving evaluation method and device based on a safety evaluation model, the scheme comprising: obtaining a test scenario library for performing safety evaluation on an automatic driving vehicle under test, screening out a subset of dangerous test scenarios, identifying dangerous behaviors therein to obtain real risk points and potential risk points, selecting a corresponding safety evaluation model, and then using driving data of mature drivers to calibrate unknown parameters in the safety evaluation model to obtain a calibrated safety evaluation model; by calculating the braking response of the calibrated safety evaluation model in the dangerous test scenario in the subset of dangerous test scenarios, obtaining a safety parameter space and a safety threshold for the automatic driving vehicle under test to avoid or mitigate collisions in the dangerous test scenario; for the identified emergency dangerous scenario, selecting a corresponding test method to carry out specific testing of the automatic driving vehicle, comparing the test result with the safety threshold, and determining whether the safety threshold requirement is met.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to an autonomous driving evaluation method and device based on a safety evaluation model. Background Art

[0002] Autonomous driving technology is currently in a critical stage of transitioning from R&D testing to industrial application. Because it faces challenges such as complex systems and application scenarios, and the black box problem of artificial intelligence, traditional testing methods are unable to meet its testing and verification needs.

[0003] The "multi-pillar" test and evaluation method proposed by the United Nations World Forum for Harmonization of Vehicle Regulations (WP.29) has now been widely recognized by the international community. The safety model can set safety thresholds in situations where collisions need to be avoided and in situations where collisions cannot be avoided but need to be mitigated. It is crucial for the test and evaluation of autonomous driving in key dangerous scenarios. The "autonomous driving safety model" is a mathematical model with clear definitions, quantitative indicators, and interpretable rules. It can be used to define safety distances, road rights, traffic rules, risk assessments, performance boundaries, and other contents related to autonomous driving safety, so as to support the comprehensive use of simulation tests, closed-field tests, and other methods to carry out safety evaluation of intelligent connected vehicle products. At present, autonomous driving safety evaluation models have problems such as uneven quality, insufficient scenario coverage, and inconsistent construction standards. In particular, there is a lack of a unified method framework for autonomous driving safety evaluation. Summary of the invention

[0004] The present invention provides an automatic driving evaluation method and device based on a safety evaluation model, so as to overcome at least one technical problem existing in the prior art.

[0005] According to a first aspect of an embodiment of the present invention, there is provided an autonomous driving evaluation method based on a safety evaluation model, comprising:

[0006] Obtaining a test scenario library for performing safety assessment on the autonomous driving vehicle under test, and filtering out a subset of dangerous test scenarios from the test scenario library;

[0007] Identifying dangerous behaviors in the subset of dangerous test scenarios to obtain real risk points and potential risk points, selecting corresponding safety evaluation models based on the real risk points and the potential risk points, and then calibrating unknown parameters in the safety evaluation model using driving data of mature drivers to obtain a calibrated safety evaluation model;

[0008] By calculating the braking response of the calibrated safety evaluation model in the dangerous test scenario in the dangerous test scenario subset, a safety parameter space and a safety threshold for the tested autonomous driving vehicle to avoid or mitigate a collision in the dangerous test scenario are obtained;

[0009] For the identified subset of dangerous test scenarios, select corresponding testing methods to carry out specific tests on the autonomous driving vehicle under test, obtain test results, and compare the test results with the safety threshold to determine whether the autonomous driving capability of the autonomous driving vehicle under test meets the requirements of the safety threshold.

[0010] Preferably, the acquisition of a test scenario library for safety assessment of the autonomous driving vehicle under test specifically includes: sorting out typical behavior combinations of the autonomous driving vehicle under test and the target object according to the functions and design operating conditions of the autonomous driving vehicle under test, determining the scenario parameter space, and after performing a rationality analysis on the behavior combinations and the parameter space, obtaining the test scenario library from a pre-established benchmark test scenario library.

[0011] Preferably, before identifying the dangerous behaviors in the dangerous test scenario subset to obtain the real risk points and potential risk points, the method includes: analyzing the coverage of the test scenario set, specifically including three aspects: functional scenario category, logical scenario category and specific scenario parameter coverage:

[0012] 1) Functional scenario category coverage: Traverse the main vehicle behavior, target object behavior and other ancillary information, and eliminate unreasonable functional scenarios to obtain a complete set of reasonable functional scenarios. The number of functional scenario categories involved is C f ; The number of functional scenario categories screened is c f , then the coverage of the functional scenario category is defined as:

[0013] ;

[0014] 2) Logical scenario category coverage: For each type of functional scenario, traverse the position relationship between the main vehicle and the target object, determine the logical scenario category under each type of functional scenario, and eliminate unreasonable logical scenarios to obtain a complete set of reasonable logical scenarios. The number of logical scenario categories involved is C l ; The number of logical scene categories obtained by screening is c l , then the coverage of the logical scenario category is defined as:

[0015] ;

[0016] 3) Specific scenario parameter coverage: For each type of logical scenario, analyze its reasonable and foreseeable parameter space [P1, P2, P3, …, P n ], where P1, P2, P 3、 …、P n For all scene parameters P1, P2, P3, ..., P nThe parameter range of the specific scene is [p1, p2, p3, …, p n ], where p1, p2, p 3、 …、p n For all scene parameters p1, p2, p3, ..., p n If not involved, it is set to 0; then the parameter coverage of the specific scene is defined as:

[0017] ;

[0018] Comprehensively consider the impact of the three dimensions and give each dimension different weights to obtain the coverage of the test scenario set for:

[0019] ;

[0020] Among them, the symbol Indicates the first weighting coefficient whose value is greater than 0, symbol Indicates the second weighting coefficient whose value is greater than 0, symbol represents a third weighting coefficient whose value is greater than 0, and + + =1.

[0021] Preferably, the step of identifying the dangerous behaviors in the dangerous test scenario subset to obtain real risk points and potential risk points specifically includes:

[0022] The autonomous driving safety model is divided into a conflict avoidance model and an accident avoidance model according to whether the preventive operation capability is considered, and the accident avoidance model is divided into an emergency braking model and an emergency steering model according to standard operation actions;

[0023] Firstly, based on whether there is a conflict with foreshadowing, the scenarios in the subset of dangerous test scenarios are divided into conflict scenarios with foreshadowing and conflict scenarios without foreshadowing;

[0024] For the unforeseeable conflict scenario, an accident avoidance model is selected, and based on the possible behavioral intentions of the target object, potential risks are identified and the time point when the risk occurs is determined; if the risk occurs at a low speed, an emergency braking model is selected; if the risk occurs at a high speed and there is enough lateral avoidance space, an emergency steering model is selected, otherwise the emergency braking model is selected;

[0025] For the predicted conflict scenario, the conflict avoidance model is selected, and based on the potential risks that occur, the time point when the risk occurs is determined.

[0026] According to a second aspect of an embodiment of the present invention, there is provided an automatic driving evaluation device based on a safety evaluation model, comprising:

[0027] A dangerous test scenario subset acquisition module, used to acquire a test scenario library for safety evaluation of the autonomous driving vehicle under test, and to filter out a dangerous test scenario subset from the test scenario library;

[0028] A safety evaluation model calibration module, used to identify dangerous behaviors in the subset of dangerous test scenarios to obtain real risk points and potential risk points, select corresponding safety evaluation models based on the real risk points and the potential risk points, and then calibrate unknown parameters in the safety evaluation model using driving data of mature drivers to obtain a calibrated safety evaluation model;

[0029] A parameter acquisition module, configured to obtain a safety parameter space and a safety threshold for avoiding or mitigating a collision of the autonomous driving vehicle under test in the dangerous test scenario by calculating a braking response of the calibrated safety evaluation model in the dangerous test scenario subset;

[0030] The safety verification module is used to select corresponding testing methods for the identified subset of dangerous test scenarios to carry out specific tests on the autonomous driving vehicle under test, obtain test results, compare the test results with the safety threshold, and determine whether the autonomous driving capability of the autonomous driving vehicle under test meets the requirements of the safety threshold.

[0031] An embodiment of this specification can at least achieve the following beneficial effects: The technical solution of this application proposes a method for serving the safety assessment of autonomous driving based on key dangerous scenarios and using the safety assessment model as the passing criterion. By calculating the braking response of the calibrated safety assessment model in the dangerous test scenario subset, the safety parameter space of the tested autonomous driving vehicle and the safety threshold for avoiding collision or mitigating collision of the tested autonomous driving vehicle in the dangerous test scenario are obtained. For the identified dangerous test scenario subset, the corresponding test means are selected to carry out specific tests on the tested autonomous driving vehicle, and the test results are obtained. The test results are compared with the safety threshold to determine whether the autonomous driving capability of the tested autonomous driving vehicle meets the requirements of the safety threshold. At the same time, the technical solution of this application analyzes the coverage of the test scenario set from three dimensions: functional scenario category, logical scenario category and specific scenario parameters. These different dimensions play an important role in the safety assessment of the tested autonomous driving vehicle, which is conducive to discovering potential problems, ensuring the comprehensiveness of the test, and discovering the blind spots and deficiencies of the test cases. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 A flowchart of an autonomous driving evaluation method based on a safety evaluation model provided in an embodiment of this specification;

[0034] Figure 2 A flowchart of an autonomous driving vehicle under test avoiding danger in a dangerous test scenario in an autonomous driving evaluation method based on a safety evaluation model provided in an embodiment of this specification;

[0035] Figure 3 A schematic diagram of the composition of parameters of a safety model obtained by a tested autonomous driving vehicle in a dangerous test scenario in an autonomous driving evaluation method based on a safety evaluation model provided in an embodiment of this specification;

[0036] Figure 4 The embodiments of this specification provide corresponding to Figure 1 A structural schematic diagram of an autonomous driving evaluation device based on a safety evaluation model. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of one or more embodiments of this specification clearer, the technical solutions of one or more embodiments of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of one or more embodiments of this specification.

[0038] It should be understood that although the terms first, second, third, etc. may be used in this application document to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other.

[0039] Next, an autonomous driving evaluation method based on a safety evaluation model provided in an embodiment of the specification will be described in detail with reference to the accompanying drawings.

[0040] like Figure 1 As shown, an automatic driving evaluation method based on a safety evaluation model provided by an embodiment of the present invention may include the following steps:

[0041] Step 102: Obtain a test scenario library for performing safety assessment on the autonomous driving vehicle under test, and filter out a subset of dangerous test scenarios from the test scenario library.

[0042] In this step, based on the functions and operational design conditions (ODC) of the autonomous vehicle, typical behavior combinations of the host vehicle and the target object can be sorted out, the scenario parameter space can be determined, and after a rationality analysis of the behavior combination and parameter space (such as eliminating unreasonable behaviors and parameter combinations), the corresponding test scenario set can be screened out from the benchmark test scenario library. This test scenario set can be used as the basis for subsequent safety testing of the autonomous vehicle under test.

[0043] Step 104: Identify dangerous behaviors in the subset of dangerous test scenarios to obtain real risk points and potential risk points, select corresponding safety evaluation models based on the real risk points and the potential risk points, and then use mature driver driving data to calibrate unknown parameters in the safety evaluation model to obtain a calibrated safety evaluation model.

[0044] Step 106: By calculating the braking response of the calibrated safety evaluation model in the dangerous test scenario in the dangerous test scenario subset, the safety parameter space and the safety threshold of the tested autonomous driving vehicle to avoid or mitigate collision in the dangerous test scenario are obtained.

[0045] The safety parameter space and safety threshold calculated in this step can be used to determine the automatic driving capability of the tested automatic driving vehicle in the subsequent steps. The following takes a specific "real risk point" or "potential risk point" as an example to explain the specific calculation steps for calculating the safety parameter space and safety threshold in this step. For example, in the real risk point scenario of the front vehicle cutting in urgently, in order to ensure driving safety, the emergency braking model is selected as a safety evaluation model. According to the safety evaluation model, the emergency braking of the main vehicle can be disassembled into three stages: free driving stage (before the risk point appears), decision response stage (after the risk point appears, before braking), and braking effectiveness stage (after braking). The parameters of the safety evaluation model are calibrated through driving simulation experiments or real vehicle experiments to obtain a safety evaluation model with parameter determination. By calculating the braking response of the safety evaluation model in the emergency cutting-in scenario, the safety parameter space and the safety threshold that the tested vehicle should avoid or mitigate collision are finally obtained.

[0046] Step 108: For the identified subset of dangerous test scenarios, select corresponding testing methods to carry out specific tests on the autonomous driving vehicle under test, obtain test results, and compare the test results with the safety threshold to determine whether the autonomous driving capability of the autonomous driving vehicle under test meets the requirements of the safety threshold.

[0047] In an optional embodiment, the method of obtaining a test scenario library for safety assessment of the autonomous driving vehicle under test specifically includes: sorting out typical behavior combinations of the autonomous driving vehicle under test and the target object according to the functions and design operating conditions of the autonomous driving vehicle under test, determining the scenario parameter space, and performing a rationality analysis on the behavior combinations and parameter space, and then selecting the test scenario library from a pre-established benchmark test scenario library.

[0048] In an optional embodiment, the identification of dangerous behaviors in the dangerous test scenario subset to obtain real risk points and potential risk points includes: analyzing the coverage of the test scenario set, specifically including three aspects: functional scenario category, logical scenario category and specific scenario parameter coverage:

[0049] 1) Functional scenario category coverage: Traverse the main vehicle behavior, target object behavior and other ancillary information, and eliminate unreasonable functional scenarios to obtain a complete set of reasonable functional scenarios. The number of functional scenario categories involved is C f ; The number of functional scenario categories screened is c f , then the coverage of the functional scenario category is defined as:

[0050] ;

[0051] 2) Logical scenario category coverage: For each type of functional scenario, traverse the position relationship between the main vehicle and the target object, determine the logical scenario category under each type of functional scenario, and eliminate unreasonable logical scenarios to obtain a complete set of reasonable logical scenarios. The number of logical scenario categories involved is C l ; The number of logical scene categories obtained by screening is c l , then the coverage of the logical scenario category is defined as:

[0052] ;

[0053] 3) Specific scenario parameter coverage: For each type of logical scenario, analyze its reasonable and foreseeable parameter space [P1, P2, P3, …, P n ], where P1, P2, P 3、 …、P n For all scene parameters P1, P2, P3, ..., P n The parameter range of the specific scene is [p1, p2, p3, …, p n ], where p1, p2, p 3、 …、p n For all scene parameters p1, p2, p3, ..., p nIf not involved, it is set to 0; then the parameter coverage of the specific scene is defined as:

[0054] ;

[0055] Comprehensively consider the impact of the three dimensions and give each dimension different weights to obtain the coverage of the test scenario set for:

[0056] ;

[0057] Among them, the symbol Indicates the first weighting coefficient whose value is greater than 0, symbol Indicates the second weighting coefficient whose value is greater than 0, symbol represents a third weighting coefficient whose value is greater than 0, and + + =1.

[0058] In the technical solution of this embodiment, the coverage of the test scenario set is analyzed from three dimensions: functional scenario category, logical scenario category and specific scenario parameters. These different dimensions play an important role in the safety assessment of the autonomous driving vehicle under test, which is conducive to discovering potential problems, ensuring the comprehensiveness of the test, and discovering the blind spots and shortcomings of the test cases.

[0059] In an optional embodiment, the driver's behavior actions are first decomposed and simplified to form a basic model, and the decomposed and simplified behavior actions are sorted out to form parameters for constructing a safety model to form a basic model.

[0060] In an optional embodiment, the identifying of dangerous behaviors in the dangerous test scenario subset to obtain real risk points and potential risk points may specifically include:

[0061] The autonomous driving safety model can be divided into conflict avoidance model and accident avoidance model according to whether it has the ability to consider preventive operations. The accident avoidance model can be divided into emergency braking model and emergency steering model according to its standard operation actions.

[0062] Firstly, based on whether there is a forewarning conflict, the scenarios in the subset of urgent danger scenarios are divided into forewarning conflict scenarios and non-forewarning conflict scenarios.

[0063] For the unforeseen conflict scenario, the accident avoidance model is selected, and based on the possible behavioral intentions of the target object, potential risks are identified and the time when the risk occurs is determined; if the risk occurs at a low speed, the emergency braking model is selected; if the risk occurs at a high speed and there is enough lateral avoidance space, the emergency steering model is selected, otherwise the emergency braking model is selected;

[0064] For scenarios with foreshadowed conflicts, a conflict avoidance model is selected, and based on the potential risks that arise, the time point when the risk will occur is determined.

[0065] (1) Determine the spatial layout of the scene: Based on the autonomous driving function and its designed operating conditions, identify the combination of the main vehicle behavior and the target object behavior, and combine it with other auxiliary information such as the road and environment to form a reasonable spatial layout of the test scene.

[0066] (2) Screening the test scene set: Compare the combined scene space layout with the benchmark scene library and screen the test scene set from the benchmark scene library.

[0067] (3) Analysis of test scenario coverage:

[0068] Specifically, it includes three aspects: functional scenario category, logical scenario category and specific scenario parameter coverage.

[0069] 1) Functional scenario category coverage: Traverse the main vehicle behavior, target object behavior and other ancillary information, and eliminate unreasonable functional scenarios to obtain a complete set of reasonable functional scenarios. The number of functional scenario categories involved is C f The number of functional scenario categories screened is c f , then the coverage of the functional scenario category is defined as:

[0070] ;

[0071] 2) Logical scenario category coverage: For each type of functional scenario, traverse the position relationship between the main vehicle and the target object, determine the logical scenario category under each type of functional scenario, and eliminate unreasonable logical scenarios to obtain a complete set of reasonable logical scenarios. The number of logical scenario categories involved is C l The number of logical scene categories obtained by screening is c l , then the coverage of the logical scenario category is defined as:

[0072] ;

[0073] 3) Specific scenario parameter coverage: For each type of logical scenario, analyze its reasonable and foreseeable parameter space [P1, P2, P3, …, P n ], where P1, P2, P 3、 …、P n For all scene parameters P1, P2, P3, ..., P n The parameter space of the specific scene obtained by screening [p1, p2, p3, …, p n ], where p1, p2, p 3、 …、p n For all scene parameters p1, p2, p3, ..., pn If not involved, set it to 0. The parameter coverage of the specific scene is defined as:

[0074] ;

[0075] Taking into account the impact of the three dimensions and assigning different weights to each dimension, the coverage of the test scenario set is obtained as follows:

[0076] ;

[0077] Among them, the symbol Indicates the first weighting coefficient whose value is greater than 0, symbol Indicates the second weighting coefficient whose value is greater than 0, symbol represents a third weighting coefficient whose value is greater than 0, and + + =1.

[0078] In the above step 104, it is mentioned that the dangerous behaviors in the subset of dangerous test scenarios are identified to obtain real risk points and potential risk points, and the corresponding safety evaluation model is selected based on the real risk points and the potential risk points. The specific contents of risk point determination and safety model selection are described below:

[0079] The autonomous driving safety model can be divided into conflict avoidance model and accident avoidance model according to whether it has the ability to consider preventive operations, while the accident avoidance model can be divided into emergency braking model and emergency steering model according to its standard operating actions.

[0080] First, based on whether it is a conflict with warning signs, the scenarios in the emergency danger scenario subset are divided into conflict scenarios with warning signs (such as the scenario where the oncoming vehicle turns on the turn signal in advance) and conflict scenarios without warning signs (such as the oncoming vehicle only has the intention to cut in).

[0081] For the unforeseen conflict scenario, the accident avoidance model is selected, and based on the possible behavioral intentions of the target object, potential risks are identified and the time when the risk occurs is determined. If the risk occurs at a low speed, the emergency braking model is selected; if the risk occurs at a high speed and there is enough lateral avoidance space, the emergency steering model is selected, otherwise the emergency braking model is selected.

[0082] For scenarios with foreshadowed conflicts, the conflict avoidance model is selected, and based on the potential risks that arise, the time point of risk occurrence is determined. For specific risks that arise, if there is enough lateral operating space, the lateral distance is increased to avoid the risk; otherwise, the vehicle speed is reduced to a reasonable level to stay away from the risk.

[0083] At the same time, the calibration of the unknown parameters in the safety evaluation model is mentioned in step 104 above. The specific content of the calibration method is explained below: First, the driver's behavior is decomposed and simplified to form a basic model. For example, for the emergency braking model, the driver's emergency braking action can be decomposed and simplified into three stages: free driving stage (before the risk point appears), decision response stage (after the risk point appears, before braking), and braking effectiveness stage (after braking). Then, for the decomposed and simplified behavior, the parameters used to build the safety model are sorted out to form a basic model. Finally, the driving data of mature drivers is used to calibrate the unknown parameters in the basic model. For example, for the decomposed and simplified emergency braking action, the model parameters such as the time of risk occurrence, braking response time, and maximum braking deceleration are sorted out.

[0084] Specifically, taking the emergency braking model as an example, the driver's emergency braking action can be decomposed and simplified into three stages: free driving stage, decision response stage, and braking effectiveness stage. Figure 3 According to the analysis of braking actions at different stages, the specific parameters of the safety evaluation model are obtained, such as the duration t1 of the decision response stage, the maximum braking deceleration a m , braking force boost time t2, maximum braking time t3, etc. Finally, design an experimental plan for each model parameter, recruit experienced drivers to conduct driving simulation experiments or real vehicle experiments, collect time series data related to the emergency braking model, and use it to calibrate the model parameters. Perform statistical analysis on the experimental results to determine the model parameters.

[0085] The technical solution of this application proposes a method for serving the safety assessment of autonomous driving based on key dangerous scenarios and using safety assessment models as the passing criteria. By calculating the braking response of the calibrated safety assessment model in the dangerous test scenario subset, the safety parameter space of the tested autonomous driving vehicle and the safety threshold for avoiding collisions or mitigating collisions of the tested autonomous driving vehicle in the dangerous test scenario are obtained. For the identified subset of dangerous test scenarios, the corresponding test means are selected to carry out specific tests on the tested autonomous driving vehicle, and the test results are obtained. The test results are compared with the safety threshold to determine whether the autonomous driving capability of the tested autonomous driving vehicle meets the requirements of the safety threshold. At the same time, the technical solution of this application analyzes the coverage of the test scenario set from three dimensions: functional scenario category, logical scenario category, and specific scenario parameters. These different dimensions play an important role in the safety assessment of the tested autonomous driving vehicle, which is conducive to discovering potential problems, ensuring the comprehensiveness of the test, and discovering the blind spots and deficiencies of the test cases.

[0086] Based on the same idea, the embodiments of this specification also provide a device corresponding to the above method. Figure 4The embodiments of this specification provide corresponding to Figure 1 A schematic diagram of the structure of an autonomous driving evaluation device based on a safety evaluation model. Figure 4 As shown, the device may include:

[0087] A dangerous test scenario subset acquisition module 402 is used to acquire a test scenario library for safety evaluation of the autonomous driving vehicle under test, and filter out a dangerous test scenario subset from the test scenario library;

[0088] A safety evaluation model calibration module 404 is used to identify dangerous behaviors in the subset of dangerous test scenarios to obtain real risk points and potential risk points, select corresponding safety evaluation models based on the real risk points and the potential risk points, and then calibrate unknown parameters in the safety evaluation model using driving data of mature drivers to obtain a calibrated safety evaluation model;

[0089] A parameter acquisition module 406 is used to obtain a safety parameter space and a safety threshold for avoiding or mitigating a collision of the autonomous driving vehicle under test in the dangerous test scenario by calculating a braking response of the calibrated safety evaluation model in the dangerous test scenario subset;

[0090] The safety verification module 408 is used to select corresponding testing methods for the identified subset of dangerous test scenarios to carry out specific tests on the autonomous driving vehicle under test, obtain test results, compare the test results with the safety threshold, and determine whether the autonomous driving capability of the autonomous driving vehicle under test meets the requirements of the safety threshold.

[0091] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0092] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An autonomous driving evaluation method based on a safety evaluation model, characterized in that: The method comprises: Obtaining a test scenario library for performing safety assessment on the autonomous driving vehicle under test, and filtering out a subset of dangerous test scenarios from the test scenario library; Identifying dangerous behaviors in the subset of dangerous test scenarios to obtain real risk points and potential risk points, selecting corresponding safety evaluation models based on the real risk points and the potential risk points, and then calibrating unknown parameters in the safety evaluation model using driving data of mature drivers to obtain a calibrated safety evaluation model; By calculating the braking response of the calibrated safety evaluation model in the dangerous test scenario in the dangerous test scenario subset, a safety parameter space of the tested autonomous driving vehicle and a safety threshold of the tested autonomous driving vehicle to avoid or mitigate a collision in the dangerous test scenario are obtained; For the identified subset of dangerous test scenarios, select a corresponding test method to carry out a specific test on the autonomous driving vehicle under test, obtain a test result, and compare the test result with the safety threshold to determine whether the autonomous driving capability of the autonomous driving vehicle under test meets the requirements of the safety threshold; The step of identifying the dangerous behaviors in the dangerous test scenario subset to obtain the real risk points and the potential risk points includes: Analyze the coverage of the test scenario set, including three aspects: functional scenario category, logical scenario category and specific scenario parameter coverage: 1) Functional scenario category coverage: Traverse the main vehicle behavior, target object behavior and other ancillary information, and eliminate unreasonable functional scenarios to obtain a complete set of reasonable functional scenarios. The number of functional scenario categories involved is C f ; The number of functional scenario categories screened is c f , then the coverage of the functional scenario category is defined as: ; 2) Logical scenario category coverage: For each type of functional scenario, traverse the position relationship between the main vehicle and the target object, determine the logical scenario category under each type of functional scenario, and eliminate unreasonable logical scenarios to obtain a complete set of reasonable logical scenarios. The number of logical scenario categories involved is C l ; The number of logical scene categories obtained by screening is c l , then the coverage of the logical scenario category is defined as: ； 3) Specific scenario parameter coverage: For each type of logical scenario, analyze its reasonable and foreseeable parameter space [P1, P2, P3, …, P n ], where P1, P2, P 3、 …、P n For all scene parameters P1, P2, P3, ..., P n The parameter range of the specific scene is [p1, p2, p3, …, p n ], where p1, p2, p 3、 …、p n For all scene parameters p1, p2, p3, ..., p n If not involved, it is set to 0; then the parameter coverage of the specific scene is defined as: ； Comprehensively consider the impact of the three dimensions and give each dimension different weights to obtain the coverage of the test scenario set for: ； Among them, the symbol Indicates the first weighting coefficient whose value is greater than 0, symbol Indicates the second weighting coefficient whose value is greater than 0, symbol represents a third weighting coefficient whose value is greater than 0, and ; The identifying of dangerous behaviors in the dangerous test scenario subset to obtain real risk points and potential risk points specifically includes: The autonomous driving safety model is divided into a conflict avoidance model and an accident avoidance model according to whether the preventive operation capability is considered, and the accident avoidance model is divided into an emergency braking model and an emergency steering model according to standard operation actions; Firstly, based on whether there is a conflict with foreshadowing, the scenarios in the subset of dangerous test scenarios are divided into conflict scenarios with foreshadowing and conflict scenarios without foreshadowing; For the unforewarned collision scenario, an accident avoidance model is selected, and based on the behavioral intention of the target object, potential risks are identified and the time point when the risk occurs is determined; if the risk occurs at a low speed, an emergency braking model is selected; if the risk occurs at a high speed and there is lateral avoidance space, an emergency steering model is selected, otherwise the emergency braking model is selected; For the predicted conflict scenario, the conflict avoidance model is selected, and based on the potential risks that occur, the time point when the risk occurs is determined.

2. The automatic driving evaluation method based on the safety evaluation model according to claim 1, characterized in that: The method of obtaining a test scenario library for safety assessment of the autonomous driving vehicle under test specifically includes: sorting out typical behavior combinations of the autonomous driving vehicle under test and the target object according to the functions and design operating conditions of the autonomous driving vehicle under test, determining the scenario parameter space, and performing a rationality analysis on the behavior combination and the parameter space, and then selecting the test scenario library from a pre-established benchmark test scenario library.

3. An automatic driving evaluation device based on a safety evaluation model, characterized in that: The device comprises: A dangerous test scenario subset acquisition module, used to acquire a test scenario library for safety evaluation of the autonomous driving vehicle under test, and to filter out a dangerous test scenario subset from the test scenario library; A safety assessment model calibration module is used to identify dangerous behaviors in the subset of dangerous test scenarios to obtain real risk points and potential risk points, select corresponding safety assessment models based on the real risk points and the potential risk points, and then use mature driver driving data to calibrate unknown parameters in the safety assessment model to obtain a calibrated safety assessment model; the identification of dangerous behaviors in the subset of dangerous test scenarios to obtain real risk points and potential risk points includes: Analyze the coverage of the test scenario set, including three aspects: functional scenario category, logical scenario category and specific scenario parameter coverage: 1) Functional scenario category coverage: Traverse the main vehicle behavior, target object behavior and other ancillary information, and eliminate unreasonable functional scenarios to obtain a complete set of reasonable functional scenarios. The number of functional scenario categories involved is C f ; The number of functional scenario categories screened is c f , then the coverage of the functional scenario category is defined as: ； 2) Logical scenario category coverage: For each type of functional scenario, traverse the position relationship between the main vehicle and the target object, determine the logical scenario category under each type of functional scenario, and eliminate unreasonable logical scenarios to obtain a complete set of reasonable logical scenarios. The number of logical scenario categories involved is C l ; The number of logical scene categories obtained by screening is c l , then the coverage of the logical scenario category is defined as: ； 3) Specific scenario parameter coverage: For each type of logical scenario, analyze its reasonable and foreseeable parameter space [P1, P2, P3, …, P n ], where P1, P2, P 3、 …、P n For all scene parameters P1, P2, P3, ..., P n The parameter range of the specific scene is [p1, p2, p3, …, p n ], where p1, p2, p 3、 …、p n For all scene parameters p1, p2, p3, ..., p n If not involved, it is set to 0; then the parameter coverage of the specific scene is defined as: ； Comprehensively consider the impact of the three dimensions and give each dimension different weights to obtain the coverage of the test scenario set for: ； Among them, the symbol Indicates the first weighting coefficient whose value is greater than 0, symbol Indicates the second weighting coefficient whose value is greater than 0, symbol represents a third weighting coefficient whose value is greater than 0, and ; The identifying of dangerous behaviors in the dangerous test scenario subset to obtain real risk points and potential risk points specifically includes: The autonomous driving safety model is divided into a conflict avoidance model and an accident avoidance model according to whether the preventive operation capability is considered, and the accident avoidance model is divided into an emergency braking model and an emergency steering model according to standard operation actions; Firstly, based on whether there is a conflict with foreshadowing, the scenarios in the subset of dangerous test scenarios are divided into conflict scenarios with foreshadowing and conflict scenarios without foreshadowing; For the unforewarned collision scenario, an accident avoidance model is selected, and based on the behavioral intention of the target object, potential risks are identified and the time point when the risk occurs is determined; if the risk occurs at a low speed, an emergency braking model is selected; if the risk occurs at a high speed and there is lateral avoidance space, an emergency steering model is selected, otherwise the emergency braking model is selected; For the foreshadowed conflict scenario, the conflict avoidance model is selected, and based on the potential risks that occur, the time point when the risk occurs is determined; A parameter acquisition module, configured to obtain a safety parameter space and a safety threshold for avoiding or mitigating a collision of the autonomous driving vehicle under test in the dangerous test scenario by calculating a braking response of the calibrated safety evaluation model in the dangerous test scenario subset; The safety verification module is used to select corresponding testing methods for the identified subset of dangerous test scenarios to carry out specific tests on the autonomous driving vehicle under test, obtain test results, compare the test results with the safety threshold, and determine whether the autonomous driving capability of the autonomous driving vehicle under test meets the requirements of the safety threshold.

Images