Scenario Validity Analysis Method and Device Based on Autonomous Driving Function Requirements

Abstract

A method and device for scenario effectiveness analysis based on autonomous driving function requirements belong to the technical field of autonomous driving test scenario development, and solve the problem of how to distinguish whether a test scenario has an actual verification effect on autonomous driving functions. During the scenario development process, first analyze all the function requirements that the autonomous driving system should meet, analyze and transform each function requirement into a corresponding typical scenario training set, then use machine learning methods to train a scenario effectiveness classification tool corresponding to the autonomous driving function requirements, and then add a step of effectiveness analysis for individual scenarios during the scenario extraction process, eliminate scenarios with relatively low relevance to specific function verification, and retain strongly relevant scenarios, which not only saves test resources, but also improves the efficiency and accuracy of simulation test work.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving test scenario development, and particularly to a method and device for analyzing the effectiveness of scenarios based on the requirements of autonomous driving functions. Background Art

[0002] During the development process of automotive autonomous driving functions, simulation testing is a very important link, and the development of test scenarios is the top priority. The development of test scenarios generally refers to relevant standards and regulations, specific driving scenarios, and some dangerous and extreme scenarios to create test scenarios. The data sources for scenario development are diverse, including real road acquisition data, feedback data from virtual simulation testing, statistical data from traffic departments or other relevant parties, and the previously mentioned forward-developed scenario data. The final unified format of test scenarios is extremely large in scale. Therefore, if all the obtained scenarios are directly used to detect the reliability of autonomous driving algorithms, it will pose a great challenge to the testing work, requiring a huge investment in both human and material resources. Even for a relatively small testing team with insufficient equipment attributes, it is almost impossible to carry out the testing work. Facing this practical problem, the current mainstream approach in the industry is to adopt ultra-real-time and high-concurrency cloud platform technology to build a virtual testing platform. This is the most direct and effective method to solve this problem and is also the inevitable path to meet the large number of autonomous driving function verification requirements in the future. The cloud platform testing technology aims to centrally deploy the test tool chain on the cloud computing platform, use a large number of virtual hosts to replace real test tools, and through unified supervision and control, achieve ultra-real-time and high-concurrency computing characteristics, improve the efficiency of testing work, and save human and material resources. This method optimizes the testing process from the aspects of equipment and process, thus solving the problem of the large scale of test scenario quantity, but it does not consider whether a large number of test scenarios are all valid for verifying autonomous driving functions, and does not screen test scenarios from the perspective of actual testing requirements.

[0003] Therefore, the defect of the existing technical means is: the lack of a process for screening the effectiveness of individual test scenarios according to the actual autonomous driving function test verification requirements.

[0004] In the prior art, the patent document CN110553853A discloses a "method for testing and evaluating an autonomous driving function based on searching for a relatively poor scenario in a field". The actual test field is used to test the autonomous driving function to be tested, making the test conclusion closer to the real situation; and on the basis of continuously searching for relatively poor test scenarios, the autonomous driving function is tested, and improvement suggestions are directly put forward for the improvement degree of the autonomous driving function. The patent document CN112256590A discloses a "method, device and autonomous driving system for determining the effectiveness of a virtual scenario". According to the analysis results of the matching degree and restoration degree between the virtual simulation scenario and the real scenario, the effectiveness of the virtual scenario is determined, avoiding the interference brought by invalid virtual scenarios to the simulation test, thereby helping to improve the accuracy of the simulation test results in the subsequent simulation test process.

[0005] In summary, the existing technical means aim to solve the problem of the matching degree and accuracy between the simulation scenario and the real scenario, and lack the analysis process of whether the simulation scenario itself has a verification effect on the autonomous driving function. Summary of the Invention

[0006] The present invention solves the problem of how to distinguish whether a test scenario has an actual verification effect on the autonomous driving function.

[0007] The method for analyzing the effectiveness of a scenario based on the requirements of an autonomous driving function according to the present invention includes the following steps:

[0008] Step S1, collect the latest autonomous driving-related materials, and sort out the function requirements, typical dynamic events, and common static scenarios proposed by the autonomous driving system;

[0009] Step S2, after classifying the scenarios according to the function requirements of autonomous driving, form the function scenarios of each type of scenario, and extract the definitions of the scenario parameter types from the OpenScenario standard as the set of scenario parameter types;

[0010] Step S3, according to the function scenarios and scenario parameter types of each type of scenario, convert all the function scenarios into corresponding logical scenarios, and then, according to the actual situation, reasonably generalize all the logical scenarios into corresponding specific test scenarios. So far, a parameterized test scenario library classified according to the function requirements of autonomous driving is obtained;

[0011] Step S4, the parameterized scenario library includes several test scenarios, which are classified by the requirements of the autonomous driving function. Using these multiple test scenarios as a training set, then perform machine learning on each type of scenario respectively, so as to obtain the classifier corresponding to each requirement of the autonomous driving function, and use the classifier as the determination rule for the effectiveness of each type of scenario;

[0012] Step S5: Screen the extracted test scenarios according to the determination rules for the effectiveness of each type of scenario, and update the training set and analyze the problems for the screening results respectively.

[0013] Further, in an embodiment of the present invention, in the step S1, the relevant information on autonomous driving includes autonomous driving technology development, autonomous driving test verification, and standard formulation.

[0014] Further, in an embodiment of the present invention, in the step S2, the functional scenarios for forming various scenarios are obtained by arranging and combining the dynamic events and static scenarios of various scenarios and then combining with manual analysis.

[0015] Further, in an embodiment of the present invention, in the step S4, the machine learning is an Adaboosts algorithm model.

[0016] Further, in an embodiment of the present invention, in the step S5, the screening results are divided into a valid scenario library and an invalid scenario library.

[0017] Further, in an embodiment of the present invention, the valid scenario library is a set of corresponding test scenarios that are strongly related to each autonomous driving function requirement.

[0018] Further, in an embodiment of the present invention, the test scenarios in the valid scenario library are input into the training set as training scenarios to update the classifier.

[0019] Further, in an embodiment of the present invention, the valid scenario library is a valid scenario library that is updated regularly according to the frequency.

[0020] Further, in an embodiment of the present invention, the test scenarios in the invalid scenario library are mainly analyzed for problems, the deficiencies of the classifier are found, the classifier is adjusted, and the invalid scenarios are repaired.

[0021] An electronic device according to the present invention includes a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus;

[0022] The memory is used to store a computer program;

[0023] The processor is used to implement the method steps described in any of the above methods when executing the program stored in the memory.

[0024] The present invention solves the problem of how to distinguish whether a test scenario has an actual verification effect on the autonomous driving function. The specific beneficial effects include:

[0025] 1. The method for analyzing the scenario effectiveness based on the requirements of autonomous driving functions according to the present invention is based on the detailed functional requirements of autonomous driving. By combining the typical dynamic events and static events corresponding to each functional requirement, through classification and combination of dynamic events and static events, a functional scenario library that meets the verification of autonomous driving functions is developed. Then, according to the scenario parameter types proposed by the OpenX series of standards and combined with artificial expert experience, a specific scenario library is reasonably transformed and generalized, and this is used as the training set for machine learning. Using the principle of the Adaboosts algorithm, an identification program for effectively distinguishing the effective test scenarios that play a role in the verification of autonomous driving functions is learned through integrated learning, so as to perform the first round of screening on a large number of test scenarios, exclude invalid scenarios, reduce the test workload, improve the test work efficiency, and improve the accuracy of test results.

[0026] 2. The method for analyzing the scenario effectiveness based on the requirements of autonomous driving functions according to the present invention proposes a new scenario classification method starting from the specific requirements of autonomous driving functions. Through this classification method and combined with machine learning training, it can help screen the test scenarios that are strongly related to each autonomous driving function requirement, exclude the scenarios with relatively low or no relevance, and improve the quality of individual scenarios.

[0027] 3. For the method for analyzing the scenario effectiveness based on the requirements of autonomous driving functions according to the present invention, the training set of the judgment program for scenario effectiveness analysis is not fixed. According to the specified frequency, it can be dynamically updated according to the newly added effective scenarios and the updated functional requirement conditions. In this way, a scenario effectiveness analysis tool that is updated in real time and has a high degree of fit with the verification of autonomous driving functions can be finally obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and / or additional aspects and advantages of the present invention will become apparent and be readily understood from the following description of the embodiments in conjunction with the drawings, where:

[0029] Figure 1 is the flowchart of the method for analyzing the scenario effectiveness based on the requirements of autonomous driving functions described in the specific implementation manner.

[0030] Figure 2 is an example diagram of the method for analyzing the scenario effectiveness based on the requirements of autonomous driving functions described in the specific implementation manner. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] The following will clearly and completely describe various embodiments of the present invention in conjunction with the drawings. The embodiments described by referring to the drawings are exemplary and are intended to explain the present invention, and should not be construed as limiting the present invention.

[0032] The method for analyzing the effectiveness of scenarios based on the requirements of autonomous driving functions described in this embodiment includes the following steps:

[0033] Step S1: Collect the latest autonomous driving-related materials, and sort out the function requirements, typical dynamic events, and common static scenarios proposed by the autonomous driving system.

[0034] Step S2: After classifying the scenarios according to the function requirements of autonomous driving, form the function scenarios of each type of scenario, and extract the definitions of scenario parameter types from the OpenScenario standard as the set of scenario parameter types for the whole.

[0035] Step S3: According to the function scenarios and scenario parameter types of each type of scenario, convert all function scenarios into corresponding logical scenarios, and then, according to the actual situation, reasonably generalize all logical scenarios into corresponding specific test scenarios. Thus, a parameterized test scenario library classified according to the function requirements of autonomous driving is obtained.

[0036] Step S4: The parameterized scenario library includes several test scenarios, which are classified by the function requirements of autonomous driving. Using these multiple test scenarios as the training set, then perform machine learning on each type of scenario separately, so as to obtain the classifier corresponding to each autonomous driving function requirement, and use the classifier as the determination rule for the effectiveness of each type of scenario.

[0037] Step S5: Screen the extracted test scenarios according to the determination rule for the effectiveness of each type of scenario, and perform training set update and problem analysis on the screening results respectively.

[0038] In this embodiment, in step S1, the autonomous driving-related materials include autonomous driving technology development, autonomous driving test verification, and standard formulation.

[0039] In this embodiment, in step S2, the formation of the function scenarios of each type of scenario is obtained through the permutation and combination of the dynamic events and static scenarios of each type of scenario and combined with manual analysis.

[0040] In this embodiment, in step S4, the machine learning is the Adaboosts algorithm model.

[0041] In this embodiment, in step S5, the screening results are divided into an effective scenario library and an ineffective scenario library.

[0042] In this embodiment, the effective scenario library is a set of corresponding test scenarios that are strongly related to each autonomous driving function requirement.

[0043] In this embodiment, the test scenarios in the effective scenario library are used as training scenarios and put into the training set to update the classifier.

[0044] In this implementation manner, the effective scene library is an effective scene library that is regularly updated according to a frequency.

[0045] In this implementation, the test scenarios in the invalid scenario library are analyzed in detail to find out the deficiencies of the classifier, and the classifier is adjusted to repair the invalid scenarios.

[0046] An electronic device described in this embodiment includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0047] Memory, used to store computer programs;

[0048] The processor is used to implement the method steps described in the above embodiment when executing the program stored in the memory.

[0049] This embodiment is based on the scenario effectiveness analysis method based on the autonomous driving function requirements of the present invention, referring to Figure 1 To better understand this implementation, a practical implementation is provided in combination with specific objects:

[0050] It is divided into five parts: information collection and organization, scenario combination classification, scenario transformation generalization, algorithm model training, problem analysis and training set update.

[0051] like Figure 2 As shown, the overall technical route is:

[0052] The first step is to collect and organize information. Collect the latest domestic and foreign literature on autonomous driving technology development, test verification, standard setting, etc., sort out all the functional requirements, typical dynamic events, and common static scenarios currently proposed for autonomous driving systems; at the same time, interpret the latest OpenX series standards, especially the OpenScenario part.

[0053] The second step is to classify and combine scenarios. Scenarios are classified according to the functional requirements of autonomous driving. Dynamic events and static scenarios under this category are arranged and combined, and then combined with manual experience analysis to form functional scenarios under various scenario categories. This step also has the task of interpreting standards from the previous step. From the latest OpenScenario and other series of standards, the definition of scenario parameter types is extracted, combined with professional analysis, as the scenario parameter type for the entire set.

[0054] Step 3: Transformation and generalization of scenarios. Based on the functional scenarios obtained in Step 2 and combined with the types of scenario parameters obtained from the OpenX series of standards, first, all functional scenarios can be transformed into corresponding logical scenarios. Then, according to the actual situation, all logical scenarios can be reasonably generalized into corresponding specific test scenarios. Thus, a parameterized test scenario library classified according to the requirements of autonomous driving functions is obtained. For example:

[0055] Category 1: Scenario 1-1{A, B, C, ……}, Scenario 1-2{D, E, F, ……}……, Scenario 1-N{H, J, K, ……}. In the array, each parameter represents the specific value of the specified scenario parameter.

[0056] Step 4: Training of machine learning algorithm models. According to the parameterized scenario library obtained in Step 3, each autonomous driving function requirement corresponds to a number of classic test scenarios. Using these test scenarios as the training set, then perform ensemble learning of the Adaboosts algorithm model for each type of scenario respectively, and ensure that the prediction accuracy of each type of scenario reaches the required value to obtain the strong classifier corresponding to each autonomous driving function requirement, that is, the effectiveness determination rule for each type of scenario.

[0057] Step 5: Problem analysis and training set update. After obtaining the scenario effectiveness determination rule in Step 4, screen the extracted test scenarios. The screening results are divided into an effective scenario library and invalid (failed) scenarios. For the effective scenario library, which is the set of corresponding test scenarios strongly related to each autonomous driving function requirement, at the same time, a part of the test scenarios in the effective scenario library are used as training scenarios and put into the training set at a certain frequency to update the classifier; for invalid or failed scenarios, problem analysis should be focused on to find the deficiencies of the classifier, make adjustments, repair the failed scenarios, and improve the utilization rate of scenario data.

[0058] The above has introduced in detail the scenario effectiveness analysis method based on the requirements of autonomous driving functions proposed by the present invention. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, based on the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. Scenario effectiveness analysis method based on autonomous driving function requirements, characterized in that, it includes the following steps: Step S1, collect relevant information on autonomous driving, sort out the function requirements, typical dynamic events, and common static scenarios proposed by the autonomous driving system; Step S2, after classifying the scenarios according to the function requirements of autonomous driving, form the function scenarios of each type of scenario, and extract the definitions of scenario parameter types from the OpenScenario standard as the complete set of scenario parameter types; Step S3, according to the function scenarios and scenario parameter types of each type of scenario, convert all function scenarios into corresponding logical scenarios, and then, according to the actual situation, reasonably generalize all logical scenarios into corresponding specific test scenarios. Thus, a parameterized test scenario library classified according to the autonomous driving function requirements is obtained; Step S4, the parameterized scenario library includes several test scenarios, which are classified by the autonomous driving function requirements. Using these multiple test scenarios as the training set, then perform machine learning on each type of scenario separately, so as to obtain the classifier corresponding to each autonomous driving function requirement, and use the classifier as the determination rule for the effectiveness of each type of scenario; Step S5, screen the extracted test scenarios according to the determination rules for the effectiveness of each type of scenario, and perform training set update and problem analysis on the screening results respectively; In the said Step S1, the relevant information on autonomous driving includes autonomous driving technology development, autonomous driving test verification, and standard formulation; In the said Step S5, the screening results are divided into a valid scenario library and an invalid scenario library; The said valid scenario library is a set of corresponding test scenarios that are strongly related to each autonomous driving function requirement; The test scenarios in the said valid scenario library are put into the training set as training scenarios to update the classifier; Perform key problem analysis on the test scenarios in the said invalid scenario library, find the deficiencies of the classifier, adjust the classifier, and repair the invalid scenarios.

2. The scenario effectiveness analysis method based on autonomous driving function requirements according to claim 1, characterized in that, in the said Step S2, the formation of the function scenarios of each type of scenario is obtained through the permutation and combination of the dynamic events and static scenarios of each type of scenario and combined with manual analysis.

3. The scenario effectiveness analysis method based on autonomous driving function requirements according to claim 1, characterized in that, in the said Step S4, the machine learning is an Adaboosts algorithm model.

4. The scenario effectiveness analysis method based on autonomous driving function requirements according to claim 1, characterized in that, the said valid scenario library is a valid scenario library updated regularly according to frequency.

5. An electronic device, characterized in that, it includes a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus; The memory is used to store computer programs; The processor is used to implement the method steps described in any one of claims 1-4 when executing the programs stored on the memory.

Images