Automatic driving software test evaluation method considering scene and function importance

Abstract

The invention provides an automatic driving software test evaluation method taking scene and function importance into consideration. The method comprises the following steps: carrying out classification and use frequency statistics on an automatic driving scene to determine a scene importance level; analyzing a typical fault mode of the automatic driving software based on expert knowledge; analyzing underlying functions and functional modules thereof in the typical fault mode through the fault tree; according to the frequency and importance of the fault mode, giving importance weights to the function and the module; the influence of scenes, functions and modules is comprehensively considered, and the time length of software testing and the number of testing cases are distributed; and constructing a deep neural network model, and performing model training based on a test result for test evaluation. According to the method, the test resources are optimally configured by integrating the importance degrees of scenes and functions, so that the pertinence and effectiveness of automatic driving software testing are improved.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous driving software testing and evaluation, and relates to an autonomous driving software testing and evaluation method that takes into account the importance of scenarios and functions. Background Art

[0002] With the rapid development of autonomous driving technology, ensuring the safety and reliability of autonomous driving software has become an important issue. Current testing methods include simulation testing, closed-loop testing, and real vehicle testing. Simulation testing can cover a large number of scenarios but may not fully simulate the real environment. Closed-loop testing combines actual and simulated scenarios to provide more comprehensive verification, while real vehicle testing provides the most realistic verification, but it is more expensive and takes longer.

[0003] Traditional testing methods tend to focus on specific scenarios or functions, but these methods may not fully cover complex driving environments or adapt to different functional requirements. In addition, in existing technologies, testing usually relies on standardized scenarios and preset functional models, which may result in test results that cannot truly reflect the performance of the software in actual use.

[0004] In addition, traditional testing methods also have problems in the allocation of test resources. The allocation of test time and test cases is usually based on experience or fixed standards, and fails to fully consider the actual importance and failure risk of each functional module. Therefore, there may be a risk of wasting resources or failing to fully verify key issues in the test. The current technical background urgently requires a more accurate and comprehensive method to optimize the test evaluation process of autonomous driving software and improve the overall safety and reliability of the system. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide an autonomous driving software testing and evaluation method that takes into account the importance of scenarios and functions, so as to solve the problems raised in the above background technology.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] Step 1: Classify autonomous driving scenarios and count the frequency of scenario usage to determine the importance level of the scenarios

[0008] First, the autonomous driving scenarios are classified, covering typical driving environments such as urban roads, highways, rainy days, nighttime, uphill and downhill. Then, by collecting and analyzing a large amount of actual driving data, statistical methods such as Bayesian estimation or maximum likelihood estimation are used to calculate the frequency of occurrence of various scenarios. Based on the frequency calculation results, the importance level of each scenario is determined by weighted average or entropy weight method. s . The importance of the scene I s It can be expressed by the formula:

[0009]

[0010] Among them, f s represents the frequency of scene s, w s Represents the weight of the scene, and n is the total number of scenes.

[0011] Step 2: Analyze typical failure modes of autonomous driving software

[0012] Perform statistical analysis on the operation of the autonomous driving software to identify and obtain typical failure modes. Use historical data to verify the rationality of the failure mode using methods such as likelihood ratio test or chi-square test, and calculate its failure frequency λ f :

[0013]

[0014] Among them, N f is the number of fault events, and T is the observation time.

[0015] Step 3: Establish the underlying functions and corresponding functional modules under typical failure modes based on fault tree analysis

[0016] Using the fault tree analysis method, each typical failure mode is decomposed into specific underlying functions and corresponding functional modules. By constructing a fault tree, the relationship between the functional modules is analyzed, and the impact of module failure on the system is calculated. The fault tree is quantitatively analyzed to calculate the failure probability P of each functional module. m and failure rate λ m , thereby determining the importance of the module.

[0017] Step 4: Combine the failure mode frequency and importance statistics to weight the importance of functions and modules

[0018] Combined with the scene importance I in step 1 s and the failure mode frequency λ in step 2 f , use the weighted average method or, calculate the comprehensive importance W of each function and module m The importance weighting of functions and modules can be calculated by the following formula:

[0019] W m =α×λ f +β×I s

[0020] Among them, α and β are weight coefficients, which are set according to project requirements and experience.

[0021] Step 5: Allocate software testing time and number of test cases based on the impact of scenarios, functions, and modules

[0022] Based on the comprehensive importance W calculated in step 4 m , allocate software testing time T m and the number of test cases N m . Use linear programming models or dynamic programming methods to optimize the allocation of test resources and ensure the test coverage of high-importance scenarios and functional modules. The allocation of test duration and number of cases can be achieved through the following optimization model:

[0023]

[0024]

[0025] Among them, T total and N total are the total test time and the total number of test cases respectively; λ1 and λ2 are the maximum proportion limits of important scenarios in the test time and the number of cases respectively.

[0026] Step 6: Build a neural network model, train the neural network based on the test data, and use it for test evaluation

[0027] After the test is completed, the deep neural network model is used to evaluate the test results. The neural network training is introduced to realize the output of intelligent software evaluation results under complex test data. The neural network model is trained based on the test evaluation results and multi-source data in the test process, so that the model can automatically process complex test data, generate features and output evaluation results.

[0028] Advantages of the present invention:

[0029] The advantage of the autonomous driving software test evaluation method of the present invention that considers the importance of scenarios and functions lies in its comprehensive optimization of autonomous driving software testing. First, by analyzing the importance of autonomous driving scenarios and functional modules, the method can ensure that test resources are concentrated on key scenarios and high-risk functions, thereby improving the comprehensiveness and effectiveness of the test. Secondly, the method combines the frequency and importance of failure modes and uses fault tree analysis to assign weights to test functions and modules, which can effectively avoid the problem of resource waste in traditional methods. Finally, the test is analyzed through a deep neural network model, which adds uncertainty factors, can further improve the accuracy and reliability of the test, and is also conducive to ensuring the intelligent performance of the software in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to make the purpose, technical solution and beneficial effects of the present invention clearer, the present invention provides the following figure for explanation: Figure 1 Flowchart of the autonomous driving software testing and evaluation method considering the importance of scenarios and functions. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0032] As attached Figure 1 As shown in FIG, a method for evaluating autonomous driving software testing that considers the importance of scenarios and functions is mainly used to comprehensively optimize the autonomous driving software testing through importance analysis, function weighting, and neural network fitting, including:

[0033] Step 1: Classify autonomous driving scenarios and count the frequency of scenario usage to determine the importance level of the scenarios;

[0034] Step 2: Analyze typical failure modes of autonomous driving software;

[0035] Step 3: Establish the underlying functions and corresponding functional modules under typical failure modes based on fault tree analysis;

[0036] Step 4: Combine the failure mode frequency and importance statistics to weight the importance of functions and modules;

[0037] Step 5: Allocate software testing time and the number of test cases based on the impact of scenarios, functions, and modules;

[0038] Step 6: Build a neural network model, train the neural network based on the test data, and use it for test evaluation.

[0039] In this embodiment, before testing and evaluating the car's autonomous driving software, first complete the requirements of step 1, step 2, step 3, step 4, and step 5, consider the impact of autonomous driving scenarios, functions, and modules, allocate software testing time and the number of test cases, and improve the test evaluation method. Further, the neural network described in step 6 is constructed to complete the construction of an autonomous driving software test evaluation method that considers the importance of scenarios and functions.

[0040] In specific implementation, step 1 includes:

[0041] Step 11, classify the autonomous driving scenarios, covering typical driving environments such as urban roads, highways, rainy days, nighttime, uphill and downhill driving, etc.

[0042] Step 12, by collecting and analyzing a large amount of actual driving data, using statistical methods such as Bayesian estimation or maximum likelihood estimation, calculate the frequency of occurrence of various scenarios.

[0043] Step 13: Based on the frequency calculation results, use the formula Determine the importance level of each scene s Among them, f s represents the frequency of scene s, w sRepresents the weight of the scene, and n is the total number of scenes.

[0044] In specific implementation, step 2 may include:

[0045] Step 21, perform statistical analysis on the operation of the autonomous driving software to identify and obtain typical failure modes.

[0046] Step 22, verify the rationality of the failure mode by using historical data, using likelihood ratio test or chi-square test, and then calculate the probability of failure mode according to the formula Calculate its failure frequency λ f Among them, N f is the number of fault events, and T is the observation time.

[0047] In specific implementation, step 3 may include:

[0048] Step 31, using the fault tree analysis method, decompose each typical failure mode into specific underlying functions and corresponding functional modules.

[0049] Step 32: Analyze the relationship between the functional modules by constructing a fault tree, and calculate the impact of module failure on the system. Perform quantitative analysis on the fault tree, calculate the failure probability and failure rate of each functional module, and thus determine the importance of the module.

[0050] In specific implementation, step 4 may include:

[0051] Step 41, combining the scene importance I in step 1 s and the failure mode frequency λ in step 2 f , using the importance weighting formula W for functions and modules m =α×λ f +β×I s Calculate the comprehensive importance W of each function and module m Among them, α and β are weight coefficients, which are set according to project requirements and experience.

[0052] In specific implementation, step 5 may include:

[0053] Step 51: Use a linear programming model or a dynamic programming method to optimize the test resource allocation to ensure the test coverage of high-importance scenarios and functional modules. m , allocated to the specific software testing time T m and the number of test cases N m The distribution of test duration and number of cases is achieved through the following optimization model:

[0054]

[0055]

[0056] Among them, T total and N total are the total test time and the total number of test cases respectively; λ1 and λ2 are the maximum proportion limits of important scenarios in the test time and the number of cases respectively.

[0057] In specific implementation, step 6 may include:

[0058] Step 61, after completing the autonomous driving software test, the test data is obtained and a deep neural network model is constructed to evaluate the test results.

[0059] Step 62, training the neural network model according to the test result data, so that the model can automatically process complex test data, generate features and output evaluation results.

Claims

1. A test and evaluation method for autonomous driving software considering the importance of scenarios and functions, characterized in that: include: The method completes the autonomous driving software testing through the following 7 steps. Step 1: Determine the importance level of the scene through the classification of autonomous driving scenes and the frequency of scene usage statistics; Step 2: Analyze the typical failure modes of autonomous driving software; Step 3: Determine the importance of the underlying functions and corresponding functional modules under the typical failure modes of the software based on fault tree analysis; Step 4: Combine the failure mode frequency and importance statistics to weight the importance of functions and modules; Step 5: Comprehensively consider the impact of scenes, functions, and modules, and allocate the software testing time and the number of test cases; Step 6: Introduce neural network training to realize the output of intelligent software evaluation results under complex test data.

2. The autonomous driving software testing and evaluation method considering scenarios and function importance as claimed in claim 1, characterized in that: The specific step 1 is as follows: First, consider key factors such as weather, road type, uphill and downhill slopes to classify the autonomous driving scenarios, covering typical driving environments such as urban roads, highways, rainy days, nighttime, uphill and downhill. Then, analyze the large amount of actual driving data collected, and use statistical methods such as Bayesian estimation to obtain the occurrence frequency of each scenario f s Based on the frequency results, the importance level of each scene is determined by weighted average. s . The importance of the scene I s It can be expressed by the formula: Among them, f s represents the frequency of scene s, w s Represents the weight of the scene, and n is the total number of scenes.

3. The autonomous driving software testing and evaluation method considering scenarios and function importance as claimed in claim 1, characterized in that: The third step is to analyze the operating logic of the autonomous driving software, identify and obtain typical failure modes. Through historical data, the likelihood ratio test method is used to verify the rationality of the failure mode and calculate its failure frequency λ f : Among them, N f is the number of fault events, and T is the observation time.

4. The autonomous driving software testing and evaluation method considering scenarios and function importance as claimed in claim 1, characterized in that: Step 4 combines the scene importance I s and failure mode frequency λ f , use the weighted average method or, calculate the comprehensive importance W of each function and module m The importance weighting of functions and modules can be calculated by the following formula: IN m =α×λ f +β×I s Among them, α and β are weight coefficients, which are set according to project requirements and experience.

5. The autonomous driving software testing and evaluation method considering scenarios and function importance as claimed in claim 1, characterized in that: The step 5 is based on the calculated comprehensive importance W m , allocate software testing time T m and the number of test cases N m . Use dynamic programming methods to optimize test resource allocation and ensure test coverage of high-importance scenarios and functional modules. The allocation of test duration and number of cases can be achieved through the following optimization model: Among them, T total and N total are the total test time and the total number of test cases respectively; λ1 and λ2 are the maximum proportion limits of important scenarios in the test time and the number of cases respectively.

6. The autonomous driving software testing and evaluation method considering scenarios and function importance as claimed in claim 1, characterized in that: The steps introduce neural network training to realize the output of intelligent software evaluation results under complex test data, and train the neural network model according to the test evaluation results and multi-source data in the test process, so that the model can automatically process complex test data, generate features and output evaluation results.

Images