A specific acceleration test method with automatic driving function
By constructing an accelerated testing method specific to autonomous driving functions, and utilizing modules such as problem construction and scenario evaluation, key scenarios can be accurately identified, solving the problem of long testing times for autonomous driving, and improving testing efficiency and ensuring coverage of key scenarios.
Patent Information
- Application Number
- CN202310963707.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-02
- Publication Date
- 2025-11-11
- Estimated Expiration
- 2043-08-02
AI Technical Summary
Existing testing methods for autonomous driving functions are too time-consuming, and the key scenario selection methods based on expert knowledge cannot be tailored to specific needs, resulting in low testing efficiency.
An accelerated testing method with autonomous driving function specificity is constructed, including a problem construction module, a search module, a scenario evaluation module, a calculation module, and a termination judgment module. By defining a general scenario expression form, the method solves the key scenario selection problem with function specificity, evaluates the criticality of scenarios, calculates the number of scenarios, and determines whether the test process has ended based on the termination condition.
Precisely identifying key scenarios for specific autonomous driving functions improves testing efficiency, ensures coverage of key scenarios, and accelerates autonomous driving testing.
Smart Images
Figure CN117149963B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of accelerated testing for autonomous driving, and in particular to an accelerated testing method with autonomous driving function specificity. Background Technology
[0002] Autonomous vehicles must undergo thorough evaluation to ensure safety before commercialization. As autonomous driving technology develops and evolves, its functions are becoming increasingly diverse. All functions require evaluation through testing. To expedite the development and deployment of autonomous vehicles, efficient functional verification methods have become a key research focus.
[0003] Testing methods for autonomous driving functions can be divided into two types: exhaustive testing and selective testing. The purpose of exhaustive testing is to examine the performance of autonomous vehicles under all conditions. Typically, autonomous vehicles must travel hundreds of millions of miles to cover all scenarios. This would take tens or even hundreds of years through closed-course or road testing. Even simulation testing would take several years to complete the same mileage. This time-consuming testing hinders the development of autonomous driving technology. However, most scenarios in exhaustive testing are common and pose no safety threats. When only key scenarios are selected for testing, the required time can be significantly reduced.
[0004] To screen critical scenarios, knowledge-based critical scenario screening methods have been proposed. This approach relies on expert knowledge. A scenario is decomposed into multiple elements based on expert knowledge, and the criticality level of these elements is determined to identify critical scenarios. However, this method cannot be tailored to the specific autonomous driving functions being tested. Summary of the Invention
[0005] To address the shortcomings of existing technologies, the purpose of this invention is to provide an accelerated testing method specific to autonomous driving functions, ensuring that the selected scenarios are specifically designed for the autonomous driving function under test, while simultaneously accelerating the selection process and improving testing efficiency. To achieve the above-mentioned objectives and other advantages of this invention, an accelerated testing method specific to autonomous driving functions is provided, comprising:
[0006] The system includes a problem construction module, a search module connected to the problem construction module by a signal, a scenario evaluation module connected to the search module by a signal, a calculation module connected to the scenario evaluation module by a signal, and an end judgment module for determining whether the end conditions have been met.
[0007] The accelerated testing method includes the following steps:
[0008] S1. Define the general scenario expression form and construct key scenario selection questions with specific functions through the problem construction module;
[0009] S2. Solve the problem of selecting key scenarios with specific functions through the search module, and find potential key scenarios with specific functions.
[0010] S3. Evaluate the criticality level of potential functionally specific critical scenarios through the scenario evaluation module to determine whether the scenario is truly a critical scenario.
[0011] S4. The calculation module will calculate the number of scenarios that have been tested so far and evaluate the total number of function-specific key scenarios.
[0012] S5. The termination judgment module determines whether the test process can be terminated based on the current number of tests or the total number of key scenarios, combined with the selected termination conditions. If the conditions are met, the test process ends; otherwise, it returns to step S2.
[0013] Preferably, the general scenario expression form in step S1 is as follows:
[0014]
[0015] Where S is a scene; O(t0) is the initial image of the scene, which is a snapshot of the scene at the initial time t0. Let f be the set of initial frames for all scenes; f is the scene derivation function; it is defined as the function that derives the remaining frames of the scene from the initial frames:
[0016]
[0017] in, A given time period can be discretized into an ordered time series, as follows:
[0018]
[0019] Among them, t max This is the end time of the scene;
[0020] According to the above formula, a set of scenes is defined as:
[0021]
[0022] The scene set is rewritten as a combination of the initial scene set, the derived function, and the time set:
[0023] Formula (3) shows that given a specific derived function f and a time set... and They are equivalent.
[0024] The preferred key scenario selection problem is as follows:
[0025]
[0026]
[0027]
[0028] Where |·| represents the number of elements in the given set; A collection of all scenarios; A collection of all key scenarios; The set of scenarios being tested; S is a set of key scenarios identified through testing. Tmax It is the maximum number of tests; it is known that... for in, Let f be the initial set of images for the scene being tested. It is given: It can be defined as:
[0029]
[0030]
[0031] Here, I(S) is an indicator used to determine whether a scenario is critical. If I(S) = 1, S is a critical scenario; otherwise, S is not a critical scenario. (By replacing...) It can be obtained Through calculation Size and estimation The size is:
[0032]
[0033] because and overlapping, The size can be determined by To estimate, Given by the following formula:
[0034]
[0035] in, This is an estimation function that maps the tested set of scenarios to the number of key scenarios.
[0036] The problem of selecting specific key scenarios has been rephrased as follows:
[0037]
[0038]
[0039]
[0040]
[0041] The objective function maximizes the coverage of key scenes by finding an optimal set of initial images for the tested scene. The first constraint means that the number of initial images for the tested scene is finite. The second constraint is the definition of the test scene set. The third constraint is the definition of the initial images for the test scene.
[0042] Preferably, the specific scenario evaluation metrics in step S3 are:
[0043]
[0044] Among them, C threshold C(S) represents a critical threshold; C(S) represents the criticality level of a scenario, which indicates the degree of danger a scenario poses to the tested autonomous vehicle.
[0045] The criticality level of a scenario can be defined as:
[0046]
[0047] Wherein, CL represents a scene frame O(t) i The criticality level describes the degree of danger of a scene to the autonomous vehicle under test. The criticality level of a scene is determined by the most critical scene within it.
[0048] Preferably, the calculation of the number of scenes in step S4 is as follows:
[0049]
[0050] Where Q is a set Size, The sequence of the test scenario is as follows:
[0051]
[0052] Among them, S (q) Let q be the scenario for the qth test.
[0053] The preferred total number of key scenarios is specifically assessed as follows:
[0054]
[0055] Among them, R C The number of remaining key scenarios that have not yet been found, due to R C It is unknown. It can be estimated in the following ways:
[0056]
[0057] The first item is the key scenario found through calculation, namely... The second term Δ represents the estimated remaining key scenarios that have not yet been found, i.e., R. C ; for The p-th subset is defined as:
[0058]
[0059]
[0060]
[0061] Where d is the maximum number of scenes in each subset; P is the partition number. The subset number is defined as:
[0062]
[0063] Δ is defined as:
[0064]
[0065] Preferably, the conditions for determining whether the conditions are met in step S5 include the termination condition based on the maximum number of tests and the termination condition based on the lower bound of the marginal search efficiency.
[0066] The specific termination condition for the maximum number of tests is as follows:
[0067]
[0068] Among them, S Tmax The maximum number of tests is defined as:
[0069]
[0070] Where w is a scaling factor ranging from 0 to 1, set by the user.
[0071] Preferably, the termination condition for the lower bound of the marginal search efficiency is as follows:
[0072] E≤E b
[0073] Where E represents the marginal search efficiency, describing the increase in the number of key scenarios found due to the increase in test scenarios; E b The lower bound of marginal search efficiency is set by the user; when E reaches the lower bound E... bAt that time, the coverage of key scenarios is considered to have reached its maximum;
[0074] In the above formula, the method for estimating E is as follows:
[0075]
[0076] in, This is the estimated total number of key scenarios. Δ is the number of key scenarios found, Δ is the estimated number of key scenarios not found, and d is the number of additional test scenarios.
[0077] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention constructs a function-specific key scenario selection problem based on various information within the scenario, aiming to maximize the coverage of key scenarios. By solving the key scenario selection problem, it identifies function-specific key scenarios. Furthermore, by adjusting the termination condition, it flexibly determines the termination time of the selection process. This method can accurately find key scenarios corresponding to specific autonomous driving functions, ensuring the coverage of key scenarios while improving search efficiency. By setting up a problem construction module, a search module, a scenario evaluation module, a calculation module, and a termination judgment module, the following steps are taken: the problem construction module constructs a key scenario selection problem specific to autonomous driving functions; the search module finds potential function-specific key scenarios; the scenario evaluation module assesses the criticality level of the scenarios found in the test; the calculation module calculates the current number of scenario tests and estimates the total number of function-specific key scenarios; and the termination judgment module determines whether the termination condition has been met. If not, the search continues. Thus, in scenario selection, by constructing a function-specific key scenario selection problem, key scenarios are accurately located, accelerating the search for function-specific key scenarios and promoting autonomous driving testing. Attached Figure Description
[0078] Figure 1 The flowchart is for an accelerated testing method with autonomous driving function specificity according to the present invention. Detailed Implementation
[0079] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0080] Reference Figure 1An accelerated testing method with autonomous driving function specificity includes: a problem construction module, a search module connected to the problem construction module by a signal, a scene evaluation module connected to the search module by a signal, a calculation module connected to the scene evaluation module by a signal, and a termination judgment module for determining whether the termination condition has been met.
[0081] The accelerated testing method includes the following steps:
[0082] S1. Define the general scenario expression form and construct key scenario selection questions with specific functions through the problem construction module;
[0083] S2. Solve the problem of selecting key scenarios with specific functions through the search module, and find potential key scenarios with specific functions.
[0084] S3. Evaluate the criticality level of potential functionally specific critical scenarios through the scenario evaluation module to determine whether the scenario is truly a critical scenario.
[0085] S4. The calculation module will calculate the number of scenarios that have been tested so far and evaluate the total number of function-specific key scenarios.
[0086] S5. The termination judgment module determines whether the test process can be terminated based on the current number of tests or the total number of key scenarios, combined with the selected termination conditions. If the conditions are met, the test process ends; otherwise, it returns to step S2.
[0087] In step S1, the problem construction module constructs a function-specific key scenario selection problem. This module can be divided into two sub-steps. First, in the first sub-step, the problem construction module defines a general scenario expression form with autonomous driving function specificity, which is used to construct the function-specific key scenario selection problem.
[0088] Specifically, the general scenario expression form is as follows:
[0089]
[0090] Where S is a scene; O(t0) is the initial image of the scene, which is a snapshot of the scene at the initial time t0. Let f be the set of initial frames for all scenes; f is the scene derivation function, which can derive the remaining frames of the scene from the initial frames, and can be defined as:
[0091]
[0092] in, A time period is discretized into an ordered set of time series, specifically:
[0093]
[0094] Among them, t max This is the end time of the scene.
[0095] According to the above formula, a set of scenes can be defined as:
[0096]
[0097] The scene set can be rewritten as a combination of the initial scene set, the derived function, and the time set:
[0098]
[0099] The above formula shows that, given a specific derived function f and a time set, and They are equivalent.
[0100] A scene O(t) can be defined as a set of scene elements. A scene element describes one of the characteristics or states of an object. An element is either a static element or a dynamic element. A static element describes an object or environment that remains unchanged over time, such as traffic signs or road networks. Conversely, a dynamic element describes a state of a traffic participant that changes over time, such as speed and position. Therefore, a scene is a combination of static and dynamic elements and can be defined as:
[0101] O(t): = (X s X d (t))
[0102] Among them, X s A combination of static elements; X d (t) represents the combination of dynamic elements at time t, specifically:
[0103]
[0104]
[0105] in, As a static element; for Domain; Let be a dynamic element at time t; for The domain of definition.
[0106] In summary, It can be defined as:
[0107]
[0108] Next, in the second sub-step, the problem construction module combines general scenario expression forms to construct function-specific key scenario selection questions, with the goal of maximizing the coverage of key scenarios, and outputs the selection questions to the search module.
[0109] The specific problem for selecting key scenarios with specific functions is as follows:
[0110]
[0111]
[0112]
[0113] Where |·| represents the number of elements in the given set; A collection of all scenarios; A collection of all key scenarios; The set of scenarios being tested; S is a set of key scenarios identified through testing. Tnax This is the maximum number of tests.
[0114] The objective function aims to maximize key scenario coverage. Key scenario coverage is... The first constraint means that the number of scenarios tested is a finite proportion of all key scenarios; the second constraint is... Definition. Due to the set of key scenarios. The problem is unknown, and the above-mentioned problem is unsolvable. Therefore, the above problem has been restated into another version.
[0115] The first constraint can be restated by introducing the definition of the scene set. According to this definition, Specifically:
[0116]
[0117] in, This is the initial set of images for the scene being tested. Therefore, when f and It is given, and the first constraint is specifically:
[0118]
[0119] in, It can be defined as:
[0120]
[0121] The second constraint can be defined To rephrase Specifically:
[0122]
[0123] Here, I(S) is an indicator used to determine whether a scenario is critical. If I(S) = 1, S is a critical scenario; otherwise, S is not a critical scenario. This is achieved through substitution. The second constraint can be rewritten as:
[0124]
[0125] The objective function can be calculated Size and estimation Represent it using its size. The size is:
[0126]
[0127] because and overlapping, The size can be determined by To estimate. Therefore, Given by the following formula:
[0128]
[0129] in, This is an estimation function that maps the tested set of scenarios to the number of key scenarios.
[0130] Therefore, the problem of selecting specific key scenarios can be reformulated as:
[0131]
[0132]
[0133]
[0134]
[0135] The objective function maximizes the coverage of key scenes by finding an optimal set of initial images for the tested scene. The first constraint means that the number of initial images for the tested scene is finite. The second constraint is the definition of the test scene set. The third constraint is the definition of the initial images for the test scene.
[0136] In step S3, the scenario evaluation module will use scenario evaluation indicators to assess the criticality level of key scenarios.
[0137] Specifically, the scenario evaluation metrics are:
[0138]
[0139] Among them, C threshold C(S) represents a critical threshold; C(S) represents the criticality level of a scenario, indicating the degree of danger a scenario poses to the tested autonomous vehicle. The criticality level of a scenario can be defined as:
[0140]
[0141] Wherein, CL represents a scene frame O(t) i The criticality level describes the degree of danger a scene poses to the autonomous vehicle under test. The formula above shows that the criticality level of a scene is determined by the most critical scene within it.
[0142] In step S4, the calculation module calculates the required data, which can be divided into two sub-steps. First, in the first sub-step, the calculation module calculates the number of currently tested scenarios based on the data from the search module and the evaluation module, and outputs it to the termination judgment module.
[0143] Specifically, the number of scenes is calculated as follows:
[0144]
[0145] Where Q is a set Size, For a sequence of test scenarios, it can be written as:
[0146]
[0147] Among them, S (q) Let q be the scenario for the qth test.
[0148] Next, in the second sub-step, the calculation module will evaluate the total number of function-specific key scenarios based on the information of the tested scenario and the information of the key scenarios that have been found, and output the result to the termination judgment module.
[0149] Specifically, the total number of key scenarios is estimated as follows:
[0150]
[0151] Among them, R C This represents the number of remaining key scenes that have not yet been found. Due to R... C It is unknown. Estimate using the following methods:
[0152]
[0153] The first item is the key scenario found through calculation, namely... The second term Δ represents the estimated remaining key scenarios that have not yet been found, i.e., R. C ; for The p-th subset is defined as:
[0154]
[0155]
[0156]
[0157] Where d is the maximum number of scenes in each subset; P is the partition number. The subset number is defined as:
[0158]
[0159] Δ can be defined as:
[0160]
[0161] In step S5, the termination judgment module will determine whether the test process can be terminated based on the termination conditions. There are two types of termination conditions: one is based on the maximum number of tests; the other is based on the lower bound of the marginal search efficiency.
[0162] Specifically, the termination condition based on the maximum number of tests is:
[0163]
[0164] Among them, S Tmax The maximum number of tests can be defined as:
[0165]
[0166] Where w is a scaling factor ranging from 0 to 1, which can be set by the user.
[0167] Specifically, the termination condition based on the lower bound of the marginal search efficiency is:
[0168] f≤E b
[0169] Where E represents the marginal search efficiency, which describes the increase in the number of key scenarios found due to the increase in test scenarios; E b This is a lower bound for the marginal search efficiency, which can be set by the user. The lower bound is reached when E reaches this value. b At that time, the coverage of key scenarios was considered to have reached its maximum.
[0170] In the above formula, the method for estimating E is as follows:
[0171]
[0172] in, This is the estimated total number of key scenarios. Δ is the number of key scenarios found, Δ is the estimated number of key scenarios not found, and d is the number of additional test scenarios.
[0173] In summary, the goal of constructing a function-specific key scenario selection problem is to maximize the coverage of key scenarios. By solving the key scenario selection problem, the key scenarios corresponding to specific autonomous driving functions can be accurately identified, ensuring the coverage of key scenarios while improving search efficiency.
[0174] The number of devices and processing scale described herein are for the purpose of simplifying the description of the invention, and applications, modifications and variations thereof will be apparent to those skilled in the art.
[0175] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.
Claims
1. An accelerated testing method with specific characteristics for autonomous driving functions, characterized in that, include: The system includes a problem construction module, a search module connected to the problem construction module by a signal, a scenario evaluation module connected to the search module by a signal, a calculation module connected to the scenario evaluation module by a signal, and an end judgment module for determining whether the end conditions have been met. The accelerated testing method includes the following steps: S1. Define the general scenario expression form and construct function-specific key scenario selection questions through the problem construction module; the function-specific key scenario selection questions are as follows: Where |·| represents the number of elements in the given set; A collection of all scenarios; A collection of all key scenarios; The set of scenarios being tested; S is a set of key scenarios identified through testing. Tmax It is the maximum number of tests; it is known that... for in, Let f be the initial set of images for the scene being tested. It is given: It can be defined as: Here, I(S) is an indicator used to determine whether a scenario is critical. If I(S) = 1, S is a critical scenario; otherwise, S is not a critical scenario. This is achieved through replacement. It can be obtained Through calculation Size and estimation The size is: because and overlapping, The size can be determined by To estimate, Given by the following formula: in, For example, an estimation function maps the tested scene set to the number of key scenes; The problem of selecting specific key scenarios has been rephrased as follows: The objective function maximizes the coverage of key scenes by finding an optimal set of initial images for the tested scene. The first constraint means that the number of initial images for the tested scene is finite. The second constraint is the definition of the test scene set. The third constraint is the definition of the initial images for the test scene. S2. Solve the problem of selecting key scenarios with specific functions through the search module, and find potential key scenarios with specific functions. S3. Evaluate the criticality level of potential functionally specific critical scenarios through the scenario evaluation module to determine whether the scenario is truly a critical scenario. S4. The calculation module will calculate the number of scenarios that have been tested so far and evaluate the total number of function-specific key scenarios. S5. The termination judgment module determines whether the test process can be terminated based on the current number of tests or the total number of key scenarios, combined with the selected termination conditions. If the conditions are met, the test process ends; otherwise, it returns to step S2.
2. The accelerated testing method with autonomous driving function specificity as described in claim 1, characterized in that, The general scenario expression form in step S1 is as follows: Where S is a scene; O(t0) is the initial image of the scene, which is a snapshot of the scene at the initial time t0. Let f be the set of initial frames for all scenes; f is the scene derivation function; it is defined as the function that derives the remaining frames of the scene from the initial frames: in, A given time period can be discretized into an ordered time series, as follows: Among them, t max This is the end time of the scene; According to the above formula, a set of scenes is defined as: The scene set is rewritten as a combination of the initial scene set, the derived function, and the time set: Formula (3) shows that given a specific derived function f and a time set... and They are equivalent.
3. The acceleration testing method with autonomous driving function specificity according to claim 1, the scenario evaluation index in step S3 is specifically as follows: in, Where C threshold C(S) represents a critical threshold; C(S) represents the criticality level of a scenario, which indicates the degree of danger a scenario poses to the tested autonomous vehicle. The criticality level of a scenario can be defined as: Wherein, CL represents a scene frame o(t) i The criticality level describes the degree of danger of a scene to the autonomous vehicle under test. The criticality level of a scene is determined by the most critical scene within it.
4. The accelerated testing method with autonomous driving function specificity as described in claim 1, characterized in that, The calculation of the number of scenes in step S4 is as follows: Where Q is a set Size, The sequence of the test scenario is as follows: Among them, S (q) Let q be the scenario for the qth test.
5. The accelerated testing method with autonomous driving function specificity as described in claim 3, characterized in that, The total number of key scenarios is specifically assessed as follows: Among them, R C The number of remaining key scenarios that have not yet been found, due to R C It is unknown. It can be estimated in the following ways: The first item is the key scenario found through calculation, namely... The second term Δ represents the estimated remaining key scenarios that have not yet been found, i.e., R. C ; for The p-th subset is defined as: Where d is the maximum number of scenes in each subset; P is the partition number. The subset number is defined as: Δ is defined as:
6. The accelerated testing method with autonomous driving function specificity as described in claim 1, characterized in that, The conditions for determining whether the conditions are met in step S5 include the termination condition based on the maximum number of tests and the termination condition based on the lower bound of the marginal search efficiency. The specific termination condition for the maximum number of tests is as follows: Among them, S Tmax The maximum number of tests is defined as: Where w is a scaling factor ranging from 0 to 1, set by the user.
7. The accelerated testing method with autonomous driving function specificity as described in claim 5, characterized in that, The termination condition for the lower bound of the marginal search efficiency is as follows: E≤E b , Where E represents the marginal search efficiency, describing the increase in the number of key scenarios found due to the increase in test scenarios; E b The lower bound of marginal search efficiency is set by the user; when E reaches the lower bound E... b At that time, the coverage of key scenarios is considered to have reached its maximum; In the above formula, the method for estimating E is as follows: in, This is the estimated total number of key scenarios. Δ is the number of key scenarios found, Δ is the estimated number of key scenarios not found, and d is the number of additional test scenarios.