A method, device and electronic device for determining key test scenarios

Abstract

The present application discloses a method, device and electronic device for determining key test scenarios, the method comprising: for each test scenario, combining the autonomous driving algorithm with the test scenario for simulation, determining the evaluation index value of the test scenario, and the scene parameter values ​​corresponding to multiple test scenarios constitute a first parameter space; based on the evaluation index value, classifying each test scenario to determine the mean of the classified scene parameters; determining the target projection vector based on the mean of the classified scene parameters and the scene parameter values ​​corresponding to the test scenario; using the target projection vector, projecting the first parameter space to obtain a second parameter space; and determining the key test scenario by sampling the variables in the second parameter space. By adopting the above-mentioned key test scenario determination method, device and electronic device, the problem of not being able to quickly select test scenarios that can reflect the sensitive weak links of the autonomous driving algorithm during the simulation verification of the autonomous driving algorithm is solved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a method, device and electronic equipment for determining key test scenarios. Background Art

[0002] With the continuous increase in the number of cars in cities, urban traffic problems have become increasingly prominent, such as traffic congestion and air pollution. Autonomous driving technology obtains information about surrounding vehicles and the cloud through the Internet of Vehicles to achieve efficient vehicle dispatching, which not only alleviates the traffic congestion problem caused by the excessive number of vehicles, but also improves road traffic efficiency and driving and riding comfort. At present, autonomous driving technology has been increasingly widely used, and the main obstacle to the further promotion of autonomous driving technology lies in its safety considerations. In order to ensure the safety of autonomous driving vehicles, it is necessary to conduct a large number of simulation verifications on the autonomous driving algorithm in different test scenarios on the autonomous driving simulation platform to examine the performance of the autonomous driving algorithm in different test scenarios. Therefore, how to select test scenarios and examine the performance of autonomous driving algorithms in different test scenarios are crucial for the simulation verification of autonomous driving algorithms.

[0003] In the prior art, an autonomous driving algorithm is usually combined with a pre-set test scenario for simulation testing to determine the performance of the autonomous driving algorithm in the test scenario.

[0004] However, the test scenarios selected above are all pre-set and cannot test the sensitive weaknesses of the autonomous driving algorithm. Summary of the invention

[0005] In view of this, the present application provides a key test scenario determination method, device and electronic device, whose purpose is to select test scenarios that can perform simulation tests on the sensitive weak links of the autonomous driving algorithm, so as to examine the test performance of the sensitive weak links of the autonomous driving algorithm.

[0006] In a first aspect, an embodiment of the present application provides a method for determining a key test scenario, including:

[0007] For each test scenario, the autonomous driving algorithm is combined with the test scenario for simulation, and the evaluation index value of the test scenario is determined. The scenario parameter values ​​corresponding to the multiple test scenarios constitute a first parameter space;

[0008] Based on the evaluation index value, each test scene is classified to determine the mean of the classified scene parameters, where the mean of the classified scene parameters is the mean of the scene parameters under each classification;

[0009] Determine the target projection vector based on the mean value of the classified scene parameters, the mean value of the unclassified scene parameters, the scene parameter value corresponding to the test scene, the number of similar constraints and the number of classified constraints, where the number of similar constraints refers to the number of constraints that can classify the test scene into one category, and the number of classified constraints refers to the number of constraints that cannot classify the test scene into one category;

[0010] Using the target projection vector, projecting the first parameter space to obtain the second parameter space;

[0011] By sampling the variables in the second parameter space, key test scenarios are determined.

[0012] Optionally, classifying each test scenario based on the evaluation index value may include: determining a label corresponding to the test scenario based on the distribution of the evaluation index value; and classifying the test scenario based on the label corresponding to the test scenario.

[0013] Optionally, determining the target projection vector based on the mean of classified scene parameters, the mean of unclassified scene parameters, the scene parameter values ​​corresponding to the test scene, the number of similar constraints and the number of classified constraints may include: determining the inter-class variance based on the mean of classified scene parameters and the mean of unclassified scene parameters; determining the intra-class variance based on the mean of classified scene parameters and the scene parameter values ​​corresponding to the test scene; determining the target constraint equation based on the number of similar constraints, the number of classified constraints, the scene parameter values ​​corresponding to the test scene and the projection vector, the projection vector being a vector that can project the first parameter space to the second parameter space according to the constraint conditions; constructing the objective function based on the inter-class variance, the intra-class variance and the target constraint equation; determining the value of the projection vector corresponding to the maximum value of the function value of the objective function as the target projection vector.

[0014] Optionally, determining the inter-class variance based on the mean of the classified scene parameters and the mean of the unclassified scene parameters may include: for each classification, determining the difference between the mean of the classified scene parameters and the mean of the unclassified scene parameters corresponding to the classification as the first inter-class mean difference; for each classification, determining the transpose of the difference between the mean of the classified scene parameters and the mean of the unclassified scene parameters corresponding to the classification as the second inter-class mean difference; for each classification, determining the product of the number of scenes corresponding to the classification, the first inter-class mean difference and the second inter-class mean difference corresponding to the classification as the target inter-class variance; and determining the sum of the target inter-class variances corresponding to all classifications as the inter-class variance.

[0015] Optionally, determining the intra-class variance based on the mean of the classification scene parameters and the scene parameter values ​​corresponding to the test scenes may include: for each test scene under each category, determining the difference between the scene parameter values ​​corresponding to the test scene under the category and the mean of the classification scene parameters corresponding to the category as the first scene mean difference; for each test scene under each category, determining the transpose of the difference between the scene parameter values ​​corresponding to the test scene under the category and the mean of the classification scene parameters corresponding to the category as the second scene mean difference; for each test scene under each category, determining the product of the first scene mean difference and the second scene mean difference corresponding to the test scene under the category as the target scene variance; determining the sum of the target scene variances corresponding to all test scenes under the same category as the intra-class scene variance; and determining the sum of the intra-class scene variances corresponding to all categories as the intra-class variance.

[0016] Optionally, constructing an objective function based on between-class variance, within-class variance and the objective constraint equation may include: determining the product of the between-class variance, the projection vector and the transpose of the projection vector as the numerator; determining the difference between the product of the within-class variance, the projection vector and the transpose of the projection vector and the objective constraint equation as the denominator; and constructing the objective function using the ratio of the numerator to the denominator.

[0017] Optionally, using the target projection vector to project the first parameter space to obtain the second parameter space may include: for each scene parameter in the first parameter space, obtaining multiple values ​​corresponding to the scene parameter; randomly combining the multiple values ​​corresponding to all the scene parameters to determine multiple scene parameter combinations, each scene parameter combination including all the scene parameters and the first parameter value of each scene parameter in all the scene parameters; for each scene parameter combination, substituting the value corresponding to each scene parameter in the scene parameter combination into the projection expression to determine multiple values ​​of the variable in the second parameter space, the projection expression being determined based on the sum of the products of each parameter in the target projection vector and the scene parameter value corresponding to the parameter; based on the multiple values ​​of the variable in the second parameter space, determining the value range of the variable in the second parameter space; and determining the space formed by combining the value ranges of all the variables in the second parameter space as the second parameter space.

[0018] Optionally, determining the key test scenario by sampling the variables in the second parameter space may include: for each variable in the second parameter space, sampling the variable to determine the specific value of the variable; for each variable, substituting the specific value of the variable into the projection expression to determine the second parameter value of each scene parameter in all scene parameters corresponding to the specific value of the variable; and determining the key test scenario based on the second parameter value.

[0019] In a second aspect, an embodiment of the present application provides a test scenario selection device, comprising:

[0020] An evaluation module is used to combine the autonomous driving algorithm with the test scenario for simulation for each test scenario, and determine the evaluation index value of the test scenario, wherein the scenario parameter values ​​corresponding to the multiple test scenarios constitute a first parameter space;

[0021] A classification module is used to classify each test scene based on the evaluation index value to determine the classification scene parameter mean, where the classification scene parameter mean is the mean of the scene parameters under each classification;

[0022] A calculation module, used to determine the target projection vector based on the mean value of the classified scene parameter, the mean value of the unclassified scene parameter, the scene parameter value corresponding to the test scene, the number of similar constraints and the number of classified constraints, wherein the number of similar constraints refers to the number of constraints that can classify the test scene into one category, and the number of classified constraints refers to the number of constraints that cannot classify the test scene into one category;

[0023] A projection module, used to project the first parameter space using the target projection vector to obtain a second parameter space;

[0024] The determination module is used to determine the key test scenario by sampling the variables in the second parameter space.

[0025] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the key test scenario determination method as described above are performed.

[0026] The embodiments of the present application bring the following beneficial effects:

[0027] The embodiment of the present application provides a method, device and electronic device for determining key test scenarios, including: for each test scenario, combining the autonomous driving algorithm with the test scenario for simulation, determining the evaluation index value of the test scenario, and the scene parameters corresponding to multiple test scenarios constitute a first parameter space; based on the evaluation index value, classifying each test scenario to determine the classification scene parameter mean, the classification scene parameter mean is the mean of the scene parameters under each classification; based on the classification scene parameter mean, the unclassified scene parameter mean, the scene parameter value corresponding to the test scenario, the number of similar constraints and the number of classified constraints, determining the target projection vector, the number of similar constraints refers to the number of constraints that can classify the test scenario into one category, and the number of classified constraints refers to the number of constraints that cannot classify the test scenario into one category; using the target projection vector, projecting the first parameter space to obtain a second parameter space; and determining the key test scenario by sampling the variables in the second parameter space. The present application can select variables that reflect the sensitive weak links of the autonomous driving algorithm according to the evaluation index value by means of dimensionality reduction projection, and then determine the key test scenario by the selected variables, solving the problem that the prior art cannot quickly select test scenarios that can simulate the sensitive weak links of the autonomous driving algorithm.

[0028] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the following drawings are some implementation methods of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 A flowchart of a method for determining key test scenarios provided in an embodiment of the present application;

[0031] Figure 2 A schematic diagram of the structure of a device for determining a key test scenario provided in an embodiment of the present application;

[0032] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0034] At present, when conducting simulation tests on autonomous driving algorithms, in theory, the autonomous driving algorithms should be placed in a high-dimensional scene space composed of scene parameter combinations for traversal testing to verify whether the autonomous driving algorithms meet the requirements in all aspects. However, due to the excessive number of scene parameter combinations and the efficiency of simulation verification, it is not feasible to conduct simulation verification for each scene parameter combination. Therefore, it is usually necessary to select some test scenarios from all test scenarios as pre-set test scenarios, and then combine the autonomous driving algorithm with the pre-set test scenarios for simulation testing to determine the performance of the autonomous driving algorithm in the pre-set test scenarios. However, the test scenarios selected above are all pre-set and cannot test the sensitive and weak links of the autonomous driving algorithm.

[0035] Based on this, the embodiments of the present application provide a method, device and electronic device for determining key test scenarios. Through dimensionality reduction projection, variables that reflect the sensitive weak links of the autonomous driving algorithm can be selected according to the evaluation index values, and then the key test scenarios can be determined through the selected variables, thereby solving the problem in the prior art that it is impossible to quickly select test scenarios that can perform simulation tests on the sensitive weak links of the autonomous driving algorithm.

[0036] To facilitate understanding of this embodiment, a key test scenario determination method disclosed in an embodiment of the present application is first described in detail. Figure 1 A flowchart of a method for determining a key test scenario provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method comprises the following steps:

[0037] Step S101, for each test scenario, combining the autonomous driving algorithm with the test scenario to perform simulation, determining an evaluation index value of the test scenario, and the scenario parameter values ​​corresponding to the multiple test scenarios constitute a first parameter space;

[0038] Step S102, classifying each test scene based on the evaluation index value to determine the mean value of the classified scene parameters;

[0039] Step S103, determining a target projection vector based on the classified scene parameter mean, the unclassified scene parameter mean, the scene parameter value corresponding to the test scene, the number of similar constraints and the number of classified constraints;

[0040] Step S104, using the target projection vector, projecting the first parameter space to obtain a second parameter space;

[0041] Step S105 , determining a key test scenario by sampling the variables in the second parameter space.

[0042] In step S101, the test scenario refers to a test scenario randomly selected from all test scenarios, and the test scenario can examine the performance of the autonomous driving algorithm in a certain aspect. For example, the test scenario can examine the braking effect of the autonomous driving algorithm when an obstacle appears in front of the car. The autonomous driving algorithm refers to an algorithm that can simulate the driver's control of the car. It can continuously perceive the surrounding environment and predict possible changes. The achievable functions include: object detection, object recognition, object positioning and motion prediction.

[0043] The evaluation index value refers to the specific numerical value of the evaluation index determined after the autonomous driving algorithm is simulated in a certain test scenario. For example, the collision time is 2.5 seconds, and the 2.5 seconds is the evaluation index value, and the collision time is the evaluation index.

[0044] The scenario parameter value refers to the value of the scenario parameter corresponding to the test scenario. For example, the scenario parameter may be the traffic flow, the speed of the preceding vehicle, or the acceleration of the preceding vehicle. A test scenario is composed of multiple scenario parameters and the specific value of each scenario parameter.

[0045] The first parameter space refers to a multidimensional space consisting of multiple dimensions and the value range corresponding to each dimension, with each scene parameter corresponding to the selected test scene as a dimension and multiple values ​​of each scene parameter as the value range of the corresponding dimension.

[0046] In the specific implementation, each test scenario in all test scenarios corresponds to a test file, which includes the scenario parameters, test methods and evaluation indicators of the test scenario. The number of test scenarios is very large, and it is impossible to simulate every test scenario. Therefore, some test scenarios can be selected from all test scenarios, and the autonomous driving algorithm can be combined with each selected test scenario respectively. The autonomous driving algorithm is simulated according to the test method in the test file corresponding to the test scenario. The simulation result includes the evaluation index value, which is used to measure whether the performance of the autonomous driving algorithm in the test scenario meets the requirements.

[0047] For example, if the evaluation indicator is collision time, and the collision time is greater than 3 seconds, the requirement is met. If the evaluation indicator value is greater than 3 seconds after simulation, it means that the performance of the autonomous driving algorithm in this test scenario meets the requirements.

[0048] In step S102, the classification scene parameter mean refers to the mean of the values ​​of the scene parameters corresponding to all test scenes under each classification, denoted as μ (k) Assume that there are 10 test scenes in total, and each test scene corresponds to 5 scene parameters. Then the mean of the classification scene parameters is a 5×1 dimensional vector.

[0049] In an optional example, each test scenario is classified based on the evaluation index value, including: determining a label corresponding to the test scenario through the distribution of the evaluation index value; and classifying the test scenario based on the label corresponding to the test scenario.

[0050] Here, the label refers to an identifier that can classify the evaluation index value. Taking the evaluation index as collision time, for example, the label can be long collision time, medium collision time, short collision time, or excellent collision time, up to standard collision time, and unqualified collision time.

[0051] In the specific implementation, it is assumed that the evaluation index of 5 test scenarios among all the selected test scenarios is collision time. These 5 test scenarios are test scenario 1, test scenario 2, test scenario 3, test scenario 4 and test scenario 5. After simulation, the collision times of these 5 test scenarios are 2.5 seconds, 2.52 seconds, 2.45 seconds, 2.98 seconds and 3.3 seconds respectively. According to the distribution of the values ​​of these 5 evaluation indicators, the same label is configured for the test scenarios with similar evaluation indicator values. The labels of test scenario 1, test scenario 2 and test scenario 3 can be determined as short collision time, the label of test scenario 4 can be determined as medium collision time, and the label of test scenario 5 can be determined as long collision time. The test scenes with the same labels are classified into the same category.

[0052] In step S103, the unclassified scenario parameter mean is a concept corresponding to the classified scenario parameter mean. The unclassified scenario parameter mean refers to the mean of the values ​​of the scenario parameters corresponding to all the selected test scenarios without considering the classification of the test scenarios, denoted by μ.

[0053] In an optional embodiment, a target projection vector is determined based on a classified scene parameter mean, an unclassified scene parameter mean, a scene parameter value corresponding to a test scene, the number of similar constraints and the number of classified constraints, including: determining the inter-class variance based on the classified scene parameter mean and the unclassified scene parameter mean; determining the intra-class variance based on the classified scene parameter mean and the scene parameter value corresponding to the test scene; determining the target constraint equation based on the number of similar constraints, the number of classified constraints, the scene parameter value corresponding to the test scene and the projection vector, the projection vector being a vector that can project a first parameter space to a second parameter space according to constraints; constructing an objective function based on the inter-class variance, the intra-class variance and the target constraint equation; determining the value of the projection vector corresponding to the maximum value of the function value of the objective function as the target projection vector.

[0054] Here, the between-class variance is used to indicate the degree of discreteness between test scenarios of different categories, and the within-class variance is used to indicate the degree of discreteness between different test scenarios under the same category.

[0055] In an optional embodiment, the inter-class variance is determined based on the mean of the classified scene parameters and the mean of the unclassified scene parameters, including: for each classification, determining the difference between the mean of the classified scene parameters and the mean of the unclassified scene parameters corresponding to the classification as the first inter-class mean difference; for each classification, determining the transpose of the difference between the mean of the classified scene parameters and the mean of the unclassified scene parameters corresponding to the classification as the second inter-class mean difference; for each classification, determining the product of the number of scenes corresponding to the classification, the first inter-class mean difference and the second inter-class mean difference corresponding to the classification as the target inter-class variance; and determining the sum of the target inter-class variances corresponding to all classifications as the inter-class variance.

[0056] In the specific implementation, the between-class variance S b It can be determined by the following calculation formula:

[0057]

[0058] Among them, c represents the number of categories, k represents the number of categories, and l k represents the number of test scenes under the kth category, μ (k) represents the mean parameter value of the classified scene under the kth category, μ represents the mean parameter value of the unclassified scene, T represents transposition, μ (k) -μ represents the mean difference between the first categories, (μ (k) -μ) T represents the mean difference between the second categories, l k (μ (k) -μ)(μ (k) -μ) T represents the variance between target classes.

[0059] In an optional embodiment, the intra-class variance is determined based on the classification scene parameter mean and the scene parameter value corresponding to the test scene, including: for each test scene under each category, the difference between the scene parameter value corresponding to the test scene under the category and the classification scene parameter mean corresponding to the category is determined as the first scene mean difference; for each test scene under each category, the transposition of the difference between the scene parameter value corresponding to the test scene under the category and the classification scene parameter mean corresponding to the category is determined as the second scene mean difference; for each test scene under each category, the product of the first scene mean difference and the second scene mean difference corresponding to the test scene under the category is determined as the target scene variance; the sum of the target scene variances corresponding to all test scenes under the same category is determined as the intra-class scene variance; the sum of the intra-class scene variances corresponding to all categories is determined as the intra-class variance.

[0060] Intra-class variance S ω It can be determined by the following calculation formula:

[0061]

[0062] Among them, c represents the number of categories, k represents the number of categories, and l k represents the number of test scenes under the kth category, μ (k) represents the mean parameter value of the classified scene under the kth classification, μ represents the mean parameter value of the unclassified scene, T represents transposition, X i (k) represents the scene parameter value of the i-th test scene under the k-th category, X i (k) -μ (k) represents the average difference of the first scenario, (X i (k) -μ (k) ) T represents the average difference of the second scenario, (X i (k) -μ (k) )(X i (k) -μ (k) ) T represents the target scene variance.

[0063] The goal constraint equation can be determined by the following formula:

[0064]

[0065] Among them, ω represents the projection vector, ω T represents the transpose of the projection vector, n represents the number of all test scenes selected, x i and x irepresents the specific value of any scene parameter selected in the first parameter space, C represents the classification constraint relationship, M represents the same type constraint relationship, and n c Indicates the number of classification constraints, which can be determined according to the number of set classification constraints, n M It represents the number of similar constraints, which can be determined according to the number of set similar constraints. α is used to adjust the contribution value of the similarity between test scenes under similar constraints, which can be set to 1 here. β is used to adjust the contribution value of the separation between test scenes under classification constraints, which can be set to 20 here. Those skilled in the art can determine the specific values ​​of α and β according to actual conditions, and this application does not limit them here.

[0066] It should be noted that constraints are divided into similar constraints and classification constraints. Here, when multiple test files focus on detecting the performance of autonomous driving algorithms in the same aspect and the evaluation indicators are similar, the test scenes corresponding to these test files should be classified into the same category in the low-dimensional space after projection. Therefore, the constraint that divides these test scenes into the same category is the same constraint, that is, MustLink constraint. When multiple test files focus on detecting the performance of autonomous driving algorithms in different aspects or the evaluation indicators are not similar, the test scenes corresponding to these test files should not be classified into the same category in the low-dimensional space after projection. Therefore, the constraint that distinguishes these test scenes into different categories is the classification constraint, that is, CannotLink constraint. Here, multiple test files focusing on detecting the performance of autonomous driving algorithms in the same aspect can refer to focusing on detecting the performance of autonomous driving algorithms on the same function, and similar evaluation indicators can refer to the difference between the evaluation indicators being less than the set threshold. It should be noted that for the same test file, there may be multiple sub-category scenes, and the multiple sub-category scenes can be divided into different types of constraints based on other criteria. For example, if multiple sub-category scenes include curves, straight roads, T-junctions, and crossroads, these sub-category scenes can be divided into different types of constraints.

[0067] It can be seen that by maximizing J(ω), the distance between the test scenes corresponding to the projected classification constraints can be as far as possible, and the distance between the test scenes corresponding to the same type of constraints can be as close as possible. In this way, different values ​​of the variables in the projected second parameter space can correspond to test scenes with different labels. In other words, the selected test scenes are sensitive to the variables in the second parameter space.

[0068] In another optional embodiment, an objective function is constructed based on between-class variance, within-class variance and the objective constraint equation, including: determining the product of the between-class variance, the projection vector and the transpose of the projection vector as the numerator; determining the difference between the product of the within-class variance, the projection vector and the transpose of the projection vector and the objective constraint equation as the denominator; and constructing the objective function using the ratio of the numerator to the denominator.

[0069] The objective function can be determined by the following formula:

[0070]

[0071] Among them, ω * represents the function value of the objective function, ω represents the projection vector, ω T represents the transpose of the projection vector, S b represents the between-class variance, S ω represents the intra-class variance, and J(ω) represents the target constraint equation.

[0072] In the specific implementation, the function value ω of the objective function is * When the maximum value is taken, the value of the projection vector ω is determined as the target projection vector. Here, the target projection vector can be determined in an iterative manner, for example: first determine the values ​​of 100 projection vectors, then determine the value of the function value corresponding to each projection vector value, select the largest function value as the first function value, and then determine the values ​​of the other 100 projection vectors based on the values ​​of the above 100 projection vectors and the changing trend of the function value, and then determine the values ​​of the function values ​​corresponding to the values ​​of the 100 projection vectors, and select the largest function value as the second function value, and use the larger function value of the second function value and the first function value as the target function value. Similarly, the third function value can be determined, and the third function value is compared with the target function value. If the third function value is greater than the target function value, the third function value is used as the target function value, and so on. Repeat the above iterative process multiple times, and use the final selected target function value as the maximum value of the function value, and determine the projection vector corresponding to the target function value as the target projection vector. Here, those skilled in the art can determine the specific number of iterations according to actual conditions, and this application is not limited here.

[0073] In step S104, the second parameter space refers to a low-dimensional space obtained by projecting the first parameter space, wherein the first parameter space is a high-dimensional space.

[0074] In an optional embodiment, the first parameter space is projected using a target projection vector to obtain a second parameter space, including: for each scene parameter in the first parameter space, obtaining multiple values ​​corresponding to the scene parameter; randomly combining the multiple values ​​corresponding to all the scene parameters to determine multiple scene parameter combinations, each scene parameter combination including all the scene parameters and the first parameter value of each scene parameter in all the scene parameters; for each scene parameter combination, substituting the value corresponding to each scene parameter in the scene parameter combination into a projection expression to determine multiple values ​​of the variable in the second parameter space, the projection expression being determined based on the sum of the products of each parameter in the target projection vector and the scene parameter value corresponding to the parameter; based on the multiple values ​​of the variable in the second parameter space, determining the value range of the variable in the second parameter space; and determining the space formed by combining the value ranges of all the variables in the second parameter space as the second parameter space.

[0075] Here, the scene parameter combination includes all scene parameters in the first parameter space and each scene parameter in all scene parameters corresponds to a value. The first parameter value refers to the value corresponding to the scene parameter in the first parameter space, which is directly obtained from the test file. The projection expression refers to a calculation formula that can calculate the value of the variable in the second parameter space based on the value of each scene parameter in the first parameter space.

[0076] Assume that the target projection vector is a 2×5 dimensional vector, and there are 5 scene parameters in the first parameter space. These 5 scene parameters are a, b, c, d, and e. Each scene parameter has 100 parameter values. Then, a parameter value is randomly selected from the 100 parameter values ​​of each scene parameter. The selected 5 parameter values ​​are the first parameter values. These 5 scene parameters and the 5 first parameter values ​​are combined together to form a scene parameter combination. According to the above method, multiple scene parameter combinations can be determined. Then, each scene parameter combination is substituted into the projection expression to determine multiple values ​​of the variable in the second parameter space. The projection expression is as follows:

[0077] x=ω11×a+ω12×b+ω13×c+ω14×d+ω15×e;

[0078] y=ω21×a+ω22×b+ω23×c+ω24×d+ω25×e;

[0079] Among them, x and y represent different variables in the second parameter space, ω11 to ω15 represent the parameter values ​​of the first row in the target projection vector, and ω21 to ω25 represent the parameter values ​​of the second row in the target projection vector.

[0080] Here, since each scene parameter combination in the first parameter space can obtain the specific value of the corresponding variable, the scene parameter combinations in multiple first parameter spaces can determine multiple values ​​of each variable, wherein the variables in the second parameter space may include multiple variables, each variable corresponds to multiple values, and these variables and the multiple values ​​corresponding to each variable together constitute the second parameter space.

[0081] In step S105, the key test scenario refers to a test scenario that can test sensitive weak links in the autonomous driving algorithm, and there may be multiple key test scenarios.

[0082] In an optional embodiment, the key test scenario is determined by sampling the variables in the second parameter space, including: for each variable in the second parameter space, sampling the variable to determine the specific value of the variable; for each variable, substituting the specific value of the variable into the projection expression to determine the second parameter value of each scene parameter in all scene parameters corresponding to the specific value of the variable; based on the second parameter value, determining the key test scenario.

[0083] Here, the second parameter value refers to the specific value of the scene parameter calculated after substituting the specific value of the variable into the projection expression. The second parameter value is not constrained by the value range of the first parameter space, that is, the second parameter value may fall outside the first parameter space. It can be understood that the test scene corresponding to the second parameter value is not necessarily a test scene selected from all test scenes, which expands the selection range of the test scene.

[0084] In specific implementation, after determining the second parameter space, the variables in the second parameter space can be sampled with high coverage, so that the specific values ​​of the variables can be obtained, and then through the specific values ​​of the variables (the specific values ​​of x and y in the projection expression), multiple sets of values ​​of the scene parameter combination can be inferred, and one set of values ​​of the inferred scene parameter combination can be arbitrarily selected as the scene parameter combination corresponding to the key test scene. Here, when selecting the specific values ​​of the variables in the second parameter space, the selection can be made according to the label of the test scene corresponding to the value. For example: when the x variable in the second parameter space is greater than 300 and y is greater than 100, the collision time of the selected test scene is less than 3 seconds, indicating that the evaluation index of the test scene is unqualified. In this way, x is sampled within the range of the x variable value greater than 300, and y is sampled within the range of the y variable value greater than 100. The label of the test scene corresponding to the sampled x variable and y variable value must be short collision time. Then, by substituting the determined specific values ​​of x and y into the projection expression, multiple scene parameter combinations corresponding to the specific values ​​of x and y can be calculated, and any one of the calculated multiple scene parameter combinations is selected as the scene parameter combination corresponding to the key test scene, and the key test scene is constructed according to the selected scene parameter combination.

[0085] It can be seen that after projecting the first parameter space to the second parameter space, the number of variables in the obtained second parameter space is relatively small, and the different labels of the selected test scenarios are sensitive to the value changes of the variables in the second parameter space. In this way, it is easier to determine the key test scenarios corresponding to the sensitive weak links in the autonomous driving algorithm in the second parameter space, and the inferred test scenarios are no longer limited to the pre-set test scenario range, which increases the selection range of key test scenarios.

[0086] Figure 2 A schematic diagram of the structure of a key test scenario determination device provided in an embodiment of the present application, such as Figure 2 As shown, the device includes the following modules:

[0087] An evaluation module 201 is used to combine the autonomous driving algorithm with the test scenario for simulation for each test scenario, determine the evaluation index value of the test scenario, and the scenario parameters corresponding to the multiple test scenarios constitute a first parameter space;

[0088] A classification module 202, used to classify each test scene based on the evaluation index value to determine a classification scene parameter mean, where the classification scene parameter mean is the mean of the scene parameters under each classification;

[0089] A calculation module 203 is used to determine a target projection vector based on a classified scene parameter mean, an unclassified scene parameter mean, a scene parameter value corresponding to the test scene, a number of similar constraints, and a number of classified constraints, wherein the number of similar constraints refers to the number of constraints that can classify the test scene into one category, and the number of classified constraints refers to the number of constraints that cannot classify the test scene into one category;

[0090] A projection module 204, configured to project the first parameter space using the target projection vector to obtain a second parameter space;

[0091] The determination module 205 is used to determine the key test scenario by sampling the variables in the second parameter space.

[0092] Corresponds to Figure 1 The key test scenario determination method in the present application embodiment also provides a structural schematic diagram of an electronic device 300, such as Figure 3 As shown, the electronic device 300 includes a processor 310, a memory 320 and a bus 330. The memory 320 stores machine-readable instructions executable by the processor 310. When the electronic device 300 is running, the processor 310 communicates with the memory 320 through the bus 330. When the machine-readable instructions are executed by the processor 310, the above-mentioned key test scenario determination method can be executed. By means of dimensionality reduction projection, variables reflecting the sensitive weak links of the autonomous driving algorithm can be selected according to the evaluation index value, and then the key test scenarios are determined by the selected variables, which solves the problem in the prior art that it is impossible to quickly select test scenarios that can perform simulation tests on the sensitive weak links of the autonomous driving algorithm.

[0093] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the above method embodiments and will not be repeated here.

[0094] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0095] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0096] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0097] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0098] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0099] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for determining a key test scenario, characterized in that: The method comprises: For each test scenario, the autonomous driving algorithm is combined with the test scenario to perform simulation, and an evaluation index value of the test scenario is determined, and scenario parameter values ​​corresponding to multiple test scenarios constitute a first parameter space; Based on the evaluation index value, each test scene is classified to determine a classification scene parameter mean, where the classification scene parameter mean is the mean of the scene parameters under each classification; Determine a target projection vector based on the classified scene parameter mean, the unclassified scene parameter mean, the scene parameter value corresponding to the test scene, the number of similar constraints and the number of classified constraints, wherein the number of similar constraints refers to the number of constraints that can classify the test scene into one category, and the number of classified constraints refers to the number of constraints that cannot classify the test scene into one category; Using the target projection vector, projecting the first parameter space to obtain a second parameter space; Determining key test scenarios by sampling variables in the second parameter space; The determining of the target projection vector based on the classified scene parameter mean, the unclassified scene parameter mean, the scene parameter value corresponding to the test scene, the number of similar constraints and the number of classified constraints includes: Determining the inter-class variance based on the classified scenario parameter mean and the unclassified scenario parameter mean; Determining the intra-class variance based on the classification scene parameter mean and the scene parameter value corresponding to the test scene; Determining a target constraint equation based on the number of similar constraints, the number of classified constraints, the scene parameter values ​​corresponding to the test scene, and a projection vector, wherein the projection vector is a vector capable of projecting the first parameter space to the second parameter space according to the constraint condition; Constructing an objective function based on the inter-class variance, the intra-class variance and the objective constraint equation; The value of the projection vector corresponding to the maximum value of the function value of the objective function is determined as the target projection vector; The method of projecting the first parameter space using the target projection vector to obtain the second parameter space includes: For each scene parameter in the first parameter space, obtaining multiple values ​​corresponding to the scene parameter; Randomly combining multiple values ​​corresponding to all scene parameters to determine multiple scene parameter combinations, each of which includes all scene parameters and a value of each scene parameter in all scene parameters; For each scene parameter combination, substitute the value corresponding to each scene parameter in the scene parameter combination into a projection expression to determine multiple values ​​of the variable in the second parameter space, wherein the projection expression is determined based on the sum of the products of each parameter in the target projection vector and the scene parameter value corresponding to the parameter; Determining a value range of the variable in the second parameter space based on multiple values ​​of the variable in the second parameter space; The space formed by combining the value ranges of all variables in the second parameter space is determined as the second parameter space.

2. The method for determining key test scenarios according to claim 1, characterized in that: The classifying each test scenario based on the evaluation index value includes: Determining the label corresponding to the test scenario according to the distribution of the evaluation index value; The test scenarios are classified based on labels corresponding to the test scenarios.

3. The key test scenario determination method according to claim 1, characterized in that: The determining the inter-class variance based on the classified scene parameter mean and the unclassified scene parameter mean includes: For each category, the difference between the mean of the classified scene parameters corresponding to the category and the mean of the unclassified scene parameters is determined as the first inter-category mean difference; For each category, the difference between the mean of the classified scene parameter and the mean of the unclassified scene parameter corresponding to the category is transposed to determine the second inter-category mean difference; For each category, the product of the number of scenes corresponding to the category, the first inter-category mean difference and the second inter-category mean difference corresponding to the category is determined as the target inter-category variance; The sum of the target inter-class variances corresponding to all classifications is determined as the inter-class variance.

4. The key test scenario determination method according to claim 1, characterized in that: The determining the intra-class variance based on the classification scene parameter mean and the scene parameter value corresponding to the test scene includes: For each test scene under each category, determine the difference between the scene parameter value corresponding to the test scene under the category and the mean value of the classification scene parameter corresponding to the category as the first scene mean difference; For each test scene under each category, the transposition of the difference between the scene parameter value corresponding to the test scene under the category and the mean value of the classification scene parameter corresponding to the category is determined as the second scene mean difference; For each test scene under each category, the product of the first scene mean difference and the second scene mean difference corresponding to the test scene under the category is determined as the target scene variance; The sum of the target scene variances corresponding to all test scenes under the same category is determined as the intra-class scene variance; The sum of the intra-class scene variances corresponding to all categories is determined as the intra-class variance.

5. The key test scenario determination method according to claim 1, characterized in that: The constructing of the objective function based on the inter-class variance, the intra-class variance and the objective constraint equation includes: Determine the product of the inter-class variance, the projection vector and the transpose of the projection vector as a numerator; The difference between the product of the intra-class variance, the projection vector and the transpose of the projection vector and the target constraint equation is determined as the denominator; The objective function is constructed using the ratio of the numerator to the denominator.

6. The method for determining key test scenarios according to claim 1, characterized in that: The determining of the key test scenario by sampling the variables in the second parameter space includes: For each variable in the second parameter space, sampling the variable to determine a specific value of the variable; For each variable, substitute the specific value of the variable into the projection expression to determine the second parameter value of each scene parameter in all scene parameters corresponding to the specific value of the variable; Based on the second parameter value, a key test scenario is determined.

7. A device for determining a key test scenario, characterized in that: include: An evaluation module is used to simulate each test scenario by combining the autonomous driving algorithm with the test scenario to determine an evaluation index value of the test scenario, wherein scenario parameter values ​​corresponding to multiple test scenarios constitute a first parameter space; A classification module, used to classify each test scene based on the evaluation index value to determine a classification scene parameter mean, wherein the classification scene parameter mean is a mean value of the scene parameters under each classification; A calculation module, used to determine a target projection vector based on the classified scene parameter mean, the unclassified scene parameter mean, the scene parameter value corresponding to the test scene, the number of similar constraints and the number of classified constraints, wherein the number of similar constraints refers to the number of constraints that can classify the test scene into one category, and the number of classified constraints refers to the number of constraints that cannot classify the test scene into one category; A projection module, used to project the first parameter space using the target projection vector to obtain a second parameter space; A determination module, configured to determine a key test scenario by sampling variables in the second parameter space; The computing module is specifically used for: Determining the inter-class variance based on the classified scenario parameter mean and the unclassified scenario parameter mean; Determining the intra-class variance based on the classification scene parameter mean and the scene parameter value corresponding to the test scene; Determining a target constraint equation based on the number of similar constraints, the number of classified constraints, the scene parameter values ​​corresponding to the test scene, and a projection vector, wherein the projection vector is a vector capable of projecting the first parameter space to the second parameter space according to the constraint condition; Constructing an objective function based on the inter-class variance, the intra-class variance and the objective constraint equation; The value of the projection vector corresponding to the maximum value of the function value of the objective function is determined as the target projection vector; The projection module is specifically used for: For each scene parameter in the first parameter space, obtaining multiple values ​​corresponding to the scene parameter; Randomly combining multiple values ​​corresponding to all scene parameters to determine multiple scene parameter combinations, each of which includes all scene parameters and a value of each scene parameter in all scene parameters; For each scene parameter combination, substitute the value corresponding to each scene parameter in the scene parameter combination into a projection expression to determine multiple values ​​of the variable in the second parameter space, wherein the projection expression is determined based on the sum of the products of each parameter in the target projection vector and the scene parameter value corresponding to the parameter; Determining a value range of the variable in the second parameter space based on multiple values ​​of the variable in the second parameter space; The space formed by combining the value ranges of all variables in the second parameter space is determined as the second parameter space.

8. An electronic device, characterized in that: It includes a processor, a memory and a bus, the memory stores machine-readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the key test scenario determination method as described in any one of claims 1-6.

Images