Test scene library generation method and device, computer equipment and storage medium

Abstract

The invention provides a test scene library generation method and device, computer equipment and a storage medium, and the method comprises the steps: generating the scene complexity of each test scene according to a preset attribute weight and inquired attribute value complexity; gathering the test scenes based on the scene complexity to generate a plurality of similar scene sets; calculating the average complexity of each similar scene set, sorting the plurality of similar scene sets according to a descending order of the average complexity, and generating a scene set sequence; and calculating the comprehensive complexity of each test scene in the scene set sequence, selecting and sorting the test scenes based on the comprehensive complexity, and generating a test scene library. According to the technical scheme provided by the embodiment of the invention, the simulation test of the test scene is realized through the test scene library, the reliability of the algorithm can be quickly verified, the coverage degree of the test scene is improved, the test time is shortened, and the test efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of simulation testing, and in particular to a method, device, computer device, and storage medium for generating a test scenario library. Background Art

[0002] With the continuous update of driverless technology, a large number of rapid verifications are required for driverless algorithms to meet the iterative requirements. Simulation testing can solve the problem that real vehicle testing cannot be rapidly verified. The simulation scenario generation method in the related art has the advantage of high coverage of scenario parameters and can cover a wide range of test scenarios, but it is easy to generate a large number of homogeneous scenarios, resulting in low test efficiency. Summary of the Invention

[0003] In view of the above problems, embodiments of the present invention provide a method, device, computer device, and storage medium for generating a test scenario library to solve the problem in the prior art that the simulation scenario generation method is easy to generate a large number of homogeneous scenarios, resulting in low test efficiency.

[0004] In a first aspect, an embodiment of the present invention provides a method for generating a test scenario library, the method including:

[0005] Generating the scenario complexity of each test scenario according to a preset attribute weight and the query result of the attribute value complexity;

[0006] Aggregating the test scenarios based on the scenario complexity to generate a plurality of similar scenario sets;

[0007] Calculating the average complexity of each similar scenario set, sorting the plurality of similar scenario sets in descending order according to the average complexity to generate a scenario set sequence;

[0008] Calculating the comprehensive complexity of each test scenario in the scenario set sequence, selecting and sorting the test scenarios based on the comprehensive complexity to generate a test scenario library.

[0009] In a possible implementation manner, the similar scenario set includes a first similar scenario set, and the aggregating the test scenarios based on the scenario complexity to generate a plurality of similar scenario sets includes:

[0010] Generating an attribute value set according to a preset attribute value range and value division value, where the attribute value set includes attribute value combinations under all permutation and combination modes;

[0011] Calculating the scenario complexity of the test scenario corresponding to each attribute value combination in the attribute value set;

[0012] Randomly select multiple combinations of attribute values from the set of attribute value selections as initial combinations of attribute values, where the initial combinations of attribute values are used to indicate the attribute values of the initial scenarios;

[0013] Sort the initial scenarios in descending order of scenario complexity to generate an initial scenario sequence;

[0014] Successively use the initial scenarios in the initial scenario sequence as the set centers to generate a first set of similar scenarios until the initial scenario sequence is empty.

[0015] In one possible implementation, the step of successively using the initial scenarios in the initial scenario sequence as the set centers to generate a first set of similar scenarios until the initial scenario sequence is empty includes:

[0016] Select a first target scenario from the initial scenario sequence and use the first target scenario as the set center of the first set of similar scenarios; the first target scenario is the scenario with the highest scenario complexity in the initial scenario sequence;

[0017] Delete the first target scenario from the initial scenario sequence to generate an updated initial scenario sequence;

[0018] Adjust the combinations of attribute values of all scenarios in the first set of similar scenarios according to the value division degree to generate a first set of new combinations of attribute values, where the first set of new combinations of attribute values is used to indicate the attribute values of the first set of new scenarios;

[0019] If the first set of new scenarios does not belong to the first set of similar scenarios, calculate the scenario complexity of the first set of new scenarios and add the scenario with the highest scenario complexity in the first set of new scenarios to the first set of similar scenarios;

[0020] Continue to execute the step of adjusting the combinations of attribute values of all scenarios in the first set of similar scenarios according to the value division degree to generate a first set of new combinations of attribute values until the first set of similar scenarios meets the first expansion end condition;

[0021] Based on the updated initial scenario sequence, continue to execute the step of selecting a first target scenario from the initial scenario sequence and using the first target scenario as the set center of the first set of similar scenarios until the initial scenario sequence is empty.

[0022] In one possible implementation, the first expansion end condition includes that the number of test scenarios in the first set of similar scenarios reaches a preset number threshold; or,

[0023] The first expansion end condition includes that the sum of the scenario complexities of all test scenarios in the first similar scenario set reaches a preset complexity threshold.

[0024] In a possible implementation, the similar scenario set further includes a second similar scenario set. The step of clustering the test scenarios based on the scenario complexity to generate multiple similar scenario sets further includes:

[0025] Determine the remaining attribute value combinations, where the remaining attribute value combinations include the attribute value combinations in the first newly added attribute value combinations that are not assigned to the first similar scenario set, and the attribute value combinations in the attribute value set except the initial attribute value combination; the remaining attribute value combinations are used to indicate the attribute values of the remaining test scenarios.

[0026] Sort the remaining test scenarios in descending order of scenario complexity to generate a remaining test scenario sequence.

[0027] Successively use the remaining test scenarios in the remaining test scenario sequence as the set center to generate a second similar scenario set until the to-be-allocated scenarios meet the second expansion end condition; the to-be-allocated scenarios include the scenarios in the remaining test scenario sequence that are not assigned to the second similar scenario set.

[0028] In a possible implementation, the step of successively using the remaining test scenarios in the remaining test scenario sequence as the set center to generate a second similar scenario set until the to-be-allocated scenarios meet the second expansion end condition includes:

[0029] Select a second target scenario from the remaining test scenario sequence and use the second target scenario as the set center of the second similar set; the second target scenario is the scenario with the highest scenario complexity in the remaining test scenario sequence.

[0030] Delete the second target scenario from the remaining test scenario sequence to generate an updated remaining test scenario sequence.

[0031] Adjust the attribute value combinations of all scenarios in the second similar scenario set according to the value division degree to generate a second newly added attribute value combination, where the second newly added attribute value combination is used to indicate the attribute values of the second newly added scenarios.

[0032] If the second newly added scenario does not belong to the second similar scenario set, calculate the scenario complexity of the second newly added scenario and add the scenario with the highest scenario complexity in the second newly added scenarios to the second similar scenario set.

[0033] Continue to execute the step of adjusting the attribute value combinations of all scenarios in the second similar scenario set according to the value division degree, and generating second new attribute value combinations;

[0034] Determine whether the scenario to be allocated meets the second expansion end condition;

[0035] If it is determined that the scenario to be allocated does not meet the second expansion end condition, based on the updated remaining test scenario sequence, continue to execute the step of selecting a second target scenario from the remaining test scenario sequence and using the second target scenario as the set center of the second similar set until the scenario to be allocated meets the second expansion end condition.

[0036] In a possible implementation, the second expansion end condition includes that the number of scenarios in the scenario to be allocated is less than a preset number threshold, and the sum of the scenario complexities of the scenario to be allocated is less than a preset complexity threshold.

[0037] In a possible implementation, the selecting and sorting the test scenarios based on the comprehensive complexity to generate a test scenario library includes:

[0038] According to the order of the similar scenario sets in the scenario set sequence, sequentially select the best test scenario from each of the similar scenario sets, where the best test scenario is used to indicate the test scenario with the largest comprehensive complexity in the similar scenario set;

[0039] Add the best test scenario to the test scenario set, and delete the best test scenario from the similar scenario set;

[0040] Add the attribute value combination corresponding to the best test scenario to the test scenario attribute value set until all similar scenario sets are traversed;

[0041] Update the scenario set sequence, and based on the updated scenario set sequence, continue to execute the step of sequentially selecting the best test scenario from each of the similar scenario sets in the scenario set sequence until the number of scenarios in the test scenario set reaches a preset scenario number threshold.

[0042] In a possible implementation, the updating the scenario set sequence includes:

[0043] Determine whether the similar scenario set is an empty set;

[0044] If it is determined that the similar scenario set is an empty set, delete the similar scenario set from the scenario set sequence;

[0045] If it is determined that the similar scene set is not an empty set, then based on the next similar scene set, the step of determining whether the similar scene set is an empty set is continued.

[0046] In a possible implementation manner, before adding the attribute value combination corresponding to the optimal test scenario to the test scenario attribute value set, the method further includes:

[0047] Determine whether the attribute value combination corresponding to the optimal test scenario belongs to the test scenario attribute value combination;

[0048] If it is determined that the attribute value combination does not belong to the test scenario attribute value combination, then the attribute value combination is added to the test scenario attribute value combination;

[0049] If it is determined that the attribute value combination belongs to the test scenario attribute value combination, there is no need to add the attribute value combination to the test scenario attribute value combination again.

[0050] In the technical solution provided by the embodiment of the present invention, a test scenario library can be quickly generated according to test requirements, which reduces the time for test scenario development and editing and improves test efficiency.

[0051] In the embodiment of the present invention, by implementing simulation testing of test scenarios through a test scenario library, the reliability of the algorithm can be quickly verified, and while improving the coverage of test scenarios, the test time is shortened and the test efficiency is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A flowchart of a method for generating a test scenario library provided by an embodiment of the present invention.

[0053] Figure 2 The present invention provides a flowchart of a method for generating a similar scene set in an embodiment of the present invention.

[0054] Figure 3 A flowchart of another similar scene set generation method provided by an embodiment of the present invention.

[0055] Figure 4 A flowchart of another similar scene set generation method provided by an embodiment of the present invention.

[0056] Figure 5 A flowchart of another similar scene set generation method provided by an embodiment of the present invention.

[0057] Figure 6 A flowchart of another test scenario library generation method provided by an embodiment of the present invention.

[0058] Figure 7Schematic structural diagram of a test scenario library generation device provided by an embodiment of the present invention.

[0059] Figure 8 Schematic structural diagram of a second generation module provided by an embodiment of the present invention.

[0060] Figure 9 Schematic structural diagram of a fourth generation module provided by an embodiment of the present invention.

[0061] Figure 10 Schematic diagram of a computer device provided by an embodiment of the present invention. Detailed implementation manners

[0062] To make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described below in conjunction with the accompanying drawings and specific implementation manners. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0063] Figure 1 Flowchart of a test scenario library generation method provided by an embodiment of the present invention. As Figure 1 shown, the method includes:

[0064] Step 101: Generate the scenario complexity of each test scenario according to the preset attribute weights and the query attribute value complexity.

[0065] In the embodiments of the present invention, to facilitate the construction of test scenarios, test scenarios are described by means of attribute value combinations. A test scenario has multiple scenario elements. Different scenario elements are abstracted into multiple attributes. Each scenario element corresponds to at least one attribute, and each attribute corresponds to an attribute value, and the attribute value is used to describe the state of the attribute. Among them, the scenario element is the basic constituent element of the test scenario. For example, the scenario elements include roads, ground, weather, light, etc. In practical applications, the scenario elements can be extended according to simulation requirements.

[0066] Specifically, according to the characteristics of the scenario elements, the attributes corresponding to the scenario elements are set, and different scenario elements correspond to different attributes. For example, the attributes corresponding to the road include road length, road width, road surface material type, attributes of intersections, etc.

[0067] In the embodiments of the present invention, by setting the value range and value division value of the attribute, while discretizing the attribute value, the infinite value of the attribute is avoided. For example, the value range of the road width is 10 meters - 15 meters, and the value division value is 0.5 meters. That is to say, the attribute values of the road width can be 10 meters, 10.5 meters, 11 meters, 11.5 meters, 12 meters, 12.5 meters, 13 meters, 13.5 meters, 14 meters, 14.5 meters, and 15 meters.

[0068] In the embodiments of the present invention, the scene complexity can reflect the influence of the attribute value on the performance of the driverless algorithm, and can also reflect the complexity of the test scene. The scene complexity depends on the attribute weight and the attribute value complexity. The greater the attribute weight, the greater the influence of the scene element corresponding to the attribute on the performance of the driverless algorithm; the smaller the attribute weight, the smaller the influence of the scene element corresponding to the attribute on the performance of the driverless algorithm. The attribute values are discrete, and different attribute values correspond to different attribute value complexities respectively. The greater the attribute value complexity, the higher the difficulty of the attribute value for the driverless algorithm, and the greater the complexity of the test scene; the smaller the attribute value complexity, the lower the difficulty of the attribute value for the driverless algorithm, and the smaller the complexity of the test scene.

[0069] Specifically, each attribute has a corresponding relationship between the attribute value and the attribute value complexity. The computer device queries the attribute value complexity corresponding to the attribute value based on the corresponding relationship between the attribute value and the attribute value complexity; multiplies the attribute weight by the attribute value complexity to obtain multiple complexity products, and adds the multiple complexity products to obtain the scene complexity of the test scene. For example, there are three attribute values in the attribute value combination, and the attribute weights corresponding to the three attribute values are the first attribute weight, the second attribute weight, and the third attribute weight respectively. The attribute value complexities corresponding to the three attribute values are the first attribute value complexity, the second attribute value complexity, and the third attribute value complexity respectively. At this time, the scene complexity = the first attribute weight * the first attribute value complexity + the second attribute weight * the second attribute value complexity + the third attribute weight * the third attribute value complexity. The value range of the attribute weight is 0 - 1, and the sum of all attribute weights is 1. That is, the sum of the first attribute weight, the second attribute weight, and the third attribute weight is 1.

[0070] Step 102: Aggregate the test scenes based on the scene complexity to generate multiple similar scene sets.

[0071] Figure 2 is a flowchart of a method for generating a similar scene set provided by an embodiment of the present invention. As Figure 2 shown, step 102 includes:

[0072] Step 1021a: Generate an attribute value set according to a preset attribute value range and value division value.

[0073] In this step, the attribute value set includes attribute value combinations under all permutation and combination methods.

[0074] Step 1022a: Calculate the scenario complexity of the test scenario corresponding to each attribute value combination in the attribute value set.

[0075] Step 1023a: Randomly select multiple attribute value combinations from the attribute value set as the initial attribute value combinations.

[0076] In this step, the initial attribute value combinations are used to indicate the attribute values of the initial scenarios.

[0077] Step 1024a: Sort the initial scenarios in descending order of scenario complexity to generate an initial scenario sequence.

[0078] Step 1025a: Successively use the initial scenarios in the initial scenario sequence as the set center to generate a first similar scenario set until the initial scenario sequence is empty.

[0079] Figure 3 It is a flowchart of another method for generating a similar scenario set provided by an embodiment of the present invention. As Figure 3 shown, step 1025a includes:

[0080] Step 10251: Select a first target scenario from the initial scenario sequence and use the first target scenario as the set center of the first similar scenario set.

[0081] In this step, the first target scenario is the scenario with the highest scenario complexity in the initial scenario sequence.

[0082] Step 10252: Delete the first target scenario from the initial scenario sequence to generate an updated initial scenario sequence.

[0083] Step 10253: Adjust the attribute value combinations of all scenarios in the first similar scenario set according to the value division value to generate a first new attribute value combination.

[0084] In this step, the first new attribute value combination is used to indicate the attribute values of the first new scenarios. Specifically, according to the arrangement order of the attribute values in the attribute value combination, the attribute values in the attribute value combination are successively adjusted by one division value to generate the first new attribute value combination. Here, adjusting by one division value includes increasing by one value division value or decreasing by one value division value.

[0085] For example, in the combination of attribute values, the road width is 12.5 meters. The attribute value of the road width is incremented by one value division each time until the upper limit of the value range is reached; then the attribute value of the road width is decremented by one value division each time until the lower limit of the value range is reached. At this time, all possible attribute values of the road width are traversed, and the adjustment of the next attribute value continues. In practical applications, the attribute values can also be adjusted in other orders, and the embodiments of the present invention do not limit this.

[0086] Step 10254: Determine whether the first newly added scenario belongs to the first similar scenario set; if it is determined that the first newly added scenario does not belong to the first similar scenario set, then execute Step 10255; if it is determined that the first newly added scenario belongs to the first similar scenario set, then execute Step 10256.

[0087] Step 10255: Calculate the scenario complexity of the first newly added scenario, and add the scenario with the highest scenario complexity in the first newly added scenario to the first similar scenario set.

[0088] Step 10256: There is no need to calculate the scenario complexity of the first newly added scenario.

[0089] In the embodiments of the present invention, after Step 10256, Step 10253 is continued until the first similar scenario set meets the first expansion end condition.

[0090] In this step, the first expansion end condition includes that the number of test scenarios in the first similar scenario set reaches a preset number threshold; or, the first expansion end condition includes that the sum of the scenario complexities of all test scenarios in the first similar scenario set reaches a preset complexity threshold. Among them, by restricting the number of test scenarios in a single first similar scenario set through the number threshold, it is possible to avoid an excessive number of test scenarios in the first similar scenario set, resulting in an overly large single set and making it difficult to distinguish scenarios between different similar scenario sets. By restricting the sum of the scenario complexities of all test scenarios in a single similar scenario set through the complexity threshold, the first similar scenario sets with a higher average complexity and fewer scenario numbers increase, while the first similar scenario sets with a lower average complexity and more scenario numbers decrease. In the embodiments of the present invention, by adjusting the number of test scenarios in different sets, the proportion of test scenarios with a higher scenario complexity in the subsequent test scenario library generation process is increased, thereby improving the test efficiency.

[0091] In the embodiments of the present invention, after the first similar scenario set meets the first expansion end condition, it further includes: continuing to execute Step 10251 based on the updated initial scenario sequence until the initial scenario sequence is empty.

[0092] Figure 4The flowchart of another method for generating a set of similar scenarios provided by an embodiment of the present invention is as follows. Figure 4 As shown, step 102 further includes:

[0093] Step 1021b: Determine the remaining combinations of attribute values.

[0094] In this step, the remaining combinations of attribute values include the combinations of attribute values in the first newly added combination of attribute values that have not been assigned to the first set of similar scenarios, and the combinations of attribute values in the set of attribute values other than the initial combination of attribute values; the remaining combinations of attribute values are used to indicate the attribute values of the remaining test scenarios.

[0095] Step 1022b: Sort the remaining test scenarios in descending order of scenario complexity to generate a sequence of remaining test scenarios.

[0096] Step 1023b: Successively use the remaining test scenarios in the sequence of remaining test scenarios as the set centers to generate a second set of similar scenarios until the to-be-assigned scenarios meet the second expansion end condition.

[0097] In this step, the to-be-assigned scenarios include the scenarios in the sequence of remaining test scenarios that have not been assigned to the second set of similar scenarios.

[0098] Figure 5 The flowchart of another method for generating a set of similar scenarios provided by an embodiment of the present invention is as follows. Figure 5 As shown, step 1023b includes:

[0099] Step 10231: Select a second target scenario from the sequence of remaining test scenarios and use the second target scenario as the set center of the second set of similar scenarios.

[0100] In this step, the second target scenario is the scenario with the highest scenario complexity in the sequence of remaining test scenarios.

[0101] Step 10232: Delete the second target scenario from the sequence of remaining test scenarios to generate an updated sequence of remaining test scenarios.

[0102] Step 10233: Adjust the combinations of attribute values of all scenarios in the second set of similar scenarios according to the value division degree to generate a second newly added combination of attribute values.

[0103] In this step, the second newly added combination of attribute values is used to indicate the attribute values of the second newly added scenarios.

[0104] Step 10234: Determine whether the second newly added scenario belongs to the second set of similar scenarios. If it is determined that the second newly added scenario does not belong to the second set of similar scenarios, then execute Step 10235; if it is determined that the second newly added scenario belongs to the second set of similar scenarios, then execute Step 10236.

[0105] Step 10235: Calculate the scenario complexity of the second newly added scenario, and add the scenario with the highest scenario complexity in the second newly added scenario to the second set of similar scenarios.

[0106] In this step, the second expansion end condition includes that the number of scenarios in the scenarios to be allocated is less than a preset number threshold, and the sum of the scenario complexities of the scenarios to be allocated is less than a preset complexity threshold; the scenarios to be allocated include the scenarios in the remaining test scenario sequence that have not been allocated to the second set of similar scenarios.

[0107] In the embodiment of the present invention, after Step 10235, it further includes: determining whether the scenarios to be allocated meet the second expansion end condition; if it is determined that the scenarios to be allocated do not meet the second expansion end condition, then based on the updated remaining test scenario sequence, continue to execute Step 10231 until the scenarios to be allocated meet the second expansion end condition; if it is determined that the scenarios to be allocated meet the second expansion end condition, then determine the average complexity that is closest to the scenario complexity of the scenarios to be allocated; and allocate the scenarios to be allocated to the set of similar scenarios corresponding to this average complexity. Among them, the set of similar scenarios is the first set of similar scenarios or the second set of similar scenarios.

[0108] It can be understood that when the scenarios to be allocated do not meet the second expansion end condition, it indicates that the scenario complexity of the scenarios to be allocated is relatively high, and there is still a certain degree of similarity, and a new set of similar scenarios can be formed; when the scenarios to be allocated meet the second expansion end condition, it indicates that the scenario complexity of the scenarios to be allocated is relatively low and relatively scattered, and a new set of similar scenarios cannot be formed.

[0109] Step 10236: There is no need to calculate the scenario complexity of the second newly added scenario.

[0110] Step 103: Calculate the average complexity of each set of similar scenarios, and sort the multiple sets of similar scenarios in descending order of the average complexity to generate a sequence of scenario sets.

[0111] In the embodiment of the present invention, first, a first set of similar scenarios is generated based on the initial scenario sequence, then a second set of similar scenarios is generated based on the remaining test scenario sequence, and finally, the scenarios to be allocated are allocated to the set of similar scenarios with the average complexity closest to it, obtaining a relatively complete set of similar scenarios and ensuring the coverage of the test scenarios.

[0112] Step 104: Calculate the comprehensive complexity of each test scenario in the scenario set sequence, select and sort the test scenarios based on the comprehensive complexity, and generate a test scenario library.

[0113] In this step, the comprehensive complexity is calculated according to the scenario complexity, the preset repetition weight, and the number of identical values through the comprehensive complexity calculation formula. Here, the number of identical values refers to the number of attributes with the same attribute value combination as the closest attribute value combination in the test scenario attribute value combination V. For example, if the attribute value combination is {110, 10.5, 3, 5, 10}, and the closest attribute value combination in the test scenario attribute value combination V is {110, 11.5, 2, 5, 10}, the number of identical values is 3.

[0114] In the embodiment of the present invention, the comprehensive complexity calculation formula is:

[0115]

[0116] where, C r represents the comprehensive complexity, C represents the scenario complexity, r represents the repetition weight, and n same represents the number of identical values.

[0117] In the embodiment of the present invention, the repetition weight is a constant less than and close to 1. For example, the repetition weight is 0.95. The more the number of identical values, the more lagged the order of the test scenario in the test scenario library is compared to the order when sorted only according to the scenario complexity, thereby improving the test efficiency.

[0118] Figure 6 is a flowchart of another method for generating a test scenario library provided by the embodiment of the present invention. As Figure 6 shown, step 104 includes:

[0119] Step 1041: Sequentially select the best test scenario from each similar scenario set according to the order of the similar scenario sets in the scenario set sequence.

[0120] In this step, the best test scenario is used to indicate the test scenario with the maximum comprehensive complexity in the similar scenario set.

[0121] Step 1042: Add the best test scenario to the test scenario set, and delete the best test scenario from the similar scenario set.

[0122] Step 1043: Add the attribute value combination corresponding to the best test scenario to the test scenario attribute value set until all similar scenario sets are traversed.

[0123] In this step, before step 1043, it further includes: determining whether the attribute value combination corresponding to the optimal test scenario belongs to the test scenario attribute value combination; if it is determined that the attribute value combination does not belong to the test scenario attribute value combination, adding the attribute value combination to the test scenario attribute value combination; if it is determined that the attribute value combination belongs to the test scenario attribute value combination, there is no need to add the attribute value combination to the test scenario attribute value combination again.

[0124] Step 1044, update the scenario set sequence.

[0125] In this step, updating the scenario set sequence includes: determining whether the similar scenario set is an empty set; if it is determined that the similar scenario set is an empty set, deleting the similar scenario set from the scenario set sequence; if it is determined that the similar scenario set is not an empty set, based on the next similar scenario set, continue to execute the step of determining whether the similar scenario set is an empty set.

[0126] In an embodiment of the present invention, after step 1044, it further includes: based on the updated scenario set sequence, continue to execute step 1041 until the number of scenarios in the test scenario set reaches a preset scenario number threshold.

[0127] In an embodiment of the present invention, after step 104, it further includes: constructing a basic scenario through RoadRunner software to generate a basic scenario file; batch modifying the attribute values in the basic scenario file according to the test scenario attribute value combination to generate a simulation test scenario file; opening the simulation test scenario file through CARLA software to complete the simulation test. During actual simulation testing, the unmanned driving algorithm is tested in the order in which the test scenarios are added to the test scenario library.

[0128] In an embodiment of the present invention, the basic scenario file includes an xodr file and an xosc file. The xodr file is a standard format for describing road information and is mainly used for road data files in the OpenDRIVE format. The xodr file follows the Extensible Markup Language (XML) format and is used to describe the topological structure, geometric shape, and other related attributes of the road network for use in simulation software or autonomous driving systems. The xosc file is a data description language file based on XML and is mainly used to describe the behavior of the code. The CARLA software supports importing xodr files and xosc files and supports batch testing of simulation test scenario files, thereby improving the test efficiency.

[0129] In practical applications, the coverage and scenario complexity of the test scenario library can be balanced according to the test requirements. For example, according to the test requirements, the proportion of scenarios with high complexity metrics can be increased to improve the test efficiency. Another example is to cover most of the values within different ranges of attribute values according to the test requirements to ensure the coverage of the test scenarios.

[0130] In the technical solution provided by the embodiment of the present invention, a test scenario library can be quickly generated according to the test requirements, which reduces the time for test scenario development and editing and improves the test efficiency.

[0131] In the embodiment of the present invention, the simulation test of the test scenario is implemented through the test scenario library, which can quickly verify the reliability of the algorithm. While improving the coverage of the test scenario, it shortens the test time and improves the test efficiency.

[0132] Figure 7 is a schematic structural diagram of a test scenario library generation device provided by an embodiment of the present invention. As Figure 7 shown, the device includes a first generation module 11, a second generation module 12, a third generation module 13, and a fourth generation module 14. The first generation module 11 is electrically connected to the second generation module 12, the second generation module 12 is electrically connected to the third generation module 13, and the third generation module 13 is electrically connected to the fourth generation module 14. The first generation module 11 is used to generate the scenario complexity of each test scenario according to the preset attribute weights and the queried attribute value complexity; the second generation module 12 is used to aggregate the test scenarios based on the scenario complexity to generate multiple similar scenario sets; the third generation module 13 is used to calculate the average complexity of each similar scenario set, sort the multiple similar scenario sets in descending order according to the average complexity, and generate a scenario set sequence; the fourth generation module 14 is used to calculate the comprehensive complexity of each test scenario in the scenario set sequence, select and sort the test scenarios based on the comprehensive complexity, and generate a test scenario library.

[0133] In the embodiment of the present invention, the similar scenario set includes a first similar scenario set. Figure 8 is a schematic structural diagram of a second generation module provided by an embodiment of the present invention. As Figure 8As shown in the figure, the second generation module 12 includes a first generation sub-module 121, a calculation sub-module 122, a first selection sub-module 123, a first sorting sub-module 124, and a second generation sub-module 125. The first generation sub-module 121 is electrically connected to the calculation sub-module 122 and the first selection sub-module 123 respectively. The calculation sub-module 122 is electrically connected to the first sorting sub-module 124. The first sorting sub-module 124 is electrically connected to the second generation sub-module 125. The first generation sub-module 121 is configured to generate an attribute value set according to a preset attribute value range and value division value. The attribute value set includes attribute value combinations in all permutation and combination modes. The calculation sub-module 122 is configured to calculate the scenario complexity of the test scenario corresponding to each attribute value combination in the attribute value set. The first selection sub-module 123 is configured to randomly select multiple attribute value combinations from the attribute value set as initial attribute value combinations, and the initial attribute value combinations are used to indicate the attribute values of the initial scenario. The first sorting sub-module 124 is configured to sort the initial scenarios in descending order of scenario complexity to generate an initial scenario sequence. The second generation sub-module 125 is configured to sequentially use the initial scenarios in the initial scenario sequence as the set center to generate a first similar scenario set until the initial scenario sequence is empty.

[0134] In an embodiment of the present invention, the second generation sub-module 125 is specifically configured to select a first target scenario from the initial scenario sequence, and use the first target scenario as the set center of the first similar scenario set. The first target scenario is the scenario with the highest scenario complexity in the initial scenario sequence. Delete the first target scenario from the initial scenario sequence to generate an updated initial scenario sequence. Adjust the attribute value combinations of all scenarios in the first similar scenario set according to the value division value to generate a first new attribute value combination, and the first new attribute value combination is used to indicate the attribute values of the first new scenario. If the first new scenario does not belong to the first similar scenario set, calculate the scenario complexity of the first new scenario, and add the scenario with the highest scenario complexity in the first new scenario to the first similar scenario set. Continue to execute the step of adjusting the attribute value combinations of all scenarios in the first similar scenario set according to the value division value to generate a first new attribute value combination until the first similar scenario set meets the first expansion end condition. Based on the updated initial scenario sequence, continue to execute the step of selecting a first target scenario from the initial scenario sequence and using the first target scenario as the set center of the first similar scenario set until the initial scenario sequence is empty.

[0135] In an embodiment of the present invention, the first expansion end condition includes that the number of test scenarios in the first similar scenario set reaches a preset number threshold; or, the first expansion end condition includes that the sum of the scenario complexities of all test scenarios in the first similar scenario set reaches a preset complexity threshold.

[0136] In an embodiment of the present invention, the similar scenario set further includes a second similar scenario set, and the second generation module 12 further includes a determination sub-module 126, a second sorting sub-module 127, and a third generation sub-module 128. The determination sub-module 126 is electrically connected to the second generation sub-module 125 and the second sorting sub-module 127 respectively, and the second sorting sub-module 127 is electrically connected to the third generation sub-module 128. The determination sub-module 126 is configured to determine the remaining attribute value combinations, where the remaining attribute value combinations include the attribute value combinations in the first newly added attribute value combinations that have not been assigned to the first similar scenario set, and the attribute value combinations in the attribute value set other than the initial attribute value combinations; the remaining attribute value combinations are used to indicate the attribute values of the remaining test scenarios; the second sorting sub-module 127 is configured to sort the remaining test scenarios in descending order of scenario complexity to generate a remaining test scenario sequence; the third generation sub-module 128 is configured to sequentially use the remaining test scenarios in the remaining test scenario sequence as the set center to generate a second similar scenario set until the to-be-allocated scenarios meet the second expansion end condition; the to-be-allocated scenarios include the scenarios in the remaining test scenario sequence that have not been assigned to the second similar scenario set.

[0137] In an embodiment of the present invention, the third generation sub-module 128 is specifically configured to select a second target scenario from the remaining test scenario sequence, and use the second target scenario as the set center of the second similar set; the second target scenario is the scenario with the highest scenario complexity in the remaining test scenario sequence; delete the second target scenario from the remaining test scenario sequence to generate an updated remaining test scenario sequence; adjust the attribute value combinations of all scenarios in the second similar scenario set according to the value division value to generate second newly added attribute value combinations, and the second newly added attribute value combinations are used to indicate the attribute values of the second newly added scenarios; if the second newly added scenarios do not belong to the second similar scenario set, calculate the scenario complexity of the second newly added scenarios, and add the scenario with the highest scenario complexity in the second newly added scenarios to the second similar scenario set; continue to execute the step of adjusting the attribute value combinations of all scenarios in the second similar scenario set according to the value division value to generate second newly added attribute value combinations; determine whether the to-be-allocated scenarios meet the second expansion end condition; if it is determined that the to-be-allocated scenarios do not meet the second expansion end condition, based on the updated remaining test scenario sequence, continue to execute the step of selecting a second target scenario from the remaining test scenario sequence and using the second target scenario as the set center of the second similar set until the to-be-allocated scenarios meet the second expansion end condition.

[0138] In an embodiment of the present invention, the second expansion end condition includes that the number of scenarios in the to-be-allocated scenarios is less than a preset number threshold, and the sum of the scenario complexities of the to-be-allocated scenarios is less than a preset complexity threshold.

[0139] Figure 9Schematic diagram of a fourth generation module provided by an embodiment of the present invention, as shown in Figure 9 As shown, the fourth generation module 14 includes a second selection sub-module 141, a deletion sub-module 142, a traversal sub-module 143, and an update sub-module 144. The second selection sub-module 141 is electrically connected to the deletion sub-module 142, the traversal sub-module 143, and the deletion sub-module 142 respectively. The second selection sub-module 141 is configured to sequentially select the best test scenario from each similar scenario set according to the order of the similar scenario sets in the scenario set sequence, and the best test scenario is used to indicate the test scenario with the greatest comprehensive complexity in the similar scenario set; the deletion sub-module 142 is configured to add the best test scenario to the test scenario set and delete the best test scenario from the similar scenario set; the traversal sub-module 143 is configured to add the attribute value combination corresponding to the best test scenario to the test scenario attribute value set until all similar scenario sets are traversed; the update sub-module 144 is configured to update the scenario set sequence, and based on the updated scenario set sequence, continue to execute the step of sequentially selecting the best test scenario from each similar scenario set in the scenario set sequence until the number of scenarios in the test scenario set reaches a preset scenario number threshold.

[0140] In an embodiment of the present invention, the update sub-module 144 is specifically configured to determine whether the similar scenario set is an empty set; if it is determined that the similar scenario set is an empty set, the similar scenario set is deleted from the scenario set sequence; if it is determined that the similar scenario set is not an empty set, based on the next similar scenario set, continue to execute the step of determining whether the similar scenario set is an empty set.

[0141] In an embodiment of the present invention, the fourth generation module 14 further includes a judgment sub-module 145, and the judgment sub-module 145 is electrically connected to the traversal sub-module 143. The judgment sub-module 145 is configured to determine whether the attribute value combination corresponding to the best test scenario belongs to the test scenario attribute value combination; if the judgment sub-module 145 determines that the attribute value combination does not belong to the test scenario attribute value combination, the traversal sub-module 143 is triggered to add the attribute value combination to the test scenario attribute value combination; if the judgment sub-module 145 determines that the attribute value combination belongs to the test scenario attribute value combination, the traversal sub-module 143 does not need to add the attribute value combination to the test scenario attribute value combination again.

[0142] In the technical solution provided by the embodiment of the present invention, a test scenario library can be quickly generated according to test requirements, reducing the time for test scenario development and editing and improving test efficiency.

[0143] In an embodiment of the present invention, simulation testing of test scenarios is implemented through the test scenario library, which can quickly verify the reliability of the algorithm, shorten the test time while improving the test scenario coverage, and improve test efficiency.

[0144] An embodiment of the present invention provides a computer-readable storage medium. The computer-readable storage medium includes a stored program. When the program runs, it controls the device where the computer-readable storage medium is located to execute the steps of the embodiments of the above test scenario library method. For specific descriptions, reference can be made to the embodiments of the above test scenario library method.

[0145] Figure 10 Schematic diagram of a computer device provided by an embodiment of the present invention, as Figure 10 shown, the computer device 3 of this embodiment includes: a processor 301, a memory 302, and a computer program 303 stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program 303, it implements the steps in the above method embodiments. Alternatively, when the processor 301 executes the computer program 303, it implements the functions of each module / unit in the above device embodiments.

[0146] The computer device 3 may be an electronic device such as a desktop computer, a notebook, a palm computer, and a cloud server. The computer device 3 may include, but is not limited to, the processor 301 and the memory 302. Those skilled in the art can understand that Figure 10 merely examples of the computer device 3, and do not constitute a limitation on the computer device 3. It may include more or fewer components than shown in the figure, or different components.

[0147] The processor 301 may be a central processing unit (CPU for short), or other general-purpose processors, digital signal processors (DSP for short), application specific integrated circuits (ASIC for short), field programmable gate arrays (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0148] The memory 302 may be an internal storage unit of the computer device 3, for example, the hard disk or memory of the computer device 3. The memory 302 may also be an external storage device of the computer device 3, for example, a plug-in hard disk equipped on the computer device 3, a SmartMedia Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. The memory 302 may also include both an internal storage unit and an external storage device of the computer device 3. The memory 302 is used to store computer programs and other programs and data required by the electronic device.

[0149] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the above-mentioned division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0150] As mentioned above, only the preferred specific embodiments of the present invention are described, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for generating a test scenario library, characterized in that: The method comprises: Generate the scenario complexity of each test scenario based on the preset attribute weights and the complexity of the queried attribute values; Based on the complexity of the scenarios, the test scenarios are clustered to generate multiple similar scenario sets; Calculating the average complexity of each similar scene set, sorting the multiple similar scene sets in descending order of the average complexity, and generating a scene set sequence; The comprehensive complexity of each test scenario in the scenario set sequence is calculated, and the test scenarios are selected and sorted based on the comprehensive complexity to generate a test scenario library.

2. The method according to claim 1, characterized in that The similar scene set includes a first similar scene set, and the test scenes are clustered based on the scene complexity to generate multiple similar scene sets, including: Generate an attribute value set according to a preset attribute value range and value division value, wherein the attribute value set includes attribute value combinations under all permutation and combination modes; Calculating the scenario complexity of the test scenario corresponding to each attribute value combination in the attribute value set; Randomly selecting a plurality of attribute value combinations from the attribute value set as initial attribute value combinations, wherein the initial attribute value combinations are used to indicate attribute values ​​of an initial scene; Sorting the initial scenes in descending order of scene complexity to generate an initial scene sequence; The initial scenes in the initial scene sequence are sequentially used as the collection center to generate a first similar scene collection until the initial scene sequence is empty.

3. The method according to claim 2, characterized in that The step of sequentially taking the initial scenes in the initial scene sequence as the center of the set to generate a first similar scene set until the initial scene sequence is empty includes: Selecting a first target scene from the initial scene sequence, and taking the first target scene as the center of a first similar scene set; the first target scene is the scene with the highest scene complexity in the initial scene sequence; Deleting the first target scene from the initial scene sequence to generate an updated initial scene sequence; Adjusting the attribute value combinations of all scenes in the first similar scene set according to the value division value to generate a first newly added attribute value combination, where the first newly added attribute value combination is used to indicate the attribute value of the first newly added scene; If the first newly added scene does not belong to the first similar scene set, calculating the scene complexity of the first newly added scene, and adding the scene with the highest scene complexity among the first newly added scenes to the first similar scene set; Continue to perform the step of adjusting the attribute value combinations of all scenes in the first similar scene set according to the value division value to generate a first newly added attribute value combination until the first similar scene set meets the first expansion end condition; Based on the updated initial scene sequence, the step of selecting a first target scene from the initial scene sequence and taking the first target scene as the center of a first similar scene set is continued until the initial scene sequence is empty.

4. The method according to claim 3, characterized in that The first expansion termination condition includes that the number of test scenarios in the first similar scenario set reaches a preset number threshold; or, The first expansion termination condition includes that the sum of the scene complexity of all test scenes in the first similar scene set reaches a preset complexity threshold.

5. The method according to claim 3, characterized in that: The similar scene set further includes a second similar scene set, and the clustering of the test scenes based on the scene complexity to generate multiple similar scene sets further includes: Determine the remaining attribute value combinations, the remaining attribute value combinations including the attribute value combinations in the first newly added attribute value combinations that are not assigned to the first similar scene set, and the attribute value combinations in the attribute value set other than the initial attribute value combinations; the remaining attribute value combinations are used to indicate the attribute values ​​of the remaining test scenes; Sorting the remaining test scenarios in descending order of scenario complexity to generate a sequence of remaining test scenarios; The remaining test scenes in the remaining test scene sequence are sequentially used as the set center to generate a second similar scene set until the scenes to be allocated meet the second expansion end condition; the scenes to be allocated include scenes in the remaining test scene sequence that are not allocated to the second similar scene set.

6. The method according to claim 5, characterized in that The step of sequentially taking the remaining test scenes in the remaining test scene sequence as the set center to generate a second similar scene set until the scene to be allocated meets the second expansion end condition includes: Selecting a second target scene from the remaining test scene sequence, and taking the second target scene as the set center of the second similarity set; the second target scene is the scene with the highest scene complexity in the remaining test scene sequence; Deleting the second target scene from the remaining test scene sequence to generate an updated remaining test scene sequence; Adjusting the attribute value combinations of all scenes in the second similar scene set according to the value division value to generate a second newly added attribute value combination, where the second newly added attribute value combination is used to indicate the attribute value of the second newly added scene; If the second newly added scene does not belong to the second similar scene set, calculating the scene complexity of the second newly added scene, and adding the scene with the highest scene complexity among the second newly added scenes to the second similar scene set; Continue to perform the step of adjusting the attribute value combinations of all scenes in the second similar scene set according to the value division value to generate a second newly added attribute value combination; Determining whether the scene to be allocated satisfies a second expansion end condition; If it is determined that the scene to be allocated does not meet the second expansion end condition, then based on the updated remaining test scene sequence, continue to execute the step of selecting a second target scene from the remaining test scene sequence and taking the second target scene as the set center of the second similarity set until the scene to be allocated meets the second expansion end condition.

7. The method according to claim 6, characterized in that The second expansion end condition includes that the number of scenes to be allocated is less than a preset number threshold, and the sum of the scene complexities of the scenes to be allocated is less than a preset complexity threshold.

8. The method according to claim 1, characterized in that The selecting and sorting the test scenarios based on the comprehensive complexity to generate a test scenario library includes: According to the order of similar scene sets in the scene set sequence, the best test scene is selected from each of the similar scene sets in turn, wherein the best test scene is used to indicate the test scene with the greatest comprehensive complexity in the similar scene sets; Adding the best test scenario to a test scenario set, and deleting the best test scenario from the similar scenario set; Adding the attribute value combination corresponding to the best test scenario to the test scenario attribute value set until all similar scenario sets are traversed; Update the scene set sequence, and based on the updated scene set sequence, continue to execute the step of selecting the best test scene from each of the similar scene sets in turn according to the order of similar scene sets in the scene set sequence, until the number of scenes in the test scene set reaches a preset scene number threshold.

9. The method according to claim 8, characterized in that The updating of the scene set sequence comprises: Determining whether the similar scene set is an empty set; If it is determined that the similar scene set is an empty set, deleting the similar scene set from the scene set sequence; If it is determined that the similar scene set is not an empty set, then based on the next similar scene set, the step of determining whether the similar scene set is an empty set is continued.

10. The method according to claim 8, characterized in that Before adding the attribute value combination corresponding to the optimal test scenario into the test scenario attribute value set, the method further includes: Determine whether the attribute value combination corresponding to the optimal test scenario belongs to the test scenario attribute value combination; If it is determined that the attribute value combination does not belong to the test scenario attribute value combination, then the attribute value combination is added to the test scenario attribute value combination; If it is determined that the attribute value combination belongs to the test scenario attribute value combination, there is no need to add the attribute value combination to the test scenario attribute value combination again.

Images