Simulation test scene recommendation method and device, storage medium and computer device

Abstract

The application provides a simulation test scene recommendation method and device, a storage medium and a computer device. The method comprises the following steps: acquiring first automatic driving data output by a first automatic driving algorithm under each simulation test scene, and second automatic driving data output by a second automatic driving algorithm under each simulation test scene; determining a difference value of each simulation test scene under each difference evaluation index according to the first automatic driving data, the second automatic driving data and the pre-set difference evaluation indexes; for each difference evaluation index, filtering out a simulation test scene with a large difference degree as a target scene corresponding to the difference evaluation index from the simulation test scenes according to the difference values corresponding to the difference evaluation index; and determining a recommended scene set according to the target scenes corresponding to the difference evaluation indexes. The recommended scene set comprises at least one target scene. The application can improve the test efficiency.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a method, apparatus, storage medium, and computer equipment for recommending simulation test scenarios. Background Technology

[0002] With the continuous iteration of autonomous driving algorithms, more and more versions of these algorithms have emerged. To evaluate the merits of different versions, A / B testing is required. A / B testing involves simulating both versions under the same testing environment and scenario set, and then comparing their performance based on the autonomous driving data output in different simulation scenarios. However, A / B testing often requires a large number of simulation scenarios, while current technology relies on manual verification to check the performance of each version of the autonomous driving algorithm in each simulation scenario, which is time-consuming and inefficient. Summary of the Invention

[0003] The purpose of this application is to at least address one of the aforementioned technical defects, particularly the technical defects of long testing time and low testing efficiency in the prior art.

[0004] In a first aspect, embodiments of this application provide a method for recommending simulation test scenarios, the method comprising:

[0005] Acquire first autonomous driving data output by the first autonomous driving algorithm in each simulation test scenario, and second autonomous driving data output by the second autonomous driving algorithm in each simulation test scenario;

[0006] Based on the first autonomous driving data, the second autonomous driving data, and the pre-set difference evaluation indicators, determine the difference value of each simulation test scenario under each difference evaluation indicator;

[0007] For each difference evaluation index, based on the difference values ​​corresponding to the difference evaluation index, the simulation test scenarios with a large degree of difference are selected from each of the simulation test scenarios as the target scenarios corresponding to the difference evaluation index.

[0008] Based on the target scenarios corresponding to each of the aforementioned difference evaluation indicators, a set of recommended scenarios is determined; the set of recommended scenarios includes at least one of the aforementioned target scenarios.

[0009] In one embodiment, the step of determining the recommended scenario set based on the target scenarios corresponding to each of the difference evaluation indicators includes:

[0010] The number of target scenarios corresponding to each of the aforementioned difference evaluation indicators is summed to obtain the total number of target scenarios;

[0011] Based on the total number of target scenarios and the number of target scenarios corresponding to each difference evaluation indicator, the proportion of target scenarios corresponding to each difference evaluation indicator is determined respectively;

[0012] Get the total number of pre-set recommended scenarios;

[0013] Based on the total number of recommended scenarios and the proportion of target scenarios corresponding to each of the difference evaluation indicators, K recommended scenarios are selected from the target scenarios corresponding to each of the difference evaluation indicators to obtain a set of recommended scenarios including K recommended scenarios; where K is the total number of recommended scenarios.

[0014] In one embodiment, the difference value is positively correlated with the degree of difference;

[0015] The step of selecting K recommended scenarios from the target scenarios corresponding to each of the difference evaluation indicators based on the total number of recommended scenarios and the proportion of target scenarios corresponding to each of the difference evaluation indicators to obtain a set of recommended scenarios including K recommended scenarios includes:

[0016] Obtain the recommended upper limit and recommended lower limit values ​​corresponding to each of the pre-set difference evaluation indicators;

[0017] For each difference evaluation index, the difference values ​​of each target scenario corresponding to the difference evaluation index are sorted in descending order, and the target scenarios corresponding to the first to Nth difference values ​​are taken as each of the recommended scenarios. Each of the recommended scenarios is added to the recommended scenario set, where N is the lower limit value of the recommendation corresponding to the difference evaluation index.

[0018] Obtain the number of confirmed scenarios in the recommended scenario set;

[0019] If the number of identified scenarios is less than the total number of recommended scenarios, then based on the total number of recommended scenarios, the proportion of target scenarios corresponding to each of the difference evaluation indicators, and the recommendation upper limit value corresponding to each of the difference evaluation indicators, the actual upper limit value corresponding to each of the difference evaluation indicators is determined respectively.

[0020] For each difference evaluation index, the target scenarios corresponding to the difference values ​​of each target scenario corresponding to the difference evaluation index, ranked from (N+1) to Mth difference values, are taken as each of the recommended scenarios, and each of the recommended scenarios is added to the recommended scenario set, where M is the actual limit value corresponding to the difference evaluation index.

[0021] In one embodiment, each of the difference evaluation indicators includes a gear level difference indicator;

[0022] The step of determining the difference value of each simulation test scenario under each difference evaluation index based on the first autonomous driving data, the second autonomous driving data, and each pre-set difference evaluation index includes:

[0023] Based on the gear information in the first autonomous driving data, generate the first gear string information corresponding to each of the simulation test scenarios;

[0024] Based on the gear information in the second autonomous driving data, generate the second gear string information corresponding to each of the simulation test scenarios;

[0025] For each simulation test scenario, a first edit distance is calculated between the first gear string information and the second gear string information corresponding to the simulation test scenario, and the first edit distance is used as the difference value of the simulation test scenario under the gear difference index.

[0026] In one embodiment, each of the difference evaluation indicators includes a turn signal difference indicator;

[0027] The step of determining the difference value of each simulation test scenario under each difference evaluation index based on the first autonomous driving data, the second autonomous driving data, and each pre-set difference evaluation index includes:

[0028] Based on the turn signal status in the first autonomous driving data, generate the first turn signal string information corresponding to each of the simulation test scenarios;

[0029] Based on the turn signal status in the second autonomous driving data, generate the second turn signal string information corresponding to each of the simulation test scenarios;

[0030] For each simulation test scenario, a second edit distance is calculated between the first turn signal string information and the second turn signal string information corresponding to the simulation test scenario, and the second edit distance is used as the difference value of the simulation test scenario under the turn signal difference index.

[0031] In one embodiment, each of the difference evaluation indicators includes an acceleration difference indicator and a lateral displacement difference indicator;

[0032] The step of determining the difference value of each simulation test scenario under each difference evaluation index based on the first autonomous driving data, the second autonomous driving data, and each pre-set difference evaluation index includes:

[0033] For each simulation test scenario, the sliding window method is used to process the acceleration of the first autonomous driving data and the acceleration of the second autonomous driving data in the simulation test scenario to obtain the average value of each acceleration difference. The maximum average acceleration difference is taken as the difference value of the simulation test scenario under the acceleration difference index. The sliding window method is also used to process the steering wheel rotation angle of the first autonomous driving data and the steering wheel rotation angle of the second autonomous driving data in the simulation test scenario to obtain the average value of each turning angle difference. The maximum average turning angle difference is taken as the difference value of the simulation test scenario under the lateral movement difference index.

[0034] In one embodiment, each of the difference evaluation indicators includes a driving trajectory difference indicator;

[0035] The step of determining the difference value of each simulation test scenario under each difference evaluation index based on the first autonomous driving data, the second autonomous driving data, and each pre-set difference evaluation index includes:

[0036] For each simulation test scenario, the Euclidean distance is calculated based on the driving position in the first autonomous driving data corresponding to the simulation test scenario and the driving position in the second autonomous driving data corresponding to the simulation test scenario, and the Euclidean distance is used as the difference value of the simulation test scenario under the driving trajectory difference index.

[0037] Secondly, embodiments of this application provide a simulation test scenario recommendation device, the device comprising:

[0038] An autonomous driving data acquisition module is used to acquire first autonomous driving data output by a first autonomous driving algorithm in each simulation test scenario, and second autonomous driving data output by a second autonomous driving algorithm in each simulation test scenario;

[0039] The difference determination module is used to determine the difference value of each simulation test scenario under each difference evaluation index based on the first autonomous driving data, the second autonomous driving data and each pre-set difference evaluation index;

[0040] The target scenario determination module is used to select the simulation test scenarios with a large degree of difference from each of the simulation test scenarios as the target scenarios corresponding to the difference evaluation index, based on the difference values ​​corresponding to the difference evaluation index.

[0041] The recommended scenario determination module is used to determine a set of recommended scenarios based on the target scenarios corresponding to each of the difference evaluation indicators; the set of recommended scenarios includes at least one of the target scenarios.

[0042] Thirdly, embodiments of this application provide a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the simulation test scenario recommendation method described in any of the above embodiments.

[0043] Fourthly, embodiments of this application provide a computer device, including: one or more processors, and a memory;

[0044] The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the simulation test scenario recommendation method described in any of the above embodiments.

[0045] In the simulation testing recommendation method, apparatus, storage medium, and computer equipment of this application, the computer equipment can use the same simulation testing scenario to conduct simulation tests on the first autonomous driving algorithm and the second autonomous driving algorithm. Based on pre-set difference evaluation indicators, the first autonomous driving data output by the first autonomous driving algorithm, and the second autonomous driving data output by the second autonomous driving algorithm, the computer equipment determines the difference values ​​between the first and second autonomous driving algorithms under each simulation testing scenario and each difference evaluation indicator. For each difference evaluation indicator, the computer equipment can select simulation testing scenarios with sufficiently large differences from each simulation testing scenario as the target scenario corresponding to that difference evaluation indicator, based on the difference values ​​corresponding to that indicator. After determining the target scenarios corresponding to each difference evaluation indicator, the computer equipment can determine the recommended scenarios included in the recommended scenario set within each target scenario. In this way, the computer equipment can automatically select the simulation testing scenarios most likely to show version changes from a large number of simulation testing scenarios, and obtain a recommended scenario set accordingly, facilitating manual verification of a small number of scenarios in the recommended scenario set by engineers. This ensures that the recommended simulation testing scenarios accurately reflect algorithm version changes while significantly reducing the number of simulation scenarios requiring manual verification, thereby reducing the time consumed by manual verification and improving testing efficiency. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating a simulation test scenario recommendation method in one embodiment;

[0048] Figure 2 This is a flowchart illustrating the steps for determining the recommended scenario set in one embodiment;

[0049] Figure 3 This is a flowchart illustrating the steps for selecting K recommended scenarios in one embodiment;

[0050] Figure 4 This is one of the flowcharts illustrating the steps for determining the difference value of each simulation test scenario under each of the difference evaluation indicators in one embodiment;

[0051] Figure 5 This is a second flowchart illustrating the steps for determining the difference value of each simulation test scenario under each difference evaluation index in one embodiment.

[0052] Figure 6 This is a schematic diagram of the structure of a simulation test scenario recommendation device in one embodiment;

[0053] Figure 7 This is a schematic diagram of the structure of a computer device in one embodiment. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] In one embodiment, this application provides a method for recommending simulation test scenarios. The following embodiments illustrate this method applied to a computer device. It is understood that the computing device refers to a device with data processing capabilities, and may be, but is not limited to, a personal laptop, a desktop laptop, a single server, or a server cluster. Figure 1 As shown, the method of this application may include the following steps:

[0056] S102: Obtain the first autonomous driving data output by the first autonomous driving algorithm in each simulation test scenario, and the second autonomous driving data output by the second autonomous driving algorithm in each simulation test scenario.

[0057] The first and second autonomous driving algorithms can be different versions of the autonomous driving algorithm. During A / B testing, the computer equipment can use the same set of simulation test scenarios to simulate and test the different versions of the autonomous driving algorithm. This set of simulation test scenarios can include multiple simulation test scenarios. The computer equipment uses each simulation test scenario to simulate and test the first autonomous driving algorithm, and also uses each simulation test scenario to simulate and test the second autonomous driving algorithm, to obtain the first and second autonomous driving data output by the first and second autonomous driving algorithms for the same set of simulation test scenarios.

[0058] The first autonomous driving data reflects the behavioral state and / or driving behavior of the simulated vehicle controlled by the first autonomous driving algorithm in each simulated test scenario. Similarly, the second autonomous driving data reflects the behavioral state and / or driving behavior of the simulated vehicle controlled by the second autonomous driving algorithm in each simulated test scenario. The first and second autonomous driving data can include the same type of driving data; for example, both the first and second autonomous driving data can include trajectory data, acceleration data, steering wheel angle information, gear information, and turn signal information.

[0059] S104: Based on the first autonomous driving data, the second autonomous driving data, and the pre-set differential evaluation indicators, determine the difference value of each simulation test scenario under each differential evaluation indicator.

[0060] Among them, the difference evaluation index can be an index used to evaluate the difference in behavioral state and / or driving behavior between the first autonomous driving algorithm and the second autonomous driving algorithm in the same simulation test scenario.

[0061] Specifically, one or more difference evaluation indicators can be pre-set. For each simulation test scenario, the computer device can extract the autonomous driving data output by the first autonomous driving algorithm in that simulation test scenario from the first autonomous driving data, and extract the autonomous driving data output by the second autonomous driving algorithm in that simulation test scenario from the second autonomous driving data. Based on the autonomous driving data output by different versions of the autonomous driving algorithm in the same simulation test scenario, the difference value of that simulation test scenario under each difference evaluation indicator is determined. This difference value is used to reflect the degree of difference in the behavioral state and / or driving behavior between the first and second autonomous driving algorithms in the same simulation test scenario. It is understood that for any two difference evaluation indicators, the computer device can obtain the difference value of the simulation test scenario under each difference evaluation indicator by using the same or different calculation methods, and this application does not impose specific limitations in this regard.

[0062] S106: For each difference evaluation index, based on the difference values ​​corresponding to the difference evaluation index, select the simulation test scenarios with a large degree of difference from each of the simulation test scenarios as the target scenarios corresponding to the difference evaluation index.

[0063] Following step S104, the computer device obtains the difference values ​​for each simulation test scenario under various difference evaluation indicators. Each difference evaluation indicator can correspond to multiple difference values, each reflecting the degree of difference between the simulation test scenarios under the same difference evaluation indicator. For each difference evaluation indicator, the computer device can determine the degree of difference between the first and second autonomous driving algorithms under each simulation test scenario and the corresponding difference evaluation indicator based on the difference values ​​of each simulation test scenario under that indicator. Based on this, the computer device selects the simulation test scenarios with the largest degree of difference under that difference evaluation indicator as the target scenarios corresponding to that difference evaluation indicator.

[0064] It is understood that computer devices can use any method to determine the degree of difference between each simulation test scenario based on various difference values. In one embodiment, for each difference evaluation index, the computer device can obtain the difference threshold corresponding to that difference evaluation index, and take the simulation test scenario with a difference value greater than the difference threshold as the target scenario corresponding to that difference evaluation index. Furthermore, the difference threshold corresponding to each difference evaluation index can be set based on human experience.

[0065] S108: Determine a set of recommended scenarios based on the target scenarios corresponding to each of the aforementioned difference evaluation indicators; the set of recommended scenarios includes at least one of the aforementioned target scenarios.

[0066] After obtaining the target scenarios corresponding to each difference evaluation index, the computer device can identify noteworthy recommended scenarios among the target scenarios corresponding to each difference evaluation index, thereby obtaining a set of recommended scenarios including each recommended scenario. In one embodiment, the computer device can use some or all of the target scenarios as recommended scenarios to obtain a set of recommended scenarios.

[0067] In one embodiment, after determining the set of recommended scenarios, the computer device can recommend various recommended scenarios in the set to the engineer so that the engineer can manually verify each recommended scenario and evaluate the merits of the first autonomous driving algorithm and the second autonomous driving algorithm.

[0068] In this application, a computer device can automatically select the simulation test scenarios most likely to reflect version changes from a large number of simulation test scenarios, and obtain a recommended scenario set accordingly. This allows engineers to manually verify a small number of scenarios within the recommended scenario set. This ensures that the recommended simulation test scenarios accurately reflect algorithm version changes while significantly reducing the number of simulation scenarios requiring manual verification, thereby reducing the time consumed by manual verification and improving testing efficiency.

[0069] In one embodiment, such as Figure 2 As shown, the step of determining the recommended scenario set based on the target scenarios corresponding to each of the aforementioned difference evaluation indicators includes:

[0070] S202: Sum the number of target scenarios corresponding to each of the aforementioned difference evaluation indicators to obtain the total number of target scenarios.

[0071] Among them, the target scenario indicator corresponding to the difference evaluation indicator refers to the number of target scenarios corresponding to that difference evaluation indicator, and the total number of target scenarios refers to the total number of target scenarios corresponding to all difference evaluation indicators. Specifically, each difference evaluation indicator may correspond to one or more target scenarios. The computer equipment can determine the number of target scenarios corresponding to each difference evaluation indicator separately, and accumulate the numbers of each target scenario to obtain the total number of target scenarios.

[0072] S204: Based on the total number of target scenarios and the number of target scenarios corresponding to each difference evaluation index, determine the proportion of target scenarios corresponding to each difference evaluation index.

[0073] The target scenario percentage corresponding to the difference evaluation index refers to the proportion of the number of target scenarios corresponding to that difference evaluation index to the total number of target scenarios. For each difference evaluation index, the computer device can calculate the ratio between the number of target scenarios corresponding to that difference evaluation index and the total number of target scenarios, and use this ratio as the target scenario percentage corresponding to that difference evaluation index.

[0074] S206: Get the total number of pre-set recommended scenarios.

[0075] The total number of recommended scenarios refers to the pre-defined total number of simulation test scenarios that engineers wish to focus on. In other words, the computer equipment can identify K significantly different simulation test scenarios from P simulation test scenarios as recommended scenarios. Here, P and K are both positive integers, with P greater than K, and K being the total number of recommended scenarios.

[0076] S208: Based on the total number of recommended scenarios and the proportion of target scenarios corresponding to each of the difference evaluation indicators, select K recommended scenarios from the target scenarios corresponding to each of the difference evaluation indicators to obtain a set of recommended scenarios including K recommended scenarios; where K is the total number of recommended scenarios.

[0077] Specifically, the computer device can determine recommended scenarios by combining the proportion of target scenarios corresponding to each difference evaluation index, thereby obtaining a set of recommended scenarios. This set of recommended scenarios can reflect the proportion of simulation test scenarios (i.e., target scenarios) that show significant differences for each difference evaluation index. For example, for each difference evaluation index, the computer device can calculate the product Q between the total number of recommended scenarios and the proportion of target scenarios corresponding to that difference evaluation index, and select Q target scenarios from the target scenarios corresponding to that difference evaluation index as recommended scenarios.

[0078] In this embodiment, recommended scenarios are selected from the target scenarios corresponding to each difference evaluation index based on the total number of recommended scenarios and the proportion of target scenarios corresponding to each difference evaluation index. This ensures that the set of recommended scenarios reflects the proportion of simulation test scenarios with significant differences in each difference evaluation index, which facilitates manual verification based on this proportion and further improves testing efficiency.

[0079] In one embodiment, the difference value is positively correlated with the degree of difference. In other words, the larger the difference value, the greater the degree of difference.

[0080] like Figure 3 As shown, the step of selecting K recommended scenarios from the target scenarios corresponding to each of the difference evaluation indicators based on the total number of recommended scenarios and the proportion of target scenarios corresponding to each of the difference evaluation indicators to obtain a set of recommended scenarios including K recommended scenarios includes:

[0081] S302: Obtain the recommended upper limit and recommended lower limit values ​​corresponding to each of the pre-set difference evaluation indicators.

[0082] Each difference evaluation indicator has its corresponding recommended upper limit and recommended lower limit. It should be noted that the recommended upper limit for each difference evaluation indicator can be the same or different. Similarly, the recommended lower limit for each difference evaluation indicator can be the same or different. This application does not impose specific restrictions on this.

[0083] S304: For each difference evaluation index, sort the difference values ​​of each target scenario corresponding to the difference evaluation index in descending order, and take the target scenarios corresponding to the first to Nth difference values ​​as each of the recommended scenarios, and add each of the recommended scenarios to the recommended scenario set, where N is the recommendation lower limit value corresponding to the difference evaluation index.

[0084] Specifically, for each difference evaluation index, the computer device can sort the difference values ​​corresponding to that index in descending order. Since the difference value is positively correlated with the degree of difference, the larger the difference value, the greater the degree of difference, and the higher the ranking. After sorting, the computer device can select the target scenarios corresponding to the top N difference values ​​under each difference evaluation index as recommended scenarios. In this way, the N target scenarios with the largest differences in each difference evaluation index can be selected into the recommended scenario set.

[0085] S306: Obtain the number of determined scenarios in the recommended scenario set.

[0086] The number of confirmed scenarios can be the number of recommended scenarios already included in the set of recommended scenarios, that is, the number of recommended scenarios that have been selected.

[0087] S308: If the number of determined scenarios is less than the total number of recommended scenarios, then based on the total number of recommended scenarios, the proportion of target scenarios corresponding to each of the difference evaluation indicators, and the recommendation upper limit value corresponding to each of the difference evaluation indicators, the actual upper limit value corresponding to each of the difference evaluation indicators is determined respectively.

[0088] Generally, the total number of recommended scenarios is greater than the sum of the recommended lower limits corresponding to each difference evaluation metric. After selecting N target scenarios corresponding to each difference evaluation metric into the recommended scenario set, the number of confirmed scenarios in the recommended scenario set is generally less than the total number of recommended scenarios. Therefore, it is still necessary to filter recommended scenarios from the target scenarios that have not yet been selected into the recommended scenario set.

[0089] In this scenario, for each difference evaluation indicator, the computer device can determine the actual limit value corresponding to the difference evaluation indicator based on the total number of recommended scenarios, the proportion of target scenarios corresponding to the difference evaluation indicator, and the recommendation upper limit value corresponding to the difference evaluation indicator. In one embodiment, the computer device can calculate the product Q between the total number of recommended scenarios and the proportion of target scenarios corresponding to the difference evaluation indicator, and take the smaller value between Q and the recommendation upper limit value corresponding to the difference evaluation indicator as the actual limit value corresponding to the difference evaluation indicator.

[0090] S310: For each difference evaluation index, among the difference values ​​of each target scenario corresponding to the difference evaluation index, the target scenarios corresponding to the (N+1)th to Mth difference values ​​are taken as each of the recommended scenarios, and each of the recommended scenarios is added to the recommended scenario set, where M is the actual limit value corresponding to the difference evaluation index.

[0091] After determining the actual limit value M corresponding to each difference evaluation index, the computer device can select the target scenarios corresponding to the (N+1)th to Mth difference values ​​in the sorted order for each difference evaluation index as recommended scenarios. For example, for a certain difference evaluation index, N is 5 and M is 15, then the target scenarios corresponding to the 6th to 15th (a total of 10) difference values ​​in the sorted order can be selected as recommended scenarios.

[0092] In this embodiment, considering that the number of target scenarios corresponding to different difference evaluation indicators may be unbalanced, this application first selects the N target scenarios with the largest differences among each difference evaluation indicator into the recommended scenario set, and then allocates the remaining recommended scenarios according to the proportion of target scenarios corresponding to each difference evaluation indicator. In this way, it can avoid a particularly unbalanced number of recommended scenarios corresponding to each difference evaluation indicator, and can reflect the original proportion as much as possible, so as to facilitate engineers' subsequent manual verification, thereby further improving testing efficiency.

[0093] In one embodiment, each of the difference evaluation indicators includes a gear difference indicator. That is, among the various difference evaluation indicators, one of them is a gear difference indicator. The gear difference indicator refers to an indicator used to evaluate the difference in vehicle gears between a simulated vehicle controlled by a first autonomous driving algorithm and a simulated vehicle controlled by a second autonomous driving algorithm in the same simulation test scenario.

[0094] like Figure 4 As shown, the step of determining the difference value of each simulation test scenario under each difference evaluation index based on the first autonomous driving data, the second autonomous driving data, and each pre-set difference evaluation index includes:

[0095] S402: Based on the gear information in the first autonomous driving data, generate the first gear string information corresponding to each of the simulation test scenarios.

[0096] The first gear position string information corresponding to each simulation test scenario reflects the vehicle gear position information of the simulated vehicle controlled by the first autonomous driving algorithm in each simulation frame of that simulation test scenario. In one embodiment, for each simulation test scenario, the computer device can obtain the gear position information of that simulation test scenario in each simulation frame from the gear position information of the first autonomous driving data, and concatenate the gear position information corresponding to each simulation frame into a string to obtain the first gear position string information. For example, the first gear position string information can be PPPPDDDD.

[0097] S404: Based on the gear information in the second autonomous driving data, generate the second gear string information corresponding to each of the simulation test scenarios.

[0098] The second gear position string information corresponding to each simulation test scenario can reflect the vehicle gear position information of the simulated vehicle controlled by the second autonomous driving algorithm in each simulation frame of that simulation test scenario. Furthermore, the process of obtaining the second gear position string information corresponding to each simulation test scenario by the computer device is similar to the process of obtaining the first gear position string information, and will not be elaborated here.

[0099] S406: For each of the simulation test scenarios, calculate the first edit distance between the first gear string information and the second gear string information corresponding to the simulation test scenario, and use the first edit distance as the difference value of the simulation test scenario under the gear difference index.

[0100] Edit distance is a quantitative measure of the difference between two strings (such as English strings). It measures the minimum number of processing steps required to transform one string into another.

[0101] The computer device can calculate the edit distance (first edit distance) between the first string of gear position information and the second string of gear position information corresponding to the same simulation test scenario, thereby obtaining the gear position difference between the first autonomous driving algorithm and the second autonomous driving algorithm in the same simulation test scenario. The computer device can use the calculated first edit distance as the difference value of the simulation test scenario under the gear position difference index.

[0102] In this embodiment, by setting a gear difference index among various difference evaluation indicators, and using the edit distance between the first gear string information and the second gear string information as the difference value of the simulation test scenario under the gear difference index, the difference between the first autonomous driving algorithm and the second autonomous driving algorithm for the same simulation test scenario can be accurately evaluated from multiple dimensions, so as to accurately identify the change scenarios worthy of attention in multiple simulation test scenarios in the future.

[0103] In one embodiment, each of the difference evaluation metrics includes a turn signal difference metric. That is, among the various difference evaluation metrics, one of them is a turn signal difference metric. This turn signal difference metric refers to an indicator used to evaluate the difference in turn signals between a simulated vehicle controlled by a first autonomous driving algorithm and a simulated vehicle controlled by a second autonomous driving algorithm in the same simulation test scenario.

[0104] like Figure 5 As shown, the step of determining the difference value of each simulation test scenario under each difference evaluation index based on the first autonomous driving data, the second autonomous driving data, and each pre-set difference evaluation index includes:

[0105] S502: Based on the turn signal status in the first autonomous driving data, generate the first turn signal string information corresponding to each of the simulation test scenarios.

[0106] The first turn signal string information corresponding to each simulation test scenario reflects the vehicle turn signal information of the simulated vehicle controlled by the first autonomous driving algorithm in each simulation frame of that simulation test scenario. In one embodiment, for each simulation test scenario, the computer device can obtain the turn signal status of the simulation test scenario in each simulation frame from the turn signal status of the first autonomous driving data, and concatenate the turn signal statuses corresponding to each simulation frame into a string to obtain the first turn signal string information. For example, the first turn signal string information can be LLLLNNNRRR, where L represents the left turn signal is on, N represents the left and right turn signals are not on, and R represents the right turn signal is on.

[0107] S504: Based on the turn signal status in the second autonomous driving data, generate the second turn signal string information corresponding to each of the simulation test scenarios.

[0108] The second turn signal string information corresponding to each simulation test scenario can reflect the vehicle turn signal information of the simulated vehicle controlled by the second autonomous driving algorithm in each simulation frame of that simulation test scenario. Furthermore, the process by which the computer device obtains the second turn signal string information corresponding to each simulation test scenario is similar to the process of obtaining the first turn signal string information, and will not be elaborated further here.

[0109] S506: For each of the simulation test scenarios, calculate the second edit distance between the first turn signal string information corresponding to the simulation test scenario and the second turn signal string information corresponding to the simulation test scenario, and use the second edit distance as the difference value of the simulation test scenario under the turn signal difference index.

[0110] For a detailed explanation of the edit distance, please refer to the above embodiments, which will not be repeated here. The computer device can calculate the edit distance (second edit distance) between the first turn signal string information and the second turn signal string information corresponding to the same simulation test scenario, thereby obtaining the difference in turn signal state between the first autonomous driving algorithm and the second autonomous driving algorithm in the same simulation test scenario. The computer device can use the calculated second edit distance as the difference value of the simulation test scenario under the turn signal difference index.

[0111] In this embodiment, by setting a turn signal difference index among various difference evaluation indicators, and using the edit distance between the first turn signal string information and the second turn signal string information as the difference value of the simulation test scenario under the turn signal difference index, the difference between the first autonomous driving algorithm and the second autonomous driving algorithm for the same simulation test scenario can be accurately evaluated from multiple dimensions, so as to accurately identify the change scenarios worthy of attention in multiple simulation test scenarios in the future.

[0112] In one embodiment, each of the aforementioned difference evaluation indicators includes an acceleration difference indicator and a lateral displacement difference indicator. It should be noted that, in addition to the acceleration difference indicator and the lateral displacement difference indicator, more difference evaluation indicators may be included, and this document does not impose any limitations on this.

[0113] The acceleration difference index can be used to evaluate the acceleration difference between a simulated vehicle controlled by a first autonomous driving algorithm and a simulated vehicle controlled by a second autonomous driving algorithm in the same simulation test scenario. Similarly, the lateral movement difference index can be used to evaluate the lateral movement difference between a simulated vehicle controlled by a first autonomous driving algorithm and a simulated vehicle controlled by a second autonomous driving algorithm in the same simulation test scenario.

[0114] The step of determining the difference value of each simulation test scenario under each difference evaluation index based on the first autonomous driving data, the second autonomous driving data, and each pre-set difference evaluation index includes:

[0115] For each simulation test scenario, the sliding window method is used to process the acceleration of the first autonomous driving data and the acceleration of the second autonomous driving data in the simulation test scenario to obtain the average value of each acceleration difference. The maximum average acceleration difference is taken as the difference value of the simulation test scenario under the acceleration difference index. The sliding window method is also used to process the steering wheel rotation angle of the first autonomous driving data and the steering wheel rotation angle of the second autonomous driving data in the simulation test scenario to obtain the average value of each turning angle difference. The maximum average turning angle difference is taken as the difference value of the simulation test scenario under the lateral movement difference index.

[0116] Specifically, when using the sliding window method to process the acceleration of the first and second autonomous driving algorithms in the same simulation test scenario, a sliding window of a pre-set length can be set, and this fixed-length sliding window can be moved from the first simulation frame to the last simulation frame of the simulation test scenario. For each sliding window, the acceleration difference in each simulation frame within that sliding window is calculated, and the average acceleration difference corresponding to that sliding window is obtained. After obtaining the average acceleration differences for each simulation test scenario, the computer device uses the maximum average acceleration difference as the difference value of the simulation test scenario under the acceleration difference index.

[0117] Taking a window length of 1 second and 10 simulation frames per second as an example, as shown in the following formula, for each simulation test scenario, the computer device can use a sliding window with a length of 1 second to calculate the average difference in vehicle acceleration (i.e., the average value of acceleration difference) corresponding to 10 simulation frames, and select the maximum average value of acceleration difference in all sliding windows of the simulation test scenario.

[0118]

[0119]

[0120] In the formula, DIST accel (t0) represents the average difference in acceleration at time t0. o(t) Let accel be the acceleration at time t output by the first autonomous driving algorithm. s(t) MAXDIST represents the acceleration at time t, output by the second autonomous driving algorithm. accel This represents the average value of the maximum acceleration difference.

[0121] Similarly, when using the sliding window method to process the steering wheel rotation angle of the first and second autonomous driving algorithms in the same simulation test scenario, the specific implementation process can refer to the above process of using the sliding window method to process acceleration, and will not be repeated here.

[0122] The computer equipment can use the sliding window method to process the steering wheel rotation angle of the first autonomous driving algorithm and the second autonomous driving algorithm in the same simulation test scenario, so as to obtain the average value of the differences of each angle, and take the average value of the maximum rotation angle difference as the difference value of the simulation test scenario under the lateral movement difference index.

[0123] In this embodiment, the average value of acceleration difference and average value of turning angle difference under the same simulation test scenario are calculated by using the sliding window method. The average value of the maximum acceleration difference and the average value of the maximum turning angle difference among the average values ​​of acceleration difference are taken as the difference values ​​of the simulation test scenario under the acceleration difference index and the lateral movement difference index. This allows for an accurate evaluation of the differences between the first autonomous driving algorithm and the second autonomous driving algorithm for the same simulation test scenario from multiple dimensions, so as to accurately identify the change scenarios worth paying attention to in multiple simulation test scenarios in the future.

[0124] In one embodiment, each of the difference evaluation indicators includes a driving trajectory difference indicator. That is, among the various difference evaluation indicators, one of them is a driving trajectory difference indicator. This driving trajectory difference indicator refers to an indicator used to evaluate the driving trajectories of a simulated vehicle controlled by a first autonomous driving algorithm and a simulated vehicle controlled by a second autonomous driving algorithm in the same simulation test scenario.

[0125] The step of determining the difference value of each simulation test scenario under each difference evaluation index based on the first autonomous driving data, the second autonomous driving data, and each pre-set difference evaluation index includes:

[0126] For each simulation test scenario, the Euclidean distance is calculated based on the driving position in the first autonomous driving data corresponding to the simulation test scenario and the driving position in the second autonomous driving data corresponding to the simulation test scenario, and the Euclidean distance is used as the difference value of the simulation test scenario under the driving trajectory difference index.

[0127] In one embodiment, for each simulation test scenario, the computer device can calculate the Euclidean distance between the first autonomous driving algorithm and the second autonomous driving algorithm in that simulation test scenario according to the following formula:

[0128]

[0129] In the formula, DIST traj For Euclidean distance, t start t represents the simulation time corresponding to the first simulation frame of this simulation test scenario. end pos represents the simulation time corresponding to the last simulation frame of this simulation test scenario. o(t) .x represents the x-coordinate of the simulated vehicle controlled by the first autonomous driving algorithm at time t, and pos represents the x-coordinate of the vehicle at time t. s(t) .x represents the x-coordinate of the simulated vehicle controlled by the second autonomous driving algorithm at time t, and pos represents the x-coordinate of the vehicle at time t. o(t) .y represents the y-coordinate of the simulated vehicle controlled by the first autonomous driving algorithm at time t, and pos represents the y-coordinate of the vehicle at time t. s(t).y represents the y-coordinate of the simulated vehicle controlled by the second autonomous driving algorithm at time t, and n represents the total number of simulation frames in this simulation test scenario.

[0130] In this embodiment, the differences between the first autonomous driving algorithm and the second autonomous driving algorithm for the same simulation test scenario can be accurately evaluated from multiple dimensions, so as to accurately identify noteworthy change scenarios in multiple simulation test scenarios in the future.

[0131] The simulation test scenario recommendation device provided in the embodiments of this application is described below. The simulation test scenario recommendation device described below can be referred to in correspondence with the simulation test scenario recommendation method described above.

[0132] In one embodiment, this application also provides a simulation test scenario recommendation device 600. For example... Figure 6 As shown, the device 600 includes an autonomous driving data acquisition module 610, a difference determination module 620, a target scene determination module 630, and a recommended scene determination module 640. Wherein:

[0133] The autonomous driving data acquisition module 610 is used to acquire first autonomous driving data output by the first autonomous driving algorithm in each simulation test scenario, and second autonomous driving data output by the second autonomous driving algorithm in each simulation test scenario.

[0134] The difference determination module 620 is used to determine the difference value of each simulation test scenario under each difference evaluation index based on the first autonomous driving data, the second autonomous driving data and each pre-set difference evaluation index;

[0135] The target scenario determination module 630 is used to select the simulation test scenario with a large degree of difference from each of the simulation test scenarios as the target scenario corresponding to the difference evaluation index, based on the difference values ​​corresponding to the difference evaluation index.

[0136] The recommended scenario determination module 640 is used to determine a set of recommended scenarios based on the target scenarios corresponding to each of the difference evaluation indicators; the set of recommended scenarios includes at least one of the target scenarios.

[0137] In one embodiment, the recommended scenario determination module 640 includes an accumulation unit, a percentage calculation unit, a total quantity acquisition unit, and a recommendation unit. The accumulation unit accumulates the number of target scenarios corresponding to each of the difference evaluation indicators to obtain the total number of target scenarios. The percentage calculation unit determines the percentage of each target scenario corresponding to each difference evaluation indicator based on the total number of target scenarios and the number of target scenarios corresponding to each difference evaluation indicator. The total quantity acquisition unit acquires a pre-set total number of recommended scenarios. The recommendation unit selects K recommended scenarios from the target scenarios corresponding to each difference evaluation indicator based on the total number of recommended scenarios and the percentage of target scenarios corresponding to each difference evaluation indicator, to obtain a set of recommended scenarios including K recommended scenarios; where K is the total number of recommended scenarios.

[0138] In one embodiment, the difference value is positively correlated with the degree of difference. The recommendation unit includes a limit value acquisition unit, a first selection unit, a determined scenario quantity determination unit, an actual limit value determination unit, and a second selection unit. The limit value acquisition unit acquires a pre-set upper and lower recommendation limit value for each difference evaluation indicator. The first selection unit sorts the difference values ​​of each target scenario corresponding to each difference evaluation indicator in descending order, and adds the target scenarios corresponding to the first to Nth difference values ​​as recommended scenarios to the recommended scenario set, where N is the lower recommendation limit value corresponding to the difference evaluation indicator. The determined scenario quantity determination unit acquires the number of determined scenarios in the recommended scenario set. The actual limit value determination unit determines the actual limit value for each difference evaluation indicator based on the total number of recommended scenarios, the proportion of target scenarios corresponding to each difference evaluation indicator, and the upper recommendation limit value corresponding to each difference evaluation indicator if the number of determined scenarios is less than the total number of recommended scenarios. The second selection unit is used to select the target scenarios corresponding to the (N+1)th to Mth difference values ​​of each target scenario for each difference evaluation index as each recommended scenario, and add each recommended scenario to the recommended scenario set, where M is the actual limit value corresponding to the difference evaluation index.

[0139] In one embodiment, each of the difference evaluation indicators includes a gear difference indicator. The difference determination module 620 includes a first gear information determination unit, a second gear information determination unit, and a first difference value determination unit. The first gear information determination unit is used to generate first gear string information corresponding to each simulation test scenario based on the gear information in the first autonomous driving data. The second gear information determination unit is used to generate second gear string information corresponding to each simulation test scenario based on the gear information in the second autonomous driving data. The first difference value determination unit is used to calculate a first edit distance between the first gear string information and the second gear string information corresponding to each simulation test scenario, and use the first edit distance as the difference value of the simulation test scenario under the gear difference indicator.

[0140] In one embodiment, each of the difference evaluation indicators includes a turn signal difference indicator. The difference determination module 620 includes a first turn signal information determination unit, a second turn signal information determination unit, and a second difference value determination unit. The first turn signal information determination unit is used to generate first turn signal string information corresponding to each simulation test scenario based on the turn signal status in the first autonomous driving data. The second turn signal information determination unit is used to generate second turn signal string information corresponding to each simulation test scenario based on the turn signal status in the second autonomous driving data. The second difference value determination unit is used to calculate a second edit distance between the first turn signal string information and the second turn signal string information corresponding to each simulation test scenario, and use the second edit distance as the difference value of the simulation test scenario under the turn signal difference indicator.

[0141] In one embodiment, each of the difference evaluation indicators includes an acceleration difference indicator and a lateral movement difference indicator. The difference determination module 620 includes a third difference value determination unit. This third difference value determination unit is used to process the acceleration of the first autonomous driving data and the acceleration of the second autonomous driving data in the simulation test scenario using a sliding window method for each simulation test scenario, to obtain an average value of each acceleration difference, and to use the maximum average acceleration difference as the difference value of the simulation test scenario under the acceleration difference indicator; and to process the steering wheel rotation angle of the first autonomous driving data and the steering wheel rotation angle of the second autonomous driving data in the simulation test scenario using a sliding window method, to obtain an average value of each angle difference, and to use the maximum average angle difference as the difference value of the simulation test scenario under the lateral movement difference indicator.

[0142] In one embodiment, each of the difference evaluation indicators includes a driving trajectory difference indicator. The difference determination module 620 includes a fourth difference value determination unit. This fourth difference value determination unit is used to calculate a Euclidean distance for each simulation test scenario based on the driving position in the first autonomous driving data corresponding to the simulation test scenario and the driving position in the second autonomous driving data corresponding to the simulation test scenario, and use the Euclidean distance as the difference value of the simulation test scenario under the driving trajectory difference indicator.

[0143] In one embodiment, this application also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the simulation test scenario recommendation method described in any of the above embodiments.

[0144] In one embodiment, this application also provides a computer device. The computer device stores computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the simulation test scenario recommendation method described in any of the above embodiments.

[0145] Indicatively, Figure 7 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of this application. In one example, the computer device can be a server. (Refer to...) Figure 7 The computer device 900 includes a processing component 902, which further includes one or more processors, and memory resources represented by memory 901 for storing instructions, such as application programs, that can be executed by the processing component 902. The application programs stored in memory 901 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 902 is configured to execute instructions to perform the steps of the simulation test scenario recommendation method described in any of the above embodiments.

[0146] The computer device 900 may also include a power supply component 903 configured to perform power management of the computer device 900, a wired or wireless network interface 904 configured to connect the computer device 900 to a network, and an input / output (I / O) interface 905. The computer device 900 may operate on an operating system stored in memory 901, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or similar.

[0147] Those skilled in the art will understand that the internal structure of the computer device shown in this application is merely a block diagram of a portion of the structure related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0148] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. In this document, "a," "an," "the," "the," and "its" may also include plural forms unless the context clearly indicates otherwise. "Multiple" refers to at least two, such as 2, 3, 5, or 8, etc. "And / or" includes any and all combinations of the related listed items.

[0149] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0150] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for recommending simulation test scenarios, characterized in that, The method includes: Acquire first autonomous driving data output by the first autonomous driving algorithm in each simulation test scenario, and second autonomous driving data output by the second autonomous driving algorithm in each simulation test scenario; Based on the first autonomous driving data, the second autonomous driving data, and the pre-set difference evaluation indicators, determine the difference value of each simulation test scenario under each difference evaluation indicator; For each difference evaluation index, based on the difference values ​​corresponding to the difference evaluation index, the simulation test scenarios with a large degree of difference are selected from each of the simulation test scenarios as the target scenarios corresponding to the difference evaluation index. Based on the target scenarios corresponding to each of the aforementioned difference evaluation indicators, a set of recommended scenarios is determined; the set of recommended scenarios includes at least one of the aforementioned target scenarios; specifically, it includes: The number of target scenarios corresponding to each of the aforementioned difference evaluation indicators is summed to obtain the total number of target scenarios; Based on the total number of target scenarios and the number of target scenarios corresponding to each difference evaluation indicator, the proportion of target scenarios corresponding to each difference evaluation indicator is determined respectively; Get the total number of pre-set recommended scenarios; Based on the total number of recommended scenarios and the proportion of target scenarios corresponding to each of the difference evaluation indicators, K recommended scenarios are selected from the target scenarios corresponding to each of the difference evaluation indicators to obtain a set of recommended scenarios including K recommended scenarios; where K is the total number of recommended scenarios.

2. The simulation test scenario recommendation method according to claim 1, characterized in that, The difference value is positively correlated with the degree of difference; The step of selecting K recommended scenarios from the target scenarios corresponding to each of the difference evaluation indicators based on the total number of recommended scenarios and the proportion of target scenarios corresponding to each of the difference evaluation indicators to obtain a set of recommended scenarios including K recommended scenarios includes: Obtain the recommended upper limit and recommended lower limit values ​​corresponding to each of the pre-set difference evaluation indicators; For each difference evaluation index, the difference values ​​of each target scenario corresponding to the difference evaluation index are sorted in descending order, and the target scenarios corresponding to the first to Nth difference values ​​are taken as each of the recommended scenarios. Each of the recommended scenarios is added to the recommended scenario set, where N is the lower limit value of the recommendation corresponding to the difference evaluation index. Obtain the number of confirmed scenarios in the recommended scenario set; If the number of identified scenarios is less than the total number of recommended scenarios, then based on the total number of recommended scenarios, the proportion of target scenarios corresponding to each of the difference evaluation indicators, and the recommendation upper limit value corresponding to each of the difference evaluation indicators, the actual upper limit value corresponding to each of the difference evaluation indicators is determined respectively. For each difference evaluation index, the target scenarios corresponding to the difference values ​​of each target scenario corresponding to the difference evaluation index, ranked from (N+1) to Mth difference values, are taken as each of the recommended scenarios, and each of the recommended scenarios is added to the recommended scenario set, where M is the actual limit value corresponding to the difference evaluation index.

3. The simulation test scenario recommendation method according to claim 1, characterized in that, Each of the aforementioned difference evaluation indicators includes grade difference indicators; The step of determining the difference value of each simulation test scenario under each difference evaluation index based on the first autonomous driving data, the second autonomous driving data, and each pre-set difference evaluation index includes: Based on the gear information in the first autonomous driving data, generate the first gear string information corresponding to each of the simulation test scenarios; Based on the gear information in the second autonomous driving data, generate the second gear string information corresponding to each of the simulation test scenarios; For each simulation test scenario, a first edit distance is calculated between the first gear string information and the second gear string information corresponding to the simulation test scenario, and the first edit distance is used as the difference value of the simulation test scenario under the gear difference index.

4. The simulation test scenario recommendation method according to claim 1, characterized in that, The aforementioned difference evaluation indicators include the turn signal difference indicator; The step of determining the difference value of each simulation test scenario under each difference evaluation index based on the first autonomous driving data, the second autonomous driving data, and each pre-set difference evaluation index includes: Based on the turn signal status in the first autonomous driving data, generate the first turn signal string information corresponding to each of the simulation test scenarios; Based on the turn signal status in the second autonomous driving data, generate the second turn signal string information corresponding to each of the simulation test scenarios; For each simulation test scenario, a second edit distance is calculated between the first turn signal string information and the second turn signal string information corresponding to the simulation test scenario, and the second edit distance is used as the difference value of the simulation test scenario under the turn signal difference index.

5. The simulation test scenario recommendation method according to any one of claims 1 to 4, characterized in that, The aforementioned difference evaluation indicators include acceleration difference indicators and lateral displacement difference indicators; The step of determining the difference value of each simulation test scenario under each difference evaluation index based on the first autonomous driving data, the second autonomous driving data, and each pre-set difference evaluation index includes: For each simulation test scenario, the sliding window method is used to process the acceleration of the first autonomous driving data and the acceleration of the second autonomous driving data in the simulation test scenario to obtain the average value of each acceleration difference. The maximum average acceleration difference is taken as the difference value of the simulation test scenario under the acceleration difference index. The sliding window method is also used to process the steering wheel rotation angle of the first autonomous driving data and the steering wheel rotation angle of the second autonomous driving data in the simulation test scenario to obtain the average value of each turning angle difference. The maximum average turning angle difference is taken as the difference value of the simulation test scenario under the lateral movement difference index.

6. The simulation test scenario recommendation method according to any one of claims 1 to 4, characterized in that, Each of the aforementioned difference evaluation indicators includes a driving trajectory difference indicator; The step of determining the difference value of each simulation test scenario under each difference evaluation index based on the first autonomous driving data, the second autonomous driving data, and each pre-set difference evaluation index includes: For each simulation test scenario, the Euclidean distance is calculated based on the driving position in the first autonomous driving data corresponding to the simulation test scenario and the driving position in the second autonomous driving data corresponding to the simulation test scenario, and the Euclidean distance is used as the difference value of the simulation test scenario under the driving trajectory difference index.

7. A simulation test scenario recommendation device, characterized in that, The device includes: An autonomous driving data acquisition module is used to acquire first autonomous driving data output by a first autonomous driving algorithm in each simulation test scenario, and second autonomous driving data output by a second autonomous driving algorithm in each simulation test scenario; The difference determination module is used to determine the difference value of each simulation test scenario under each difference evaluation index based on the first autonomous driving data, the second autonomous driving data and each pre-set difference evaluation index; The target scenario determination module is used to select the simulation test scenarios with a large degree of difference from each of the simulation test scenarios as the target scenario corresponding to the difference evaluation index, based on the difference values ​​corresponding to the difference evaluation index. The recommended scenario determination module is used to determine a set of recommended scenarios based on the target scenarios corresponding to each of the difference evaluation indicators; the set of recommended scenarios includes at least one of the target scenarios; specifically, it includes: summing the number of target scenarios corresponding to each of the difference evaluation indicators to obtain a total number of target scenarios; determining the proportion of target scenarios corresponding to each difference evaluation indicator based on the total number of target scenarios and the number of target scenarios corresponding to each difference evaluation indicator; obtaining a pre-set total number of recommended scenarios; and selecting K recommended scenarios from the target scenarios corresponding to each difference evaluation indicator based on the total number of recommended scenarios and the proportion of target scenarios corresponding to each difference evaluation indicator to obtain a set of recommended scenarios including K recommended scenarios; where K is the total number of recommended scenarios.

8. A storage medium, characterized in that, The storage medium stores computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the simulation test scenario recommendation method as described in any one of claims 1 to 6.

9. A computer device, characterized in that, include: One or more processors, and memory; The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the simulation test scenario recommendation method as described in any one of claims 1 to 6.

Images