Automatic driving simulation scene library construction method and device, equipment and storage medium

Abstract

The application discloses an automatic driving simulation scene library construction method and device, equipment and a storage medium. The method comprises the following steps: acquiring real vehicle performance indexes of an automatic driving system on a plurality of historical software versions; acquiring test results of simulation of the plurality of historical software versions using an initial simulation scene library; and optimizing the initial simulation scene library based on the correlation between the test results corresponding to the plurality of historical software versions and the real vehicle performance indexes to obtain a target simulation scene library. The initial simulation scene library is optimized and adjusted based on the correlation between the simulation test results and the real vehicle performance indexes of the plurality of historical software versions, so that the correlation between the simulation test results and the real vehicle performance indexes is maximized, thereby establishing a reliable correlation between simulation and real vehicles, so that the simulation test results are no longer isolated indexes, and the simulation test results can accurately reflect the performance on a real road.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, specifically to a method, apparatus, device, and storage medium for constructing an autonomous driving simulation scenario library. Background Technology

[0002] The development of autonomous driving systems relies heavily on extensive testing and validation. Real-world road testing is costly, time-consuming, and carries safety risks; therefore, simulation-based testing has become an indispensable component. The effectiveness of simulation hinges on whether its scenario library can realistically reflect the complexity and challenges of the real world.

[0003] Currently, the establishment of simulation scenario libraries typically involves extracting and generalizing key scenarios from traffic accident reports and natural driving data, and / or generating scenarios based on logical or random rules.

[0004] However, the scenario library obtained in this way lacks correlation between simulation test results and real vehicle performance. That is, an autonomous driving system that performs well in simulation may perform poorly in real vehicle road tests, which greatly reduces the credibility and guiding value of simulation tests. Summary of the Invention

[0005] The purpose of this application is to provide a method, apparatus, device, and storage medium for constructing an autonomous driving simulation scenario library, addressing the shortcomings of the prior art. This purpose is achieved through the following technical solutions.

[0006] The first aspect of this application proposes a method for constructing an autonomous driving simulation scenario library, the method comprising: Obtain real-vehicle performance metrics for the autonomous driving system across multiple historical software versions; Obtain the test results of simulations performed on the multiple historical software versions using the initial simulation scenario library; Based on the correlation between the test results corresponding to the multiple historical software versions and the actual vehicle performance indicators, the initial simulation scenario library is optimized to obtain the target simulation scenario library.

[0007] A second aspect of this application proposes an apparatus for constructing an autonomous driving simulation scenario library, the apparatus comprising: The real-world testing module is used to obtain real-vehicle performance metrics of the autonomous driving system across multiple historical software versions. The simulation module is used to obtain the test results of the simulation of the multiple historical software versions using the initial simulation scenario library; The library optimization module is used to optimize the initial simulation scenario library based on the correlation between the test results corresponding to the multiple historical software versions and the actual vehicle performance indicators, so as to obtain the target simulation scenario library.

[0008] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the method described in the first aspect above.

[0009] The fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the method described in the first aspect above.

[0010] Based on the above-described method, apparatus, equipment, and storage medium for constructing an autonomous driving simulation scenario library, this application has at least the following beneficial effects or advantages: By acquiring real-vehicle performance metrics for autonomous driving across multiple historical software versions, and simulation test results based on an initial simulation scenario library for multiple historical software versions, and then optimizing and adjusting the initial simulation scenario library based on the correlation between simulation test results and real-vehicle performance metrics across multiple historical software versions, the correlation between simulation test results and real-vehicle performance metrics is maximized. This establishes a reliable link between simulation and real-vehicle performance, ensuring that simulation test results are no longer isolated metrics but accurately reflect performance on real roads.

[0011] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart illustrating an embodiment of a method for constructing an autonomous driving simulation scenario library according to an exemplary embodiment; Figure 2 This is a schematic diagram illustrating the structure of an autonomous driving simulation scene library construction device according to an exemplary embodiment; Figure 3 This is a schematic diagram of the hardware structure of an electronic device according to an exemplary embodiment. Detailed Implementation

[0013] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0014] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0015] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0016] As mentioned earlier, the simulation scenario library generated by traditional solutions lacks a clear and stable statistical correlation between simulation test results and real vehicle performance indicators, resulting in low credibility of simulation test results.

[0017] Based on this, this application proposes a scheme for constructing an autonomous driving simulation scenario library that correlates with real vehicle performance indicators. This enables the simulation test results on the simulation scenario library to accurately reflect and predict the performance of the autonomous driving system on real roads, thereby establishing a reliable correlation between simulation and real vehicles.

[0018] That is, by obtaining real-vehicle performance indicators of autonomous driving in multiple historical software versions, and obtaining simulation test results of multiple historical software versions based on the initial simulation scenario library, and then by optimizing and adjusting the initial simulation scenario library based on the correlation between simulation test results of multiple historical software versions and real-vehicle performance indicators, the correlation between simulation test results and real-vehicle performance indicators is maximized. The simulation test results are no longer isolated indicators, but can accurately reflect the performance on real roads.

[0019] The technical solution of this application and how it solves the aforementioned technical problems are described in detail below with specific embodiments. The listed specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0020] Figure 1 This is a flowchart illustrating an embodiment of a method for constructing an autonomous driving simulation scenario library according to an exemplary embodiment. This embodiment can be applied to autonomous driving scenarios such as autonomous driving systems and driver assistance systems. The method can be executed by an autonomous driving simulation scenario library construction device, which can be implemented in hardware and / or software. This device can be configured in any electronic device with network communication and computing capabilities. Figure 1 As shown, the method includes the following steps: Step 101: Obtain real-vehicle performance metrics for the autonomous driving system across multiple historical software versions.

[0021] Step 102: Obtain the test results of simulations performed on multiple historical software versions using the initial simulation scenario library.

[0022] Step 103: Based on the correlation between the test results of multiple historical software versions and the performance indicators of real vehicles, optimize the initial simulation scenario library to obtain the target simulation scenario library.

[0023] Real-world performance metrics measure the actual performance of an autonomous driving system in real-world road environments, providing a realistic and representative benchmark for building a simulation scenario library. Common real-world performance metrics include Mean Intervention Mileage (MPI), Mean Intervention Critical Intervention Mileage (MPCI), Navigation Intervention MPI, and Regulatory Intervention MPI. These metrics reflect the average distance the autonomous driving system can autonomously travel between two manual interventions; higher values ​​indicate better system performance.

[0024] Autonomous driving systems differ in functionality, algorithms, and performance across different historical software versions. By using data from multiple historical software versions, the performance of autonomous driving systems at different stages of development can be covered, making the constructed simulation scenario library more widely applicable and representative.

[0025] The initial simulation scenario library can be understood as a pre-built collection of various scenarios, serving as the starting point for building the target simulation scenario library. These scenarios can simulate various real-world road conditions that autonomous driving systems may encounter. The scenarios in the initial scenario library are typically based on multiple sources, such as problem cases discovered during real-vehicle road tests, which reflect scenarios where autonomous driving systems are prone to errors in actual operation; artificially designed hazardous scenarios used to test the autonomous driving system's ability to cope with extreme or complex situations; and typical scenarios extracted from natural driving data to cover common driving environments and behavioral patterns.

[0026] Simulation test results can be understood as the simulated performance of the autonomous driving system under the current scenario library. For each historical software version, after batch simulation testing using the initial simulation scenario library, a series of results are obtained. These results are usually expressed as pass or fail, representing the performance of the autonomous driving system in each scenario. By statistically analyzing these results, the overall pass rate of the current historical software version across all scenarios can be obtained, which serves as the final output test result. The overall pass rate reflects the overall performance level of the historical software version under the initial simulation scenario library.

[0027] In addition, accuracy, recall, and false detection rates can be used as simulation test results. These metrics evaluate the autonomous driving system's perception and decision-making capabilities regarding the surrounding environment from different perspectives. For example, accuracy measures the system's ability to correctly identify targets such as road signs, vehicles, and pedestrians; recall indicates the proportion of targets that the system can detect out of the total number of actual targets; and false detection rate reflects the situation where the system incorrectly identifies non-targets as targets.

[0028] By analyzing the correlation between simulation test results and real vehicle performance indicators, problems and deficiencies in the initial simulation scenario library can be identified. Based on the analysis results, the initial simulation scenario library can be iteratively optimized and adjusted to ensure that the test results of the constructed target simulation scenario library can truly reflect the performance of the autonomous driving system on real roads.

[0029] This completes the above. Figure 1 The process of building an autonomous driving simulation scenario library shown enables simulation test results on the constructed target simulation scenario library to accurately reflect and predict the performance of the autonomous driving system on real roads, thereby establishing a reliable correlation between simulation and real vehicles.

[0030] In some embodiments of this application, the process of obtaining test results for multiple historical software versions in step 102 above may include: Obtain the initial simulation scenario library, which is a subset selected from the basic scenario pool or a scenario library used in historical simulation test processes; The initial simulation scenario library was used to perform simulation tests on each of the multiple historical software versions, and the test results for each historical software version were obtained.

[0031] The basic scenario pool is a comprehensive set of scenarios, which can be derived from problem cases discovered through real-vehicle road tests, manually designed hazardous scenarios, and typical scenarios extracted from natural driving data. When selecting the initial simulation scenario library, a subset can be chosen from the basic scenario pool as the starting point for building the target simulation scenario library, based on the typicality, severity, and diversity of the scenarios.

[0032] In addition, in practical applications, simulation scenario libraries are also used for testing and simulation of autonomous driving systems. Therefore, the scenario library used in actual simulation testing can also be used as the starting point for building the target simulation scenario library.

[0033] For each historical software version, after performing batch simulation tests using the initial simulation scenario library, a series of test results will be obtained. For example, for n historical software versions: V1, V2, …, Vn, n test results can be obtained: R1, R2, …, Rn.

[0034] For example, the test results can be any one of the following: overall pass rate, recognition accuracy, recall rate, or false positive rate.

[0035] Therefore, by using the data prepared in the early stages (such as a basic scenario pool or the scenario library currently in use) to build an initial simulation scenario library as a priority starting point, and then using the initial simulation scenario library to conduct batch simulation tests on each historical software version, a series of test results can be obtained, providing data support for subsequent analysis of the differences between simulation and real vehicles and for optimizing the scenario library.

[0036] In some embodiments of this application, the optimization process for the initial simulation scene library in step 103 above may include: Determine the first correlation coefficient between test results for multiple historical software versions and real-vehicle performance indicators; Iterate through each scene in the initial simulation scene library. For the currently traversed scene, determine the scene influence degree corresponding to the scene. If the scene influence degree is less than a threshold, perform optimization operations on the initial simulation scene library. The optimization operations are to remove the scene or adjust the parameters of the scene. The second correlation coefficient between the test results of simulations using the optimized initial simulation scenario library for multiple historical software versions and the actual vehicle performance indicators was determined. If the second correlation coefficient is greater than the first correlation coefficient, or if the scene influence corresponding to the scene is greater than the threshold, continue to traverse the next scene in the initial simulation scene library until the second correlation coefficient is greater than the preset value and the traversal ends. The optimized initial simulation scenario library is used as the target simulation scenario library.

[0037] Among them, the first correlation coefficient is a quantitative indicator of the degree of correlation between simulation test results and real vehicle performance indicators under the initial simulation scenario library. It serves as a reference for subsequent optimization and is used to measure the accuracy and effectiveness of the initial simulation scenario library. By comparing it with the optimized correlation coefficient, it is possible to intuitively determine whether the optimization operation has improved the quality of the scenario library.

[0038] The first correlation coefficient can be understood as a statistical correlation coefficient. Commonly used correlation coefficients include the Pearson correlation coefficient and the Spearman rank correlation coefficient. The magnitude of the correlation coefficient reflects the degree of correlation between simulation results and real vehicle performance indicators. The closer the coefficient is to 1, the stronger the correlation between the two, meaning that the simulation results can better reflect the real vehicle performance.

[0039] Scene impact is used to measure the contribution of each scene to the overall relevance, helping to identify which scenes play an important role in improving the relevance between simulation and real vehicle performance indicators, and which scenes have a smaller effect or even have a negative impact, thus providing a basis for optimizing the scene library.

[0040] When the impact of a scenario is less than a threshold, it indicates that the scenario contributes little to improving the correlation between simulation and real vehicle performance indicators, and may even interfere with the overall correlation. During optimization, removing or adjusting this scenario can optimize the structure of the scenario library and improve its effectiveness.

[0041] In practice, scenarios with minimal or negative impact are directly removed from the scenario library to reduce unnecessary interference with overall relevance. For some scenarios, although their overall impact is small, adjusting parameters may improve their contribution to relevance. For example, adjusting parameters such as traffic flow density, weather conditions, and participant behavior patterns can make the scenarios better reflect the performance differences of the autonomous driving system under different conditions.

[0042] The second correlation coefficient is used to evaluate the impact of optimization operations on the scenario library. The method is the same as that used to calculate the first correlation coefficient. Appropriate statistical methods are used to calculate the correlation coefficient between the simulation test results and the real vehicle performance indicators under the optimized scenario library.

[0043] The second correlation coefficient is compared with the first correlation coefficient to determine whether the optimization operation has effectively improved the correlation between the simulation test results and the real vehicle performance indicators, thereby deciding whether to continue to traverse the scenario for optimization or to end the process.

[0044] The conditions for continuing the iteration include: the second correlation coefficient being greater than the first correlation coefficient, indicating that the optimization operation has effectively improved the quality of the scene library, and it is necessary to continue iterating through other scenes to further optimize the scene library; and the scene influence corresponding to the scene being greater than the threshold, indicating that the scene makes a significant contribution to the overall relevance, and it is necessary to retain it and continue to analyze other scenes.

[0045] The traversal ends when the second correlation coefficient is greater than a preset value, indicating that the scene library has been optimized to the required level. At this point, the traversal stops and the optimization process ends. The preset value is determined based on actual needs and industry standards; it represents the required correlation level of the scene library, for example, 0.9.

[0046] As can be seen, by continuously traversing the scenario library and performing optimization operations through a closed-loop optimization process, invalid or interfering scenarios are filtered out, and the correlation between simulation test results and real vehicle performance indicators is gradually improved. This ensures that the scenarios in the optimized scenario library are those key scenarios that can effectively distinguish the strengths and weaknesses of different versions, making the test more efficient and focused.

[0047] In some embodiments of this application, the process of determining the first correlation coefficient between test results corresponding to multiple historical software versions and real-vehicle performance indicators may include: Convert the test results corresponding to multiple historical software versions into simulation equivalent indicators; The correlation coefficient between the simulation equivalent index and the actual vehicle performance index is used as the first correlation coefficient.

[0048] Among them, the simulation equivalent index can be understood as an equivalent index that is similar to or related to the performance index of the real vehicle. By converting the test results into the simulation equivalent index, the test results of different forms can be unified under a comparable measurement standard, laying the foundation for subsequent calculation of correlation coefficients.

[0049] During the conversion, a suitable mapping model can be selected based on the characteristics of the test results and the type of real-vehicle performance indicators. Understanding the meaning, form, and dimensions of each test result helps to grasp its characteristics. For example, the pass rate is a percentage indicator that represents the proportion of times the system successfully completes a task in a specific scenario; the number of errors is a count indicator that records the number of errors that occur during the simulation.

[0050] Common mapping models include linear functions, nonlinear functions, or machine learning-based models. The parameters of a mapping model can be set empirically or calibrated with a small sample size. For example, a simple mapping model that represents the overall pass rate can be: eMPI_i = k * Ri / (1 - Ri), where k is a constant coefficient.

[0051] The first correlation coefficient serves as a quantitative indicator of the correlation between simulated and real-vehicle performance metrics within the initial simulation scenario library. It is calculated using statistical methods, such as the Pearson correlation coefficient or the Spearman rank correlation coefficient. The calculated first correlation coefficient typically ranges from -1 to 1, with a value closer to 1 indicating a stronger correlation.

[0052] For example, suppose the real-world performance metrics for n historical software versions are: MPI_1, MPI_2, …, MPI_n, and the simulation equivalent metrics are eMPI_1, eMPI_2, …, eMPI_n. The statistical correlation coefficient ρ between them is calculated using the Pearson correlation coefficient method. In other words, the first correlation coefficient is the Pearson correlation coefficient.

[0053] In this embodiment, by converting the test results into simulation equivalent indicators, different forms of test results can be unified under a comparable metric, enhancing comparability and facilitating the calculation of correlation coefficients.

[0054] In some embodiments of this application, the process of determining the scene impact corresponding to the scene may include: Obtain the test results for each historical software version in the scenario, wherein the test results are either pass or fail. Based on the test results and real-vehicle performance indicators of each historical software version in the aforementioned scenario, the scenario impact is obtained.

[0055] In order to assess the impact of each scenario on the overall relevance, test results for each historical software version were obtained for a single scenario, so as to focus on the analysis of a single scenario and understand the performance of different software versions in a specific scenario.

[0056] To obtain test results for a single scenario, during simulation testing based on the initial simulation scenario library, the test results for each historical software version running that specific scenario can be recorded. These test results, in both pass and failure formats, are binary result formats that reflect whether the software can run normally in that scenario. When calculating the scenario's impact, a pass result can be defined as 1, and a failure result as 0.

[0057] The scenario impact, obtained by comparing test results for each historical software version with real-vehicle performance metrics for a single scenario, can be defined as the correlation between the pass / fail form of the scenario and the upward or downward trend of the real-vehicle performance metrics. In practice, the scenario impact is determined by ranking the test results and real-vehicle performance metrics for n software versions and then calculating the Spearman correlation coefficient between the two ranked sequences.

[0058] In this embodiment, by utilizing the binary result form of a single scenario test result, it is easy to compare with the actual vehicle performance indicators, thereby realizing the calculation of the scenario impact.

[0059] In other embodiments of this application, the process of determining the scene impact corresponding to the scene may further include: The third correlation coefficient was determined between the test results of simulations using the initial simulation scenario library after removing the aforementioned scenario from multiple historical software versions and the actual vehicle performance indicators. The change in the third correlation coefficient relative to the first correlation coefficient is taken as the scene influence.

[0060] The third correlation coefficient is calculated using the same method as the first correlation coefficient.

[0061] The change, or the difference between the third correlation coefficient and the first correlation coefficient, reflects the degree of impact of the removed scenario on the overall correlation. The larger the difference, the greater the impact of the scenario on the overall correlation; if the difference is negative, it indicates that the presence of the scenario may have reduced the overall correlation.

[0062] In this embodiment, by removing the scenario to be analyzed from the initial simulation scenario library, the remaining scenario is used for simulation testing to obtain new test results, and the third correlation coefficient between the new test results and the actual vehicle performance indicators is calculated. The change in the third correlation coefficient and the first correlation coefficient reflects the degree of influence of the removed scenario on the overall correlation.

[0063] In other embodiments of this application, optimization of the scenario library may also involve adding new scenarios to achieve the optimization goal. For example, new scenarios that can better expose the differences in capabilities between different software versions can be added from the basic scenario pool, or the scenario types corresponding to the problems fixed or introduced in the new version can be located by analyzing historical road test issues and version change logs, and then generalized and added.

[0064] The specific implementation includes: when a suggestion to add a new scenario to the target simulation scenario library is detected, determining the fourth correlation coefficient between the test results of simulations performed by multiple historical software versions using the simulation scenario library after the addition of the new scenario and the actual vehicle performance indicators; If the fourth correlation coefficient is greater than the first correlation coefficient, suggestions will be added for the new scenario.

[0065] In other words, the test results of adding new scenarios to the scenario library showed improved correlation with real-vehicle performance indicators, indicating that the optimization of the new scenarios can effectively improve the quality of the scenario library, and therefore the suggestion to add them is acceptable.

[0066] It should be noted that after obtaining the target simulation scenario library, it can be used to conduct simulation tests on new software versions, so as to quickly and cost-effectively evaluate its performance level in the early stages of development, accurately locate possible performance regressions, and significantly reduce the blind spots and costs of later real vehicle testing.

[0067] That is, for the new software version V_new to be evaluated, simulation tests are performed using the target simulation scenario library S_final to obtain the overall pass rate R_new, and then its simulation equivalent index eMPI_new is calculated through the mapping model. This eMPI_new is the predicted real-vehicle MPI level of this software version in a real road environment.

[0068] Based on the descriptions of the above embodiments, the following detailed explanation of the construction and optimization process of the simulation scene library is provided using a specific embodiment, including: S1: Data Preparation Stage: Collect historical data: Obtain the sequence of real-vehicle performance indicators of the autonomous driving system on n historical software versions (V1, V2, …, Vn): MPI_1, MPI_2, …, MPI_n.

[0069] Obtain a basic scenario pool: Construct a basic scenario pool P containing a large number of scenarios. These scenarios are derived from problem cases discovered in real vehicle road tests, manually designed dangerous scenarios, and typical scenarios extracted from natural driving data.

[0070] S2: Initial scene library construction and simulation equivalence index calculation stage: From the basic scenario pool P, a subset is selected as the initial scenario library S0 based on the typicality, severity, and diversity of the scenarios.

[0071] For each historical software version Vi, perform batch simulation tests using the initial scenario library S0, and calculate the overall pass rate Ri of that software version in S0.

[0072] The overall pass rate Ri for each historical software version is converted into a simulation equivalent index eMPI_i using a mapping model. The mapping model can be a linear function, a nonlinear function, or a machine learning-based model.

[0073] S3: Correlation Analysis and Optimization Goal Setting Phase: Calculate the statistical correlation coefficient ρ between the real vehicle performance index sequence: MPI_1, MPI_2, …, MPI_n and the simulation equivalent index sequence: eMPI_1, eMPI_2, …, eMPI_n, such as the Pearson correlation coefficient and the Spearman rank correlation coefficient.

[0074] The optimization objective is to maximize the correlation coefficient ρ so that the trend of the simulation results is highly consistent with the trend of the actual vehicle performance.

[0075] S4: Scene library iterative optimization phase: This stage is a closed-loop optimization process, which aims to improve the relevance ρ by adjusting the content of the scenario library S0.

[0076] S4.1: Scene Impact Analysis: Iterate through each scene in the scene library S0, and for the currently iterated scene j, calculate the scene influence degree of scene j on the overall relevance.

[0077] S4.2: Scene Library Adjustment Strategy: Based on the scene impact analysis results, the scene library S0 was optimized as follows: Remove scenes with negligible impact.

[0078] Adjust the parameters of the scenario (such as traffic flow density, weather conditions, and the aggressiveness of participant behavior) to make the differences in version performance more obvious.

[0079] From the base scenario pool P, identify new scenarios that better expose the differences in capabilities between different versions and add them. This can be done by analyzing historical road test issues and version change logs to pinpoint the scenario types corresponding to issues fixed or introduced in new versions, and then generalizing and adding them.

[0080] S4.3: Recalculation and Evaluation: Based on the adjusted new scenario library S', repeat step S2 to calculate the overall pass rate (eMPI) of each software version in the new scenario library S' and step S3 to calculate the new correlation coefficient ρ'.

[0081] S4.4: Iterative Decision: If ρ'>ρ, then set the new scene library S' as the current optimal library, continue to traverse the next scene, and repeat steps S4.1 to S4.4 to continue optimization on the current optimal library.

[0082] If ρ' reaches its maximum, the current optimal library S_final will be determined as the target simulation scenario library.

[0083] Corresponding to the aforementioned embodiments of the autonomous driving simulation scenario library construction method, this application also provides embodiments of the autonomous driving simulation scenario library construction apparatus.

[0084] Figure 2 This is a schematic diagram illustrating the structure of an autonomous driving simulation scenario library construction apparatus according to an exemplary embodiment. The apparatus is used to execute the autonomous driving simulation scenario library construction method provided in any of the above embodiments, such as... Figure 2As shown, the autonomous driving simulation scenario library construction device includes: The test module 210 is used to obtain the real vehicle performance indicators of the autonomous driving system on multiple historical software versions; The simulation module 220 is used to obtain the test results of the simulation of the multiple historical software versions using the initial simulation scenario library. The library optimization module 230 is used to optimize the initial simulation scenario library based on the correlation between the test results corresponding to the multiple historical software versions and the actual vehicle performance indicators, so as to obtain the target simulation scenario library.

[0085] In an optional implementation, the simulation module 220 is specifically used for: Obtain an initial simulation scenario library, which is a subset selected from the basic scenario pool or a scenario library used in historical simulation test processes; The initial simulation scenario library is used to perform simulation tests on each of the multiple historical software versions to obtain the test results for each historical software version; the test results are any one of the following: overall pass rate, recognition accuracy, recall rate, or false detection rate.

[0086] In an alternative implementation, the library optimization module 230 is specifically used for: Determine the first correlation coefficient between the test results corresponding to the multiple historical software versions and the actual vehicle performance indicators; Each scenario in the initial simulation scenario library is traversed. For the currently traversed scenario, the scenario influence degree is determined. If the scenario influence degree is less than a threshold, the initial simulation scenario library is optimized. The optimization operation is to remove the scenario or adjust the parameters of the scenario. Determine the second correlation coefficient between the test results of the simulations performed on the multiple historical software versions using the optimized initial simulation scenario library and the actual vehicle performance indicators; If the second correlation coefficient is greater than the first correlation coefficient, or if the scene influence corresponding to the scene is greater than the threshold, continue to traverse the next scene in the initial simulation scene library until the second correlation coefficient is greater than the preset value and the traversal ends. The optimized initial simulation scenario library is used as the target simulation scenario library.

[0087] In an optional implementation, the library optimization module 230 is specifically used to convert the test results corresponding to the multiple historical software versions into simulation equivalent indicators during the process of determining the first correlation coefficient between the test results corresponding to the multiple historical software versions and the actual vehicle performance indicators; and to calculate the correlation coefficient between the simulation equivalent indicators and the actual vehicle performance indicators as the first correlation coefficient.

[0088] In an optional implementation, the library optimization module 230 is specifically used to obtain the test results of the scenario for each historical software version during the process of determining the scenario impact degree corresponding to the scenario, wherein the test results are in the form of pass or failure; and to obtain the scenario impact degree based on the test results of the scenario for each historical software version and the actual vehicle performance indicators.

[0089] In an optional implementation, the library optimization module 230 is specifically used to determine, during the process of determining the scene influence degree corresponding to the scene, a third correlation coefficient between the test results of the simulation using the initial simulation scene library after removing the scene from the multiple historical software versions and the actual vehicle performance index; and to use the change of the third correlation coefficient relative to the first correlation coefficient as the scene influence degree.

[0090] In an alternative implementation, the library optimization module 230 is further configured to: When a suggestion to add a new scenario to the target simulation scenario library is detected, a fourth correlation coefficient is determined between the test results of the simulation using the simulation scenario library after the addition of the new scenario in the multiple historical software versions and the actual vehicle performance index. If the fourth correlation coefficient is greater than the first correlation coefficient, a new scenario addition suggestion is executed.

[0091] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0092] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0093] This application also provides an electronic device corresponding to the autonomous driving simulation scenario library construction method provided in the foregoing embodiments, for executing the aforementioned autonomous driving simulation scenario library construction method.

[0094] Figure 3The diagram illustrates the hardware structure of an electronic device according to an exemplary embodiment. The electronic device includes a communication interface 601, a processor 602, a memory 603, and a bus 604. The communication interface 601, processor 602, and memory 603 communicate with each other via the bus 604. The processor 602 can execute the autonomous driving simulation scenario library construction method described above by reading and executing machine-executable instructions corresponding to the control logic of the method in the memory 603. The specific content of this method is described in the above embodiment and will not be repeated here.

[0095] The memory 603 mentioned in this application can be any electronic, magnetic, optical, or other physical storage device, and can contain stored information such as executable instructions, data, etc. Specifically, the memory 603 can be RAM (Random Access Memory), flash memory, storage drive (such as hard disk drive), any type of storage disk (such as optical disc, DVD, etc.), or similar storage media, or combinations thereof. Communication between this system network element and at least one other network element is achieved through at least one communication interface 601 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc., can be used.

[0096] Bus 604 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 603 is used to store programs, and the processor 602 executes the programs after receiving execution instructions.

[0097] Processor 602 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 602 or by instructions in software form. The processor 602 can be a general-purpose processor, including a network processor (NP), digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor.

[0098] The electronic device provided in this application embodiment and the autonomous driving simulation scene library construction method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, run or implement.

[0099] This application also provides a computer-readable storage medium corresponding to the autonomous driving simulation scenario library construction method provided in the foregoing embodiments. It can be an optical disc, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it will execute the autonomous driving simulation scenario library construction method provided in any of the foregoing embodiments.

[0100] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0101] The computer-readable storage medium provided in the above embodiments of this application and the method for constructing an autonomous driving simulation scenario library provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0102] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0103] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0104] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for constructing an autonomous driving simulation scenario library, characterized in that, The method includes: Obtain real-vehicle performance metrics for the autonomous driving system across multiple historical software versions; Obtain the test results of simulations performed on the multiple historical software versions using the initial simulation scenario library; Based on the correlation between the test results corresponding to the multiple historical software versions and the actual vehicle performance indicators, the initial simulation scenario library is optimized to obtain the target simulation scenario library.

2. The method according to claim 1, characterized in that, The step of obtaining the test results of simulations performed on the multiple historical software versions using the initial simulation scenario library includes: Obtain an initial simulation scenario library, which is a subset selected from the basic scenario pool or a scenario library used in historical simulation test processes; The initial simulation scenario library is used to perform simulation tests on each of the multiple historical software versions to obtain the test results for each historical software version; the test results are any one of the following: overall pass rate, recognition accuracy, recall rate, or false detection rate.

3. The method according to claim 1, characterized in that, The optimization of the initial simulation scenario library based on the correlation between the test results corresponding to the multiple historical software versions and the real vehicle performance indicators includes: Determine the first correlation coefficient between the test results corresponding to the multiple historical software versions and the actual vehicle performance indicators; Each scenario in the initial simulation scenario library is traversed. For the currently traversed scenario, the scenario influence degree is determined. If the scenario influence degree is less than a threshold, the initial simulation scenario library is optimized. The optimization operation is to remove the scenario or adjust the parameters of the scenario. Determine the second correlation coefficient between the test results of the simulations performed on the multiple historical software versions using the optimized initial simulation scenario library and the actual vehicle performance indicators; If the second correlation coefficient is greater than the first correlation coefficient, or if the scene influence corresponding to the scene is greater than the threshold, continue to traverse the next scene in the initial simulation scene library until the second correlation coefficient is greater than the preset value and the traversal ends. The optimized initial simulation scenario library is used as the target simulation scenario library.

4. The method according to claim 3, characterized in that, The determination of the first correlation coefficient between the test results corresponding to the multiple historical software versions and the actual vehicle performance indicators includes: Convert the test results corresponding to the multiple historical software versions into simulation equivalent indicators; The correlation coefficient between the simulation equivalent index and the actual vehicle performance index is calculated as the first correlation coefficient.

5. The method according to claim 3, characterized in that, Determining the scene impact degree corresponding to the scene includes: Obtain the test results for each historical software version in the scenario, wherein the test results are either pass or fail. Based on the test results of each historical software version in the described scenario and the actual vehicle performance indicators, the scenario impact is obtained.

6. The method according to claim 3, characterized in that, Determining the scene impact degree corresponding to the scene includes: Determine the third correlation coefficient between the test results of the simulations performed using the initial simulation scenario library after removing the scenario from the multiple historical software versions and the actual vehicle performance indicators; The change in the third correlation coefficient relative to the first correlation coefficient is taken as the scene influence degree.

7. The method according to claim 3, characterized in that, The method further includes: When a suggestion to add a new scenario to the target simulation scenario library is detected, a fourth correlation coefficient is determined between the test results of the simulation using the simulation scenario library after the addition of the new scenario in the multiple historical software versions and the actual vehicle performance index. If the fourth correlation coefficient is greater than the first correlation coefficient, a new scenario addition suggestion is executed.

8. An apparatus for constructing an autonomous driving simulation scenario library, characterized in that, The device includes: The real-world testing module is used to obtain real-vehicle performance metrics of the autonomous driving system across multiple historical software versions. The simulation module is used to obtain the test results of the simulation of the multiple historical software versions using the initial simulation scenario library; The library optimization module is used to optimize the initial simulation scenario library based on the correlation between the test results corresponding to the multiple historical software versions and the actual vehicle performance indicators, so as to obtain the target simulation scenario library.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the program to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by a processor to implement the method as described in any one of claims 1-7.

Images