Test analysis method and device based on environmental simulation scene, medium and equipment

Abstract

The application discloses a test analysis method and device based on an environment simulation scene, a medium and equipment, and relates to the technical field of automatic driving. The method comprises the following steps: acquiring at least one environment element set, wherein the environment element set comprises at least one target environment element; the target environment element indicates a virtual traffic environment feature; generating at least one environment simulation scene according to the at least one environment element set; performing a test on an automatic driving algorithm based on the at least one environment simulation scene to obtain a test result; and determining an analysis result of the automatic driving algorithm according to the test result; the analysis result indicates analysis information corresponding to each target environment element. The application can improve the comprehensiveness of simulation testing and the accuracy of analysis, and better meet the testing requirements of high-level automatic driving.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, specifically to testing and analysis methods, devices, media, and equipment based on environmental simulation scenarios. Background Technology

[0002] The development process for autonomous driving follows a V-shaped development model, emphasizing collaboration and speed in software development. It organically combines software development and testing to shorten the development cycle while ensuring development quality. Testing includes simulation testing and real-vehicle testing. Simulation testing encompasses model-in-the-loop (MIL) testing, software-in-the-loop (Software-in-the-loop) testing, processor-in-the-loop (CIL) testing, hardware-in-the-loop (HIL) testing, and vehicle-in-the-loop (VIL) testing. Real-vehicle testing includes closed-road testing and open-road testing.

[0003] In existing technologies, simulation testing primarily targets single functional scenarios, such as Automatic Emergency Braking (AEB) and Adaptive Cruise Control (ACC). The evaluation methods for these single functional scenarios are also relatively simplistic, mainly focusing on passability as an indicator. The criteria for determining passability are largely based on regulations, and in the absence of regulations, they are usually based on enterprise standards or the experience of development and testing personnel. As the level of autonomous driving continues to improve, the performance requirements for perception, decision-making, and control in autonomous driving are also gradually increasing. This single-functional testing and evaluation method cannot comprehensively test and accurately evaluate the entire process of autonomous driving, nor can it meet the testing needs for higher-level perception, decision-making, and control performance. Summary of the Invention

[0004] To improve the comprehensiveness of simulation testing and the accuracy of analysis, and to better meet the testing requirements of high-level autonomous driving, this application provides a testing and analysis method, apparatus, medium, and equipment based on environmental simulation scenarios. The technical solution is as follows:

[0005] Firstly, this application provides a test and analysis method based on an environmental simulation scenario, the method comprising:

[0006] Obtain at least one set of environmental elements, the set of environmental elements including at least one target environmental element; the target environmental element indicates virtual traffic environment characteristics;

[0007] Based on the at least one set of environmental elements, at least one environmental simulation scene is generated;

[0008] Based on at least one of the environmental simulation scenarios, the autonomous driving algorithm is tested, and the test results are obtained.

[0009] Based on the test results, the analysis results of the autonomous driving algorithm are determined; the analysis results indicate the analysis information corresponding to each of the target environmental elements.

[0010] Optionally, obtaining at least one set of environment elements includes:

[0011] Obtain an environment element library, wherein the environment element library includes at least one environment element;

[0012] At least one environmental element is selected from the environmental element library as at least one target environmental element, and the environmental element set is obtained from the at least one target environmental element.

[0013] Optionally, the method further includes:

[0014] Acquire at least one target dynamic object; the target dynamic object represents a virtual traffic dynamic object;

[0015] The at least one target dynamic object is added to the environmental simulation scene to obtain the target simulation scene, which is then used to test the autonomous driving algorithm.

[0016] Optionally, the method further includes:

[0017] Based on the real distribution information corresponding to the virtual traffic environment features, configure the element attribute information of at least one target environment element in the environment element set.

[0018] Optionally, the step of performing tests on the autonomous driving algorithm based on the at least one environmental simulation scenario to obtain test results includes:

[0019] Acquire vehicle dynamics models, onboard sensor models, and autonomous driving algorithms;

[0020] The at least one environmental simulation scenario, the vehicle dynamics model, the on-board sensor model, and the autonomous driving algorithm are integrated to obtain at least one test case; the test case corresponds one-to-one with the environmental simulation scenario.

[0021] Execute the at least one test case to obtain test results; the test results indicate the pass information of the autonomous driving algorithm in the at least one test case.

[0022] Optionally, determining the analysis results of the autonomous driving algorithm based on the test results includes:

[0023] Determine at least one element attribute value of the target environment element;

[0024] Determine at least one set of environmental simulation scenarios, wherein the set of environmental simulation scenarios corresponds one-to-one with the attribute values ​​of the elements;

[0025] Based on the test results, determine the test pass rate information of the autonomous driving algorithm in the at least one environmental simulation scenario set;

[0026] Determine the attribute weight corresponding to the at least one element attribute value;

[0027] Based on the test pass rate information of the autonomous driving algorithm in the at least one environmental simulation scenario set and the attribute weights corresponding to the at least one element attribute values, analysis information corresponding to the at least one attribute value is obtained;

[0028] Based on the analysis information corresponding to the at least one attribute value, analysis information corresponding to the target environmental element is obtained.

[0029] Optionally, the method further includes:

[0030] The analysis information corresponding to each of the target environmental elements is added together to obtain the target analysis information of the autonomous driving algorithm, which indicates the overall performance of the autonomous driving algorithm.

[0031] Secondly, this application provides a test and analysis device based on an environmental simulation scenario, the device comprising:

[0032] An environment element acquisition module is used to acquire at least one set of environment elements, the set of environment elements including at least one target environment element, the target environment element indicating virtual traffic environment characteristics;

[0033] An environment simulation scene generation module is used to generate at least one environment simulation scene based on the at least one set of environment elements.

[0034] The testing module is used to perform tests on the autonomous driving algorithm based on at least one environmental simulation scenario and obtain test results;

[0035] An analysis module is used to determine the analysis results of the autonomous driving algorithm based on the test results; the analysis results indicate the analysis information corresponding to each of the target environmental elements.

[0036] Thirdly, this application provides a computer-readable storage medium storing at least one instruction or at least one program, which is loaded and executed by a processor to implement a test and analysis method based on an environmental simulation scenario as described in the first aspect.

[0037] Fourthly, this application provides a computer device, the computer device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or at least one program being loaded and executed by the processor to implement a test and analysis method based on an environmental simulation scenario as described in the first aspect.

[0038] Fifthly, this application provides a computer program product, which includes computer instructions that, when executed by a processor, implement a test and analysis method based on an environmental simulation scenario as described in the first aspect.

[0039] The testing and analysis method, apparatus, medium, and equipment based on environmental simulation scenarios provided in this application have the following technical effects:

[0040] The solution provided in this application generates at least one environmental simulation scenario based on at least one set of environmental elements, wherein the target environmental elements included in the set of environmental elements are used to indicate the characteristics of the virtual traffic environment; then, an autonomous driving algorithm is tested based on at least one environmental simulation scenario to obtain test results; and based on the test results, an analysis result of the autonomous driving algorithm is obtained, wherein the analysis result indicates the analysis information corresponding to each target environmental element. The solution provided in this application constructs an environmental element set from target environmental elements, and then generates an environmental simulation environment based on the environmental element set. This allows for the simulation of various traffic environments. In addition to common environments that closely resemble reality, it can also simulate extreme or complex environments that are unlikely to occur in reality, improving the scenario coverage of simulation testing and meeting the testing needs of various scenarios. This enables comprehensive and complete testing of the autonomous driving process. Furthermore, simulation testing in extreme or complex environments can help improve the stability and robustness of autonomous driving algorithms. The solution provided in this application does not only test a single autonomous driving function, but tests the entire autonomous driving process in an environmental simulation scenario, which also improves the comprehensiveness and accuracy of simulation testing, meeting the testing requirements of high-level autonomous driving. Based on the test results, the solution provided in this application further analyzes the target environmental elements, obtaining more detailed and accurate analytical information, understanding the degree of influence of different environmental elements on the performance of autonomous driving algorithms, and better promoting iterative optimization of the algorithms to meet the high performance requirements of high-level autonomous driving.

[0041] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0042] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart illustrating a test and analysis method based on an environmental simulation scenario provided in an embodiment of this application;

[0044] Figure 2 This is a schematic diagram illustrating the composition of an environmental element set provided in an embodiment of this application;

[0045] Figure 3 This is a schematic diagram of a process for testing an autonomous driving algorithm based on an environmental simulation scenario, provided in an embodiment of this application.

[0046] Figure 4 This is a flowchart illustrating a process for analyzing test results from the perspective of target environmental elements, as provided in an embodiment of this application.

[0047] Figure 5 This is a schematic diagram of another overall analysis and testing result flowchart provided in an embodiment of this application;

[0048] Figure 6 This is a schematic diagram of a test and analysis device based on an environmental simulation scenario provided in an embodiment of this application;

[0049] Figure 7 This is a schematic diagram of the hardware structure of a device for implementing a test and analysis method based on an environmental simulation scenario, provided in an embodiment of this application. Detailed Implementation

[0050] To improve the comprehensiveness of simulation testing and the accuracy of analysis, and to better meet the testing requirements of high-level autonomous driving, this application provides a testing and analysis method, apparatus, medium, and equipment based on environmental simulation scenarios. The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout.

[0051] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0052] The following describes a test and analysis method based on an environmental simulation scenario provided in this application. Figure 1 This is a flowchart illustrating a test and analysis method based on an environmental simulation scenario, as provided in this application. This application provides the operational steps described in the embodiments or flowchart, but based on conventional or non-inventive methods, it may include more or fewer operational steps. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system or server product execution, the methods shown in the embodiments or drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Please refer to... Figure 1 The test and analysis method based on an environmental simulation scenario provided in this application embodiment may include the following steps:

[0053] S100: Obtain at least one set of environmental elements, which includes at least one target environmental element; the target environmental element indicates the characteristics of the virtual traffic environment.

[0054] In this embodiment, the environmental element set is a collection of one or more target environmental elements, which are used to characterize virtual traffic environment features. In one feasible implementation, the target environmental elements can characterize broad categories such as weather environmental features, lighting environmental features, road environmental features, and landscape environmental features. Further, weather environmental features can specifically represent sunny days, rainy days, snowy days, fog, wind speed, temperature, humidity, etc.; lighting environmental features can specifically represent daytime, nighttime, etc.; road environmental features can specifically represent road topology, lane type, road slope, road material, etc.; and landscape environmental features can specifically represent trees, tall buildings, etc. In another feasible implementation, the target environmental elements are subdivided according to the category of environmental objects. For example, target environmental elements can directly characterize sunny days, daytime, trees, traffic signs, etc. The above are examples of the classification of traffic environment features provided in this application.

[0055] The target environmental elements are configured with element attribute information, which may include, but is not limited to, element category information, time information, location information, and element feature information. For example, element category information may be weather-related, road-related, or landscape-related, and element feature information may be influencing factors for the corresponding category, such as temperature, humidity, wind speed, and rainfall information in the weather category. When multiple sets of environmental elements are obtained, different dynamic object sets may include sets of environmental elements representing different types, or the same type of target environmental elements may have different element attribute information. The element attribute information of the target environmental elements can be specifically designed according to the testing requirements, and this application does not impose any limitations on it.

[0056] If feasible, step S100 may include the following steps:

[0057] S110: Obtain the environment element library, which includes at least one environment element.

[0058] In one feasible implementation, an environmental element library is pre-established, where each environmental element represents a different category of virtual traffic environment features. When constructing the environmental element library, environmental features that will affect the behavior of autonomous vehicles are represented by different environmental elements according to their categories. Element attribute variables common to that type of environmental element are configured for each environmental element. For example, the element attribute variables for environmental elements representing weather characteristics may include, but are not limited to, temperature, wind speed, humidity, rainfall, and snowfall; the element attribute variables for environmental elements representing landscape characteristics may include, but are not limited to, shape, height, and location; and the element attribute variables for environmental elements representing road characteristics may include, but are not limited to, curvature, slope, and coefficient of friction. In other words, in the environmental element library, environmental elements are abstract expressions and encapsulations of the same type of environmental features.

[0059] S120: Select at least one environment element from the environment element library as at least one target environment element, and obtain an environment element set from the at least one target environment element.

[0060] In one feasible implementation, such as Figure 2 As shown, a finite number of environment elements are randomly selected multiple times from m environment elements in the environment element library. These selected environment elements are used as target environment elements, thus forming different sets of environment elements. For example, environment element set 1 may include environment element 1, environment element 3, ...; environment element set 2 may include environment element 2, ..., environment element m; environment element set n may include environment element 1, environment element 3, ..., environment element m. A single environment element can be selected multiple times. Figure 2This is merely an example and is not a limitation on the number of environmental element sets or their composition. That is, in the embodiments of this application, the categories and attribute information of the environmental elements (which can also be referred to as target environmental elements) in the various environmental element sets obtained after extraction may be different.

[0061] In the above embodiments, by pre-building an environmental element library, environmental elements that affect the behavior of autonomous vehicles are abstracted, expressed, and encapsulated by category. This allows for the rapid generation of environmental element sets with different combinations, resulting in more comprehensive static simulation scenarios. These scenarios can simulate extreme or complex environments that are unlikely to occur in real-world scenarios, meeting the testing needs of various scenarios and enabling comprehensive testing of the autonomous driving process. Furthermore, the constructed environmental element library is reusable, reducing the development workload in the simulation testing phase and avoiding repeated development and configuration of the same category of environmental elements, effectively improving testing efficiency.

[0062] In one feasible implementation, if the environmental elements in the environmental element library are abstract expressions and encapsulations of the same type of environmental features, then the environmental elements also need to be instantiated. In this embodiment, the method may further include: configuring element attribute information of at least one target environmental element in the environmental element set based on the real distribution information corresponding to the virtual traffic environment features.

[0063] Among them, the real distribution information corresponding to the virtual traffic environment features represents the distribution of the virtual traffic environment features in the real scene. For example, taking weather environment features as an example, based on the ratio of sunny days to rainy days in a year, the target environmental elements that centrally represent weather environment features, such as rainfall information and temperature information, can be configured.

[0064] In the above embodiments, the element attribute information of the target environment elements is generalized according to the actual distribution, which can be closer to the real scene, meet the needs of various simulation tests, and make the simulation test results more effective and have practical reference value.

[0065] In another feasible implementation, during the instantiation of environmental elements, the element attribute information is artificially set to the element attribute information corresponding to extreme or complex environments. This allows the effectiveness, stability, and robustness of the autonomous driving algorithm to be tested using environmental simulation scenarios that simulate extreme or complex environments.

[0066] S200: Generate at least one environmental simulation scene based on at least one set of environmental elements.

[0067] In this embodiment of the application, one or more target environmental elements contained in an environmental element set are integrated into an environmental simulation scene. In the case of multiple environmental element sets, multiple environmental simulation scenes are generated respectively. That is, one environmental element set corresponds to one environmental simulation scene. The environmental simulation scene constitutes a specific environment that simulates reality and reflects the attribute performance of environmental elements in that specific environment.

[0068] In one embodiment of this application, a corresponding environmental simulation scene is generated based on a set of environmental elements. The environmental simulation scene can be a logical scene, a driving scene under extreme weather conditions, or a dangerous driving scene, etc. The environmental simulation scene is used to realize the autonomous driving algorithm's perception of environmental elements, decision-making and control of autonomous driving. The performance of the autonomous driving algorithm in the environmental simulation scene is used to measure its real performance in various environmental scenes.

[0069] In another embodiment of this application, the environmental simulation scene generated based on the environmental element set also includes dynamic objects. The combined simulation scene enables the autonomous driving algorithm to perceive dynamic objects and environmental elements, and to make decisions and control the autonomous driving process. Specifically, step S200 may include the following steps:

[0070] S210: Obtain at least one target dynamic object; the target dynamic object represents a virtual traffic dynamic object.

[0071] Target dynamic objects represent virtual traffic dynamic objects, such as pedestrians, motor vehicles, animals, dynamic road obstacles, dynamic aerial obstacles, and changing traffic lights. Target dynamic objects are configured with object attribute information, which may include, but are not limited to, object characteristic information, quantity information, location information, and behavioral information.

[0072] S220: Add at least one target dynamic object to the environmental simulation scene to obtain the target simulation scene for testing the autonomous driving algorithm.

[0073] By utilizing integrated target simulation scenarios, autonomous driving algorithms can perceive dynamic objects and environmental elements, and make decisions and control autonomous driving. Furthermore, there can be constraint relationships between target environmental elements and target dynamic objects. For example, road topology restricts the driving path of target dynamic objects, and lane lines restrict the turning, lane changing, and other behaviors of target dynamic objects.

[0074] In the above embodiments, the combination of environmental elements and dynamic objects makes the simulation scene closer to the real scene, enhances the scene realism of the simulation test, and improves the scene coverage of the simulation test.

[0075] S300: Based on at least one environmental simulation scenario, perform tests on the autonomous driving algorithm and obtain test results.

[0076] In this embodiment, a simulation testing platform is used to construct an environmental simulation scenario and run an autonomous driving algorithm. This yields a simulated autonomous driving scenario of the vehicle throughout the entire process guided by the algorithm, which is then used as the test result. In this embodiment, the constructed environmental simulation scenario is not used to test a single autonomous driving function, but rather to test the entire autonomous driving process from start to finish, thus meeting the requirements for high-level autonomous driving testing.

[0077] In one embodiment of this application, such as Figure 3 As shown, step S300 may include the following steps:

[0078] S310: Acquire vehicle dynamics model, onboard sensor model, and autonomous driving algorithm.

[0079] In one feasible implementation, a dynamic model of an autonomous vehicle is pre-constructed based on the vehicle type, functions, etc., and the autonomous vehicle is modeled based on dynamics, including the whole vehicle, body, engine, steering, braking, front and rear suspension, tires, aerodynamic effects, etc.

[0080] In one feasible implementation, the sensors carried by the autonomous vehicle to be tested are modeled, including cameras, lidar, millimeter-wave radar, ultrasonic radar, global positioning system, inertial measurement unit, etc.

[0081] In one embodiment of this application, the autonomous driving algorithm includes a perception algorithm, a decision-making algorithm, and a control algorithm.

[0082] S320: Integrate at least one environmental simulation scenario, vehicle dynamics model, on-board sensor model and autonomous driving algorithm to obtain at least one test case.

[0083] It is feasible to use a simulation testing platform to integrate environmental simulation scenarios, vehicle dynamics models, on-board sensor models, and autonomous driving algorithms into test cases. The test cases correspond one-to-one with the environmental simulation scenarios, which is equivalent to the test cases corresponding one-to-one with the set of environmental elements.

[0084] The integration of environmental simulation scenarios, vehicle dynamics models, on-board sensor models, and autonomous driving algorithms forms a closed-loop simulation testing system for joint simulation testing. This system can test the vehicle dynamics model and on-board sensor model simultaneously with the autonomous driving algorithm.

[0085] S330: Execute at least one test case and obtain test results; the test results indicate the pass information of the autonomous driving algorithm in at least one test case.

[0086] Specifically, during the execution of test cases on the simulation test platform, the on-board sensor model sends the perception information of the environmental simulation scene in the current test case to the perception algorithm in the autonomous driving algorithm. After detection and fusion processing, the perception algorithm obtains the target information, such as detecting weather and environmental features in the image information collected by the image sensor. Then, the decision algorithm obtains the corresponding decision information based on the target information, such as readjusting the sensor priority based on the visibility indicated by the weather and environmental features. The control algorithm determines the control information based on the decision information to control the vehicle to execute the corresponding instructions.

[0087] It is feasible to determine the test pass information of the autonomous driving algorithm based on whether it complies with standards and regulations or whether any anomalies occur during the execution of test cases. The test pass information can be expressed as pass or fail.

[0088] Furthermore, the state of the autonomous vehicle is updated in real time within the environmental simulation scenario.

[0089] In the above embodiments, by integrating environmental simulation scenarios, vehicle dynamic models, on-board sensor models, and autonomous driving algorithms, a closed-loop environmental simulation testing system is formed, thereby conducting joint simulation testing and improving the completeness, reliability, and accuracy of autonomous driving simulation testing.

[0090] S400: Based on the test results, determine the analysis results of the autonomous driving algorithm; the analysis results indicate the analysis information corresponding to each target environmental element.

[0091] In this embodiment of the application, the test results are further analyzed from the perspective of target environmental elements, so as to understand the degree of influence of each target environmental element on the autonomous driving algorithm. This allows for a more detailed evaluation of the performance of the autonomous driving algorithm, enabling more targeted iterative optimization of the autonomous driving algorithm to meet the high performance requirements of high-level autonomous driving, and also reducing the risks and costs of road testing.

[0092] Optionally, the test results can be filtered and screened before analysis and evaluation to obtain valid test results.

[0093] In one embodiment of this application, such as Figure 4 As shown, step S400 may include the following steps:

[0094] S410: Determine at least one element attribute value for a target environment element.

[0095] For example, as shown in Table 1, for target environmental elements representing weather categories, their element attribute values ​​can be expressed as sunny, rainy, snowy, hazy, etc.; for target environmental elements representing road categories, their element attribute values ​​can be expressed as curvature, slope, friction coefficient, etc., and the curvature attribute value can be further divided into large, medium, and small categories. The possible scenarios for attribute values ​​need to be determined based on the acquired set of environmental elements. Statistics can be performed on at least one set of environmental elements to determine the distribution of element attribute values ​​for each target environmental element.

[0096] Table 1 Environmental Elements and Element Attribute Values

[0097]

[0098] S420: Determine at least one set of environmental simulation scenarios, with each set of environmental simulation scenarios corresponding one-to-one with the attribute values ​​of the elements.

[0099] Specifically, based on the attribute value of each element, environmental simulation scenarios that match the attribute value are extracted from at least one environmental simulation scenario used to form a set of environmental simulation scenarios corresponding to that attribute value. For example, for the attribute value "sunny day", all environmental simulation scenarios simulating a sunny day are treated as a set.

[0100] In another feasible implementation, test cases are extracted and formed according to the element attribute values, and the test cases correspond one-to-one with the environmental simulation scenarios.

[0101] S430: Based on the test results, determine the pass rate information of the autonomous driving algorithm in at least one set of environmental simulation scenarios.

[0102] In the foregoing embodiments, the test results indicate the pass / fail information of the autonomous driving algorithm in each test case, which can be represented as pass or fail. There is a one-to-one correspondence between the environmental simulation scenarios and the test cases; that is, the test results can also indicate the pass / fail information of the autonomous driving algorithm in each environmental simulation scenario. For example, for the set of environmental simulation scenarios corresponding to the element attribute value "sunny day," dividing the number of environmental simulation scenarios with a pass test result in the set by the total number of environmental simulation scenarios corresponding to the element attribute value "sunny day" yields the test pass rate information.

[0103] S440: Determine the attribute weight corresponding to at least one element attribute value.

[0104] It is feasible to determine the ratio of attribute weights corresponding to element attribute values ​​based on the ratio of the number of environmental simulation scenarios in the environmental simulation scenario set to the number of environmental simulation scenarios corresponding to element attribute values.

[0105] It is feasible. For the same type of target environment element, the sum of the attribute weights corresponding to all the element attribute values ​​involved is 1. For example, the sum of the attribute weights corresponding to the element attribute values ​​of weather environment elements such as sunny day, rainy day, and snowy day is 1.

[0106] S450: Based on the test pass rate information of the autonomous driving algorithm in at least one set of environmental simulation scenarios and the attribute weights corresponding to at least one element attribute value, obtain the analysis information corresponding to at least one attribute value.

[0107] It is feasible to multiply the attribute weight corresponding to the element attribute value with the test pass rate information of the environmental simulation scenario corresponding to the element attribute value to obtain the analysis information of the autonomous driving algorithm under the element attribute value. In other words, it can describe the performance of the autonomous driving algorithm under the element attribute value, or understand the degree of influence of the element attribute value on the autonomous driving algorithm.

[0108] S460: Based on the analysis information corresponding to at least one attribute value, obtain the analysis information corresponding to the target environment element.

[0109] It is feasible to sum the analytical information corresponding to at least one attribute value of the same type of target environment element to obtain the analytical information corresponding to the autonomous driving algorithm and the target environment element. The analytical information can be expressed as numerical values. For example, for a target environment element that characterizes weather environment features, its corresponding analytical information can characterize the autonomous driving algorithm's perception and decision-making capabilities regarding the weather environment.

[0110] For example, for a target environmental element representing weather characteristics, taking the element attribute value "sunny" as an example, the analysis information Score (sunny) corresponding to the element attribute value is calculated as follows: Score = (Number of environmental simulation scenarios that simulate a sunny environment and pass the test / Number of environmental simulation scenarios that simulate a sunny environment) * Attribute weight corresponding to sunny. Different element attribute values ​​have different scores, and the calculation process can refer to the analysis information corresponding to sunny. Simultaneously, based on the statistical results of different weather attribute values ​​throughout the year, the weights of different weather attribute values ​​are obtained, and the sum of these weights is 1. By weighted summing the scores and corresponding weights of various weather attribute values, the score corresponding to the weather environmental characteristics can be obtained, which can be seen as the performance of the autonomous driving algorithm in relation to weather conditions.

[0111] For example, for a target environmental element characterizing the lighting environment, the element attribute value can be represented as daytime and nighttime, with each attribute having a weight of 0.5. Taking daytime as an example, the corresponding analysis information Score (daytime) = (number of simulated daytime environments that have passed the test / number of simulated daytime environments) * weight of the corresponding attribute for daytime.

[0112] For example, for target environmental elements characterizing road environment features, element attribute values ​​can be represented in a two-level hierarchy. The first-level element attribute values ​​are dimensions such as curvature, slope, and friction coefficient. Taking curvature as an example, the second-level element attribute values ​​can be represented as large, medium, and small curvature. Using the method provided in this application's embodiments, complete configuration of curvature can be achieved based on highway design specifications and non-design specifications. For instance, highway design specifications stipulate that the minimum curvature radius of curves on plain and hilly expressways is 650m, and the minimum curvature radius of curves on mountain expressways is 250m. Non-design specifications consider road curvature radii outside the standard, which can simulate some dangerous test scenarios corresponding to edge roads. In another feasible implementation, the weights corresponding to different degrees of curvature can be designed based on the normal distribution of real road curvature. Taking curvature as an example, the corresponding analysis information is: Score (road curvature) = (number of simulation scenarios that simulate a large road curvature environment and have passed the test / number of simulation scenarios that simulate a large road curvature environment) * weight corresponding to large road curvature + (number of simulation scenarios that simulate a medium road curvature environment and have passed the test / number of simulation scenarios that simulate a medium road curvature environment) * weight corresponding to medium road curvature + (number of simulation scenarios that simulate a small road curvature environment and have passed the test / number of simulation scenarios that simulate a small road curvature environment) * weight corresponding to small road curvature.

[0113] In other feasible implementations, corresponding analytical information can be determined for target environmental elements that characterize static obstacles, target environmental elements that characterize landscape environmental features, etc.

[0114] In the above embodiments, further analysis of the test results of the autonomous driving algorithm from the perspective of target environmental elements and their attribute values ​​allows us to understand the autonomous driving algorithm's performance in handling target environmental elements in simulation tests based on the scores. For example, a higher Score (sunny day) than a higher Score (rainy day) indicates that the autonomous driving performance deteriorates in rainy weather, while a lower Score (weather) than a lower Score (landscape) indicates that the autonomous driving performance is poor in terms of weather environmental feature recognition, decision-making, and control. This also suggests that weather environmental features have a greater impact on the autonomous driving algorithm. Through this more detailed evaluation of the autonomous driving algorithm's performance, it is easier to iterate and optimize the autonomous driving algorithm in a more targeted manner, continuously improve the performance of the autonomous driving algorithm, reduce the risks and costs of road testing, and meet the needs of high-level autonomous driving.

[0115] Furthermore, such as Figure 5 As shown, embodiments of this application may further include:

[0116] S500: The analysis information corresponding to each target environmental element is added together to obtain the target analysis information of the autonomous driving algorithm. The target analysis information indicates the comprehensive performance of the autonomous driving algorithm.

[0117] For example, the target analysis information for autonomous driving is Score = Score (weather) + Score (light) + Score (road) + Score (other).

[0118] Furthermore, weights can be designed for different target environment elements according to functional or testing requirements, with each weight reflecting the importance of the target environment element. By weighted summing of the weights and analytical information of the target environment elements, target analysis information that can measure the overall performance of autonomous driving algorithms can be obtained.

[0119] In the above embodiments, in addition to analyzing and evaluating the merits of autonomous driving algorithms in detail from the perspective of different target environmental elements, an overall measurement can also be performed, and it can also serve as a basis for comparison before and after algorithm iteration and optimization.

[0120] As can be seen from the above embodiments, the test analysis method based on environmental simulation scenarios provided in this application uses target environmental elements to form an environmental element set, and then generates an environmental simulation environment based on the environmental element set. This method can simulate various traffic environments. In addition to common environments that are close to reality, it can also simulate extreme or complex environments that are unlikely to occur in reality, improving the scenario coverage of simulation tests and meeting the testing needs of various scenarios. This allows for comprehensive and complete testing of the autonomous driving process. Furthermore, simulation tests in extreme or complex environments can help improve the stability and robustness of autonomous driving algorithms. In the solution provided in this application, the test is not limited to a single autonomous driving function, but rather the entire autonomous driving process is tested in an environmental simulation scenario. This also improves the comprehensiveness and accuracy of simulation tests, meeting the testing requirements of high-level autonomous driving. Based on the test results, the solution provided in this application further analyzes the target environmental elements, obtaining more detailed and accurate analytical information. This allows for understanding the degree of influence of different environmental elements on the performance of autonomous driving algorithms, thereby better promoting iterative optimization of the algorithms and meeting the high performance requirements of high-level autonomous driving.

[0121] This application also provides a test and analysis device 600 based on an environmental simulation scenario, such as... Figure 6 As shown, the device 600 may include:

[0122] The environmental element acquisition module 610 is used to acquire at least one set of environmental elements, the set of environmental elements including at least one target environmental element, the target environmental element indicating virtual traffic environment characteristics;

[0123] The environment simulation scene generation module 620 is used to generate at least one environment simulation scene based on the at least one set of environment elements.

[0124] Test module 630 is used to perform tests on the autonomous driving algorithm based on the at least one environmental simulation scenario and obtain test results;

[0125] The analysis module 640 is used to determine the analysis results of the autonomous driving algorithm based on the test results; the analysis results indicate the analysis information corresponding to each of the target environmental elements.

[0126] In one embodiment of this application, the environmental element acquisition module 610 may include:

[0127] The first acquisition unit is used to acquire an environmental element library, the environmental element library including at least one environmental element;

[0128] The second acquisition unit is used to select at least one environmental element from the environmental element library as at least one target environmental element, and obtain the environmental element set from the at least one target environmental element.

[0129] In one embodiment of this application, the device 600 may further include:

[0130] The third acquisition unit is used to acquire at least one target dynamic object; the target dynamic object represents a virtual traffic dynamic object.

[0131] The target simulation scene generation unit is used to add the at least one target dynamic object to the environment simulation scene to obtain the target simulation scene for testing the autonomous driving algorithm.

[0132] In one embodiment of this application, the device 600 may further include:

[0133] The attribute configuration module is used to configure the element attribute information of at least one target environment element in the environment element set based on the real distribution information corresponding to the virtual traffic environment features.

[0134] In one embodiment of this application, the test module 630 may include:

[0135] The fourth acquisition unit is used to acquire the vehicle dynamics model, the on-board sensor model, and the autonomous driving algorithm;

[0136] An integration unit is used to integrate the at least one environmental simulation scenario, the vehicle dynamics model, the on-board sensor model, and the autonomous driving algorithm to obtain at least one test case; the test case corresponds one-to-one with the environmental simulation scenario;

[0137] A test execution unit is used to execute the at least one test case and obtain test results; the test results indicate the test pass information of the autonomous driving algorithm in the at least one test case.

[0138] In one embodiment of this application, the analysis module 640 may include:

[0139] An element attribute value determination unit is used to determine at least one element attribute value of the target environment element;

[0140] An environment simulation scene set determination unit is used for at least one environment simulation scene set, wherein the environment simulation scene set corresponds one-to-one with the element attribute value;

[0141] The test pass rate determination unit is used to determine the test pass rate information of the autonomous driving algorithm in the at least one environmental simulation scenario set based on the test results.

[0142] An attribute weight determination unit is used to determine the attribute weight corresponding to the attribute value of the at least one element;

[0143] The first analysis unit is used to obtain analysis information corresponding to the at least one attribute value based on the test pass rate information of the autonomous driving algorithm in the at least one environmental simulation scenario set and the attribute weight corresponding to the at least one element attribute value.

[0144] The second analysis unit is used to obtain analysis information corresponding to the target environmental element based on the analysis information corresponding to the at least one attribute value.

[0145] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0146] This application provides a computer device including a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement a test and analysis method based on an environmental simulation scenario as provided in the above method embodiments.

[0147] Figure 7A schematic diagram of the hardware structure of a device for implementing a test and analysis method based on an environmental simulation scenario provided in the embodiments of this application is shown. The device may constitute or include the apparatus or system provided in the embodiments of this application. Figure 7 As shown, device 10 may include one or more processors 1002 (shown as 1002a, 1002b, ..., 1002n in the figure) 1002 (processor 1002 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 1004 for storing data, and a transmission device 1006 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 7 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, device 10 may also include a... Figure 7 The more or fewer components shown, or having the same Figure 7 The different configurations shown.

[0148] It should be noted that the aforementioned one or more processors 1002 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be wholly or partially integrated into any other element within device 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0149] The memory 1004 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method described in the embodiments of this application. The processor 1002 executes various functional applications and data processing by running the software programs and modules stored in the memory 1004, thereby realizing the above-mentioned test and analysis method based on an environmental simulation scenario. The memory 1004 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1004 may further include memory remotely located relative to the processor 1002, and these remote memories can be connected to the device 10 via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0150] The transmission device 1006 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of device 10. In one example, the transmission device 1006 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 1006 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0151] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows a user to interact with the user interface of device 10 (or a mobile device).

[0152] This application embodiment also provides a computer-readable storage medium, which can be disposed in a server to store at least one instruction or at least one program related to implementing a test and analysis method based on an environmental simulation scenario in the method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the test and analysis method based on an environmental simulation scenario provided in the above method embodiment.

[0153] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0154] This invention also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a test and analysis method based on an environmental simulation scenario provided in the various optional embodiments described above.

[0155] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0156] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device, equipment, and storage medium embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0157] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0158] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A test and analysis method based on environmental simulation scenarios, characterized in that, The method includes: Obtain at least one set of environmental elements, the set of environmental elements including at least one target environmental element; the target environmental element indicates virtual traffic environment characteristics; each target environmental element corresponds to at least one element attribute value; Based on the at least one set of environmental elements, at least one environmental simulation scene is generated; Based on at least one of the environmental simulation scenarios, the autonomous driving algorithm is tested, and the test results are obtained. Based on the test results, analytical information is determined corresponding to at least one element attribute value of each of the target environmental elements; the analytical information indicates the degree of influence of each element attribute value on the autonomous driving algorithm; The analysis result of the autonomous driving algorithm is determined based on the analysis information corresponding to at least one element attribute value of each of the target environment elements; the analysis result indicates the analysis information corresponding to each of the target environment elements. The step of determining the analysis information corresponding to at least one element attribute value of each of the target environmental elements based on the test results includes: Determine at least one set of environmental simulation scenarios from the at least one environmental simulation scenario, wherein the set of environmental simulation scenarios corresponds one-to-one with the element attribute values; Based on the test results, determine the test pass rate information of the autonomous driving algorithm in the at least one environmental simulation scenario set; Determine the attribute weight corresponding to the at least one element attribute value; Based on the test pass rate information of the autonomous driving algorithm in the at least one environmental simulation scenario set and the attribute weights corresponding to the at least one element attribute value, analysis information corresponding to the at least one element attribute value is obtained.

2. The method according to claim 1, characterized in that, The process of obtaining at least one set of environment elements includes: Obtain an environment element library, wherein the environment element library includes at least one environment element; At least one environmental element is selected from the environmental element library as at least one target environmental element, and the environmental element set is obtained from the at least one target environmental element.

3. The method according to claim 1, characterized in that, The method further includes: Acquire at least one target dynamic object; the target dynamic object represents a virtual traffic dynamic object; The at least one target dynamic object is added to the environmental simulation scene to obtain the target simulation scene, which is then used to test the autonomous driving algorithm.

4. The method according to claim 2, characterized in that, The method further includes: Based on the real distribution information corresponding to the virtual traffic environment features, configure the element attribute information of at least one target environment element in the environment element set.

5. The method according to claim 1, characterized in that, The test, based on the at least one environmental simulation scenario, is performed on the autonomous driving algorithm to obtain test results, including: Acquire vehicle dynamics models, onboard sensor models, and autonomous driving algorithms; The at least one environmental simulation scenario, the vehicle dynamics model, the on-board sensor model, and the autonomous driving algorithm are integrated to obtain at least one test case; the test case corresponds one-to-one with the environmental simulation scenario. Execute the at least one test case to obtain test results; the test results indicate the pass information of the autonomous driving algorithm in the at least one test case.

6. The method according to claim 1, characterized in that, The method further includes: The analysis information corresponding to each of the target environmental elements is added together to obtain the target analysis information of the autonomous driving algorithm, which indicates the overall performance of the autonomous driving algorithm.

7. A test and analysis device based on an environmental simulation scenario, characterized in that, The device includes: An environment element acquisition module is used to acquire at least one set of environment elements, the set of environment elements including at least one target environment element, the target environment element indicating virtual traffic environment characteristics; each target environment element corresponds to at least one element attribute value; An environment simulation scene generation module is used to generate at least one environment simulation scene based on the at least one set of environment elements. The testing module is used to perform tests on the autonomous driving algorithm based on at least one environmental simulation scenario and obtain test results; An analysis module is configured to determine analysis information corresponding to at least one element attribute value of each of the target environmental elements based on the test results; the analysis information indicates the degree of influence of each element attribute value on the autonomous driving algorithm; and determine the analysis result of the autonomous driving algorithm based on the analysis information corresponding to at least one element attribute value of each of the target environmental elements; the analysis result indicates the analysis information corresponding to each of the target environmental elements. The analysis module is used to determine at least one set of environmental simulation scenarios from the at least one environmental simulation scenario, wherein the set of environmental simulation scenarios corresponds one-to-one with the element attribute values; Based on the test results, determine the test pass rate information of the autonomous driving algorithm in the at least one environmental simulation scenario set; Determine the attribute weight corresponding to the at least one element attribute value; Based on the test pass rate information of the autonomous driving algorithm in the at least one environmental simulation scenario set and the attribute weights corresponding to the at least one element attribute value, analysis information corresponding to the at least one element attribute value is obtained.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement a test and analysis method based on an environmental simulation scenario as described in any one of claims 1 to 6.

9. A computer device, characterized in that, The computer device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement a test and analysis method based on an environmental simulation scenario as described in any one of claims 1 to 6.

Images