Simulation scenario-based automatic driving test method, device, medium and equipment

Abstract

This application discloses a method, apparatus, medium, and equipment for autonomous driving testing based on simulation scenarios, relating to the field of autonomous driving technology. The method includes: acquiring at least one dynamic object set, the dynamic object set including at least one target dynamic object, the target dynamic object representing a virtual traffic dynamic object; generating at least one simulation scenario based on the at least one dynamic object set; performing tests on an autonomous driving algorithm based on the at least one simulation scenario, obtaining test results; determining the analysis results of the autonomous driving algorithm based on the test results; the analysis results indicate the analysis information of the autonomous driving algorithm in various attribute dimensions of the target dynamic object. The technical solution provided in this application introduces traffic dynamic objects that affect autonomous vehicles during the simulation testing process and analyzes the test results from the attribute dimensions of the traffic dynamic objects, thereby enabling more comprehensive and accurate testing of autonomous driving algorithms and obtaining more detailed algorithm evaluation results.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, specifically to autonomous driving testing methods, devices, media, and equipment based on simulation scenarios. Background Technology

[0002] Testing of autonomous driving algorithms is divided into simulation testing and real-world road testing. Simulation testing can include software-in-the-loop testing, model-in-the-loop testing, driver-in-the-loop testing, and vehicle-in-the-loop testing. If the autonomous driving algorithm is not mature enough after simulation testing, real-world road testing will involve a great deal of uncertainty and danger. It will not only require a large investment of manpower, material resources, and financial resources for testing, but may also endanger the personal safety of test personnel.

[0003] In existing technologies, testing scenarios primarily target single functionalities, such as Automatic Emergency Braking (AEB) and Adaptive Cruise Control (ACC). The evaluation of these single-functionality scenarios also relies heavily on standards and regulations, such as international standards and the China New Car Assessment Program (C-NCAP), with evaluation results only indicating whether a vehicle passes or fails. This testing and evaluation method ignores dynamic traffic participants who affect autonomous vehicles and lacks an evaluation of the entire autonomous driving algorithm, making it difficult to meet the comprehensiveness and accuracy requirements of high-level autonomous driving testing. Summary of the Invention

[0004] To improve the completeness, comprehensiveness, and accuracy of simulation testing, this application provides a method, apparatus, medium, and equipment for autonomous driving testing based on simulation scenarios. The technical solution is as follows:

[0005] Firstly, this application provides an autonomous driving testing method based on simulation scenarios, the method comprising:

[0006] Obtain at least one set of dynamic objects, the set of dynamic objects including at least one target dynamic object, the target dynamic object representing a virtual traffic dynamic object;

[0007] At least one simulation scenario is generated based on the at least one set of dynamic objects;

[0008] Based on at least one simulation scenario, the autonomous driving algorithm is tested, and the test results are obtained.

[0009] Based on the test results, the analysis results of the autonomous driving algorithm are determined; the analysis results indicate the analysis information of the autonomous driving algorithm in each attribute dimension of the target dynamic object.

[0010] Optionally, determining the analysis results of the autonomous driving algorithm based on the test results includes:

[0011] Determine the attribute dimensions of the target dynamic object, wherein each attribute dimension corresponds to at least one attribute value;

[0012] Determine at least one set of simulation scenarios, wherein each set of simulation scenarios corresponds one-to-one with the attribute value;

[0013] Based on the test results, determine the test pass rate information of the autonomous driving algorithm in the at least one simulation scenario set;

[0014] Determine the attribute weight corresponding to the at least one attribute value;

[0015] Based on the test pass rate information of the autonomous driving algorithm in the at least one simulation scenario set and the attribute weights corresponding to the at least one attribute value, analysis information corresponding to the at least one attribute value is obtained;

[0016] The analysis information of the autonomous driving algorithm in the attribute dimension is obtained based on the analysis information corresponding to the at least one attribute value.

[0017] Optionally, the method further includes:

[0018] The analysis information of the autonomous driving algorithm in each attribute dimension of the target dynamic object is added together to obtain the target analysis information of the autonomous driving algorithm, which indicates the comprehensive performance of the autonomous driving algorithm.

[0019] Optionally, obtaining at least one set of dynamic objects includes:

[0020] Obtain a dynamic object library, wherein the dynamic object library includes at least one dynamic object;

[0021] Select at least one dynamic object from the dynamic object library as at least one target dynamic object, and obtain the dynamic object set from the at least one target dynamic object.

[0022] Optionally, generating at least one simulation scene based on the at least one set of dynamic objects includes:

[0023] Obtain an initial static simulation scene; the static simulation scene includes at least one target static object;

[0024] At least one target dynamic object from the dynamic object set is added to the static simulation scene to obtain the simulation scene corresponding to the dynamic object set.

[0025] Optionally, the method further includes:

[0026] Based on the real distribution information corresponding to the virtual traffic dynamic objects, configure the object attribute information of at least one target dynamic object in the dynamic object set; the object attribute information includes quantity information, location information or behavior information.

[0027] Optionally, based on the at least one simulation scenario, the autonomous driving algorithm is tested to obtain test results, including:

[0028] Acquire vehicle dynamics models, onboard sensor models, and autonomous driving algorithms;

[0029] The at least one simulation scenario, the vehicle dynamics model, the on-board sensor model, and the autonomous driving algorithm are integrated to obtain at least one test case; the test case corresponds one-to-one with the simulation scenario.

[0030] Execute the at least one test case to obtain test results; the test results indicate the pass information of the autonomous driving algorithm in the at least one test case.

[0031] Secondly, this application provides an autonomous driving testing device based on a simulation scenario, the device comprising:

[0032] A dynamic object acquisition module is used to acquire at least one set of dynamic objects, the set of dynamic objects including at least one target dynamic object, the target dynamic object representing a virtual traffic dynamic object;

[0033] The simulation scene generation module is used to generate at least one simulation scene based on the at least one set of dynamic objects.

[0034] The testing module is used to perform tests on the autonomous driving algorithm based on at least one simulation scenario and obtain test results.

[0035] An analysis module is used to determine the analysis results of the autonomous driving algorithm based on the test results; the analysis results indicate the analysis information of the autonomous driving algorithm in each attribute dimension of the target dynamic object.

[0036] Thirdly, this application provides a computer-readable storage medium storing at least one instruction or at least one program, which is loaded and executed by a processor to implement an autonomous driving test method based on a simulation scenario as described in the first aspect.

[0037] Fourthly, this application provides a computer device including a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement an autonomous driving test method based on a simulation scenario as described in the first aspect.

[0038] Fifthly, this application provides a computer program product, which includes computer instructions that, when executed by a processor, implement an autonomous driving testing method based on a simulation scenario as described in the first aspect.

[0039] The autonomous driving testing method, apparatus, medium, and equipment based on simulation scenarios provided in this application have the following technical effects:

[0040] The solution provided in this application first obtains at least one dynamic object set, which includes at least one target dynamic object. The target dynamic objects in different dynamic object sets can be different, and the target dynamic objects are used to represent virtual traffic dynamic objects. Second, based on the at least one dynamic object set, at least one simulation scenario is generated, with a one-to-one correspondence between the dynamic object set and the simulation scenario. Each simulation scenario integrates at least one target dynamic object from the corresponding dynamic object set. The autonomous driving algorithm is then tested in the at least one simulation scenario to obtain test results. Finally, analysis results are determined based on the test results, indicating the analysis information of the autonomous driving algorithm in the attribute dimensions of the target dynamic objects. Using the solution provided in this application, by integrating simulation scenarios with virtual traffic dynamic objects, more comprehensive and realistic testing of autonomous driving algorithms can be conducted. Furthermore, the simulation scenarios in this application are no longer limited to testing only a single autonomous driving function, but can test the entire process of high-level autonomous driving, meeting the requirements of high-level autonomous driving testing for the completeness, comprehensiveness, and accuracy of simulation testing. Furthermore, the solution provided in this application analyzes the test results of autonomous driving algorithms from the attribute dimension of virtual traffic dynamic objects, which can help understand the performance of autonomous driving algorithms in handling various virtual traffic dynamic objects in simulation tests. This allows for a more detailed and accurate evaluation of autonomous driving algorithms, thereby promoting the optimization and iteration of the algorithms.

[0041] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0042] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart illustrating an autonomous driving testing method based on a simulation scenario provided in an embodiment of this application.

[0044] Figure 2 This is a schematic diagram illustrating the composition of a dynamic object set provided in an embodiment of this application;

[0045] Figure 3 This is a schematic diagram of a process for testing an autonomous driving algorithm based on a simulation scenario, provided in an embodiment of this application.

[0046] Figure 4 This is a flowchart illustrating an embodiment of the present application for analyzing test results from the attribute dimension of a target dynamic object;

[0047] Figure 5 This is a schematic diagram of an autonomous driving test device based on a simulation scenario provided in an embodiment of this application;

[0048] Figure 6 This is a schematic diagram of the hardware structure of a device for implementing an autonomous driving test method based on a simulation scenario, provided in an embodiment of this application. Detailed Implementation

[0049] To improve the completeness, comprehensiveness, and accuracy of simulation testing, embodiments of this application provide autonomous driving testing methods, apparatus, media, and equipment based on simulation scenarios. The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout.

[0050] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0051] The following describes an autonomous driving testing method based on simulation scenarios provided in this application. Figure 1 This is a flowchart illustrating an autonomous driving testing method based on a simulation scenario, provided in an embodiment of this application. This application provides the operational steps described in the embodiments or flowchart, but based on conventional or non-inventive methods, it may include more or fewer operational steps. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system or server product execution, the methods shown in the embodiments or drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Please refer to... Figure 1 The autonomous driving testing method based on simulation scenarios provided in this application embodiment may include the following steps:

[0052] S100: Obtain at least one set of dynamic objects, which includes at least one target dynamic object, representing a virtual traffic dynamic object.

[0053] In this embodiment, a dynamic object set is a collection of one or more target dynamic traffic objects. These target dynamic objects represent virtual dynamic traffic objects, such as pedestrians, motor vehicles, animals, road surface dynamic obstacles, aerial dynamic obstacles, and changing traffic lights. Each target dynamic object is configured with object attribute information, which may include, but is not limited to, object feature information, quantity information, location information, and behavior information. When multiple dynamic object sets are obtained, different sets may include different numbers or types of target dynamic objects, or target dynamic objects of the same type may have different object attribute information.

[0054] In one embodiment of this application, step S100 may specifically include:

[0055] S110: Obtain the dynamic object library, which includes at least one dynamic object.

[0056] In one feasible implementation, a dynamic object library is pre-built, where each dynamic object represents a different category of virtual traffic dynamic objects. When building the dynamic object library, dynamic elements that will affect the behavior of autonomous vehicles are represented as different dynamic objects according to their categories. Common object attribute variables shared by dynamic elements of that type are configured for each dynamic object, such as shape, volume, position, speed, trajectory, and collision recovery coefficient. In other words, in the dynamic object library, a dynamic object is an abstract expression and encapsulation of dynamic elements of the same type.

[0057] S120: Select at least one dynamic object from the dynamic object library as at least one target dynamic object, and obtain the dynamic object set from the at least one target dynamic object.

[0058] In one feasible implementation, such as Figure 2 As shown, a finite number of dynamic objects are randomly selected multiple times from n dynamic objects in a dynamic object library. These selected dynamic objects are then used as target dynamic objects, thus forming different sets of dynamic objects. For example, dynamic object set 1 may include dynamic object 1, dynamic object 3, ...; dynamic object set 2 may include dynamic object 2, ..., dynamic object n; dynamic object set m may include dynamic object 1, dynamic object 3, ..., dynamic object n. A single dynamic object can be selected multiple times. Figure 2 This is merely an example and is not a limitation on the number of dynamic object sets or their components. That is, in the embodiments of this application, the number of dynamic objects (which can also be referred to as target dynamic objects) and the categories of the dynamic objects involved in the various dynamic object sets obtained after extraction may be different.

[0059] In the above embodiments, by pre-building a dynamic object library, dynamic elements that affect the behavior of autonomous vehicles are abstracted, expressed and encapsulated by category. This allows for the rapid generation of dynamic object sets with different combinations, resulting in simulation scenarios with higher coverage and closer resemblance to reality. The constructed dynamic object library is reusable, reducing the development workload in the simulation testing phase and avoiding repeated development and configuration of dynamic objects of the same category, thus effectively improving testing efficiency.

[0060] Furthermore, based on the real distribution information corresponding to the virtual traffic dynamic objects, the object attribute information of at least one target dynamic object in the dynamic object set is configured; the object attribute information includes quantity information, location information, or behavior information.

[0061] Among them, the real distribution information corresponding to the virtual traffic dynamic objects represents the distribution of the object attribute information of the virtual traffic dynamic objects in the real scene. For example, taking pedestrians as an example, based on the pedestrian density distribution at road intersections in different time periods, the number and location of target dynamic objects representing the pedestrian category are configured in the dynamic object set. For example, taking vehicles as an example, based on the actual road traffic flow, the driving trajectory or driving behavior of target dynamic objects representing the vehicle category, such as acceleration, turning, lane changing, etc., are configured in the dynamic object set.

[0062] In the above embodiments, the object attribute information of the target dynamic object is generalized according to the actual distribution, which can be closer to the real scene, meet the needs of various simulation tests, and make the simulation test results more effective and valuable for reference.

[0063] S200: Generate at least one simulation scenario based on at least one set of dynamic objects.

[0064] In this embodiment of the application, one or more target dynamic objects contained in a dynamic object set are integrated into a simulation scene. In the case of multiple dynamic object sets, multiple simulation scenes are generated respectively. That is, one dynamic object set corresponds to one simulation scene. The simulation scene constitutes a specific situation that simulates reality, reflecting the time characteristics, road characteristics, traffic flow characteristics, etc. of road traffic in that specific situation.

[0065] In one embodiment of this application, a corresponding simulation scene is generated based on a dynamic object set. The simulation scene is used to realize the autonomous driving algorithm's perception of dynamic objects, decision-making and control of autonomous driving, thereby measuring the performance of the autonomous driving algorithm in the simulation scene.

[0066] In another embodiment of this application, the simulation scene generated based on the dynamic object set also includes static objects. The simulation scene is used to enable the autonomous driving algorithm to perceive both dynamic and static objects, and to make decisions and control the autonomous driving process. Specifically, step S200 may include the following steps:

[0067] S210: Obtain the initial static simulation scene; the static simulation scene includes at least one target static object.

[0068] A static simulation scene is a simulation of static objects in a real road scene. The target static objects included in a static simulation scene can be road topology, lane lines, fixed traffic signs, etc., or static environmental objects such as constant wind speed and light intensity.

[0069] S220: Add at least one target dynamic object from the dynamic object set to the static simulation scene to obtain a simulation scene corresponding to the dynamic object set.

[0070] Furthermore, there can be constraints between the target static object and the target dynamic object. For example, the road topology restricts the driving path of the target dynamic object, and the lane lines restrict the turning, lane changing and other behaviors of the target dynamic object.

[0071] In the above embodiments, the combination of static and dynamic objects makes the simulation scene closer to the real scene, enhances the scene realism of the simulation test, and improves the scene coverage of the simulation test.

[0072] S300: Based on at least one simulation scenario, perform tests on the autonomous driving algorithm and obtain test results.

[0073] In this embodiment, a simulation test platform is used to construct a simulation scenario and run an autonomous driving algorithm to obtain the simulated autonomous driving situation of a vehicle throughout the entire simulation scenario under the guidance of the algorithm, which serves as the test result. In this embodiment, the constructed simulation scenario is not used to test a single autonomous driving function, but rather to test the entire autonomous driving process from start to finish, thus meeting the requirements of high-level autonomous driving testing.

[0074] In one embodiment of this application, specifically, as Figure 3 As shown, step S300 may include the following steps:

[0075] S310: Acquire vehicle dynamics model, onboard sensor model, and autonomous driving algorithm.

[0076] In one feasible implementation, a dynamic model of an autonomous vehicle is pre-constructed based on the vehicle type, functions, etc., and the autonomous vehicle is modeled based on dynamics, including the whole vehicle, body, engine, steering, braking, front and rear suspension, tires, aerodynamic effects, etc.

[0077] In one feasible implementation, the sensors carried by the autonomous vehicle to be tested are modeled, including cameras, lidar, millimeter-wave radar, ultrasonic radar, global positioning system, inertial measurement unit, etc.

[0078] In one embodiment of this application, the autonomous driving algorithm includes a perception algorithm, a decision-making algorithm, and a control algorithm.

[0079] S320: Integrate at least one simulation scenario, vehicle dynamics model, on-board sensor model and autonomous driving algorithm to obtain at least one test case.

[0080] It is feasible to use a simulation testing platform to integrate simulation scenarios, vehicle dynamics models, on-board sensor models, and autonomous driving algorithms into test cases. The test cases correspond one-to-one with the simulation scenarios, which is equivalent to the test cases corresponding one-to-one with the dynamic object set.

[0081] The integration of simulation scenarios, vehicle dynamics models, on-board sensor models, and autonomous driving algorithms forms a closed-loop simulation testing system for joint simulation testing.

[0082] S330: Execute at least one test case and obtain test results; the test results indicate the pass information of the autonomous driving algorithm in at least one test case.

[0083] Specifically, during the execution of test cases on the simulation test platform, the on-board sensor model sends the perception information of the simulation scene in the current test case to the perception algorithm in the autonomous driving algorithm. After detection and fusion processing, the perception algorithm obtains the target information, such as the location information of pedestrians detected in the image information collected by the image sensor. Then, the decision algorithm obtains the corresponding decision information based on the target information, such as replanning the driving path of the autonomous vehicle based on the road congestion information. The control algorithm determines the control information based on the decision information to control the vehicle to execute the corresponding instructions.

[0084] It is feasible to determine the test pass information of the autonomous driving algorithm based on whether it complies with standards and regulations or whether any anomalies occur during the execution of test cases. The test pass information can be expressed as pass or fail.

[0085] Furthermore, the state of the autonomous vehicle is updated in real time within the simulation scenario.

[0086] In the above embodiments, by integrating simulation scenarios, vehicle dynamic models, on-board sensor models, and autonomous driving algorithms, a closed-loop simulation testing system is formed, thereby conducting joint simulation testing and improving the completeness, realism, and accuracy of autonomous driving simulation testing.

[0087] S400: Based on the test results, determine the analysis results of the autonomous driving algorithm; the analysis results indicate the analysis information of the autonomous driving algorithm in each attribute dimension of the target dynamic object.

[0088] In this embodiment of the application, the test results are further analyzed from the attribute dimension of the target dynamic object. This allows us to understand the degree of influence of each dimension of the target dynamic object on the autonomous driving algorithm, thereby enabling a more detailed evaluation of the performance of the autonomous driving algorithm. This facilitates more targeted iterative optimization of the autonomous driving algorithm and reduces the risks and costs of road testing.

[0089] Optionally, the test results can be filtered and screened before analysis and evaluation to obtain valid test results.

[0090] In one embodiment of this application, such as Figure 4 As shown, specifically, step S400 may include the following steps:

[0091] S410: Determine the attribute dimensions of the target dynamic object, where each attribute dimension corresponds to at least one attribute value.

[0092] Yes, it is feasible. The attribute dimensions of the target dynamic object include quantity, position, and behavior, as shown in Table 1. The attribute values ​​for the quantity dimension can be 0, 1, 2, 3, etc. The position dimension can have multiple levels of attributes. When the first-level attribute value is "directly in front," it can be further divided into three types based on distance. That is, in Table 1, there are actually 24 possible attribute values ​​for the position dimension. The possible scenarios for attribute values ​​need to be determined based on the set of dynamic objects used. It is possible to perform statistics on at least one set of dynamic objects used to determine the attribute values ​​for each attribute dimension.

[0093] Table 1 Attribute Dimensions and Attribute Values

[0094]

[0095] S420: Determine at least one set of simulation scenarios, with each set of simulation scenarios corresponding to one of the attribute values.

[0096] Specifically, simulation scenarios that match each attribute value are extracted from at least one simulation scenario used to form a set of simulation scenarios corresponding to that attribute value. For example, for the attribute value "Quantity 3", simulation scenarios with a total number of 3 target dynamic objects in each simulation scenario are combined into a set.

[0097] In another feasible implementation, test cases are extracted and formed according to attribute values, and the test cases correspond one-to-one with the simulation scenarios.

[0098] S430: Based on the test results, determine the pass rate information of the autonomous driving algorithm in at least one set of simulation scenarios.

[0099] In the foregoing embodiments, the test result indicates the pass / fail information of the autonomous driving algorithm in each test case, which can be represented as pass or fail. There is a one-to-one correspondence between simulation scenarios and test cases; that is, the test result can also indicate the pass / fail information of the autonomous driving algorithm in each simulation scenario. For example, for the simulation scenario set corresponding to attribute value - quantity 3, dividing the number of simulation scenarios in the set that passed the test by the total number of simulation scenarios in the set corresponding to attribute value - quantity 3 yields the test pass rate information.

[0100] S440: Determine the attribute weight corresponding to at least one attribute value.

[0101] It is feasible to determine the ratio of attribute weights corresponding to attribute values ​​based on the ratio of the number of simulation scenarios in the simulation scenario set to the number of simulation scenarios corresponding to attribute values.

[0102] It is feasible. For an attribute dimension, the sum of the attribute weights corresponding to all the attribute values ​​involved is 1.

[0103] S450: Based on the test pass rate information of the autonomous driving algorithm in at least one simulation scenario set and the attribute weight corresponding to at least one attribute value, obtain the analysis information corresponding to at least one attribute value.

[0104] It is feasible to multiply the attribute weight corresponding to the attribute value with the test pass ratio information of the simulation scenario corresponding to the attribute value to obtain the analysis information of the autonomous driving algorithm under the attribute value, that is, it can describe the performance of the autonomous driving algorithm under the attribute value.

[0105] S460: Based on the analysis information corresponding to at least one attribute value, obtain the analysis information of the autonomous driving algorithm in the attribute dimension.

[0106] It is feasible to sum the analysis information corresponding to at least one attribute value under the same attribute dimension to obtain the analysis information of the autonomous driving algorithm in the attribute dimension. The analysis information can be represented by numerical values.

[0107] For example, in the quantity dimension, assume the quantity attribute values ​​are zero, single, multiple (more than 2), and traffic flow (more than 30). Taking a single target dynamic object as an example, the analysis information corresponding to the attribute value can be expressed as: Score (single target dynamic object) = (number of simulation scenarios containing a single target dynamic object that have passed the test / number of simulation scenarios containing a single target dynamic object) * weight corresponding to the single target dynamic object. Alternatively, the weights can be designed according to the normal distribution of quantity, with the sum of the weights corresponding to different quantity attribute values ​​being 1.

[0108] For example, in terms of location, the lane position can be divided into eight positions: directly in front, directly behind, left front, left rear, right front, right rear, side-by-side left, and side-by-side right. The relative distance between the target dynamic object and the autonomous vehicle is designed as near (within 50m), medium (50m-100m), and far (more than 100m). The attribute weights corresponding to near, medium, and far can also be designed according to the test requirements. The sum of the weights of all attribute values ​​is 1. Taking the front as an example, the score (front) = (Number of simulation scenarios where the target dynamic object is directly in front of the autonomous vehicle at close range and has passed the test / Number of simulation scenarios where the target dynamic object is directly in front of the autonomous vehicle at close range) * Close range weight + Number of simulation scenarios where the target dynamic object is directly in front of the autonomous vehicle at medium range and has passed the test / Number of simulation scenarios where the target dynamic object is directly in front of the autonomous vehicle at medium range) * Medium range weight + Number of simulation scenarios where the target dynamic object is directly in front of the autonomous vehicle at far range and has passed the test / Number of simulation scenarios where the target dynamic object is directly in front of the autonomous vehicle at far range) * Far range weight. The scores for other positions and relative distances can be designed similarly to the front score. The scores for different positions are also weighted and summed according to the weights corresponding to the eight positions to obtain the score in the position dimension. A simulation scenario is considered to have a target dynamic object directly in front of the autonomous vehicle at close range if at least one target dynamic object is directly in front of the autonomous vehicle at close range; it is not necessary for all target dynamic objects to be directly in front of the autonomous vehicle at close range.

[0109] For example, in the behavioral dimension, behavioral characteristics can be represented by speed, acceleration, and steering. Speed ​​can be divided into low speed (below 30 km / h), medium speed (between 30 km / h and 60 km / h), and high speed (above 60 km / h). The weights of low speed, medium speed, and high speed can be designed according to testing requirements, and all weights are summed to a value of 1. Acceleration is divided into acceleration and deceleration, each with a weight of 1 / 2. Steering can be divided into left turn and right turn, each with a weight of 1 / 2. Taking the target dynamic object's vehicle speed as an example, Score (target dynamic object's vehicle speed) = (number of simulation scenarios where the target dynamic object is in a low-speed state and has passed the test / number of simulation scenarios where the target dynamic object is in a low-speed state) * low-speed weight + (number of simulation scenarios where the target dynamic object is in a medium-speed state and has passed the test / number of simulation scenarios where the target dynamic object is in a medium-speed state) * medium-speed weight + (number of simulation scenarios where the target dynamic object is in a high-speed state and has passed the test / number of simulation scenarios where the target dynamic object is in a high-speed state) * high-speed weight. The scoring for acceleration and steering is designed with reference to the scoring for speed.

[0110] In the above embodiments, further analysis of the test results of the autonomous driving algorithm from the attribute dimensions and attribute values ​​of dynamic objects can reveal the performance of the autonomous driving algorithm in handling dynamic objects in simulation tests based on the score. For example, if the score (for a single target dynamic object) is higher than the score (for multiple target dynamic objects), it indicates that the autonomous driving performance deteriorates as the number of dynamic objects increases. This allows us to understand the degree of influence of the various dimensions of the target dynamic object's attributes on the autonomous driving algorithm, and to evaluate the performance of the autonomous driving algorithm in more detail. This enables more targeted iterative optimization of the autonomous driving algorithm, reducing the risks and costs of road testing.

[0111] In one embodiment of this application, the method may further include:

[0112] The analysis information of the autonomous driving algorithm in each attribute dimension of the target dynamic object is added together to obtain the target analysis information of the autonomous driving algorithm. The target analysis information indicates the comprehensive performance of the autonomous driving algorithm.

[0113] Furthermore, weights can be designed for different attribute dimensions according to functional or testing requirements, with each weight reflecting the importance of the attribute dimension. A weighted sum is then performed based on the weights of each attribute dimension and the analytical information for that dimension to obtain target analysis information that can measure the overall performance of the autonomous driving algorithm.

[0114] In the above embodiments, in addition to analyzing and evaluating the merits of autonomous driving algorithms in detail from different attribute dimensions, an overall measurement can also be performed, and it can also serve as a basis for comparison before and after algorithm iteration and optimization.

[0115] As can be seen from the above embodiments, the autonomous driving testing method based on simulation scenarios provided in this application, by integrating simulation scenarios with virtual traffic dynamic objects, can conduct more comprehensive and realistic testing of autonomous driving algorithms. Furthermore, the simulation scenarios in this application are no longer limited to testing only a single autonomous driving function, but can test the entire process of high-level autonomous driving, meeting the requirements of high-level autonomous driving testing for the completeness, comprehensiveness, and accuracy of simulation testing. In addition, the solution provided in this application analyzes the test results of the autonomous driving algorithm from the attribute dimension of virtual traffic dynamic objects, which can reveal the processing performance of the autonomous driving algorithm on each virtual traffic dynamic object in the simulation test, thereby enabling a more detailed and accurate evaluation of the autonomous driving algorithm and promoting its optimization and iteration.

[0116] This application also provides an autonomous driving test device 500 based on a simulation scenario, such as... Figure 5 As shown, the device 500 may include:

[0117] The dynamic object acquisition module 510 is used to acquire at least one dynamic object set, the dynamic object set including at least one target dynamic object, the target dynamic object representing a virtual traffic dynamic object;

[0118] The simulation scene generation module 520 is used to generate at least one simulation scene based on the at least one set of dynamic objects.

[0119] Test module 530 is used to perform tests on the autonomous driving algorithm based on the at least one simulation scenario and obtain test results;

[0120] Analysis module 540 is used to determine the analysis results of the autonomous driving algorithm based on the test results; the analysis results indicate the analysis information of the autonomous driving algorithm in each attribute dimension of the target dynamic object.

[0121] In one embodiment of this application, the analysis module 540 may include:

[0122] An attribute dimension determination unit is used to determine the attribute dimensions of the at least one dynamic object set, wherein the attribute dimension includes at least one attribute value;

[0123] A simulation scene set determination unit is used to determine at least one simulation scene set, wherein the simulation scene set corresponds one-to-one with the attribute value;

[0124] A ratio information determination unit is used to determine the test pass ratio information of the autonomous driving algorithm in the at least one simulation scenario set based on the test results.

[0125] An attribute weight determination unit is used to determine the attribute weight corresponding to the at least one attribute value;

[0126] The first analysis unit is used to obtain analysis information corresponding to the at least one attribute value based on the test pass rate information of the autonomous driving algorithm in the at least one simulation scenario set and the attribute weights corresponding to the at least one attribute value.

[0127] The second analysis unit is used to obtain the analysis information of the autonomous driving algorithm in the attribute dimension based on the analysis information corresponding to the at least one attribute value.

[0128] In one embodiment of this application, the device 500 may further include:

[0129] The comprehensive analysis module is used to add up the analysis information of the autonomous driving algorithm in each attribute dimension of the target dynamic object to obtain the target analysis information of the autonomous driving algorithm, which indicates the comprehensive performance of the autonomous driving algorithm.

[0130] In one embodiment of this application, the dynamic object acquisition module 510 may include:

[0131] The first acquisition unit is used to acquire a dynamic object library, wherein the dynamic object library includes at least one dynamic object;

[0132] The second acquisition unit is used to select at least one dynamic object from the dynamic object library as at least one target dynamic object, and obtain the dynamic object set from the at least one target dynamic object.

[0133] In one embodiment of this application, the simulation scene generation module 520 may include:

[0134] The third acquisition unit is used to acquire the initial static simulation scene; the static simulation scene includes at least one target static object;

[0135] An object adding unit is used to add at least one target dynamic object from the dynamic object set to the static simulation scene to obtain the simulation scene corresponding to the dynamic object set.

[0136] In one embodiment of this application, the device 500 may further include:

[0137] Based on the real distribution information corresponding to the virtual traffic dynamic objects, configure the object attribute information of at least one target dynamic object in the dynamic object set; the object attribute information includes quantity information, location information or behavior information.

[0138] In one embodiment of this application, the test module 530 may include:

[0139] The fourth acquisition unit is used to acquire the vehicle dynamics model, the on-board sensor model, and the autonomous driving algorithm;

[0140] An integration unit is used to integrate the at least one simulation scenario, the vehicle dynamics model, the on-board sensor model, and the autonomous driving algorithm to obtain at least one test case; the test case corresponds one-to-one with the simulation scenario;

[0141] A testing unit is used to execute the at least one test case and obtain test results; the test results indicate the pass information of the autonomous driving algorithm in the at least one test case.

[0142] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0143] This application provides a computer device including a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement an autonomous driving test method based on a simulation scenario as provided in the above method embodiments.

[0144] Figure 6 A schematic diagram of the hardware structure of a device for implementing a simulation-based autonomous driving testing method provided in the embodiments of this application is shown. This device may constitute or include the apparatus or system provided in the embodiments of this application. Figure 6 As shown, device 10 may include one or more processors 1002 (shown as 1002a, 1002b, ..., 1002n in the figure) 1002 (processor 1002 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 1004 for storing data, and a transmission device 1006 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 6 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, device 10 may also include a... Figure 6 The more or fewer components shown, or having the same Figure 6 The different configurations shown.

[0145] It should be noted that the aforementioned one or more processors 1002 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be wholly or partially integrated into any other element within device 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0146] The memory 1004 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method described in the embodiments of this application. The processor 1002 executes various functional applications and data processing by running the software programs and modules stored in the memory 1004, thereby realizing the above-mentioned autonomous driving test method based on simulation scenarios. The memory 1004 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1004 may further include memory remotely located relative to the processor 1002, and these remote memories can be connected to the device 10 via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0147] The transmission device 1006 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of device 10. In one example, the transmission device 1006 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 1006 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0148] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows a user to interact with the user interface of device 10 (or a mobile device).

[0149] This application embodiment also provides a computer-readable storage medium, which can be disposed in a server to store at least one instruction or at least one program related to implementing an autonomous driving test method based on a simulation scenario in the method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the autonomous driving test method based on a simulation scenario provided in the above method embodiment.

[0150] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0151] This invention also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform an autonomous driving testing method based on a simulation scenario provided in the various optional embodiments described above.

[0152] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0153] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device, equipment, and storage medium embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0154] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0155] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A simulation scenario-based automatic driving test method, characterized by, The method includes: At least one set of dynamic objects is obtained, the set of dynamic objects including at least one target dynamic object, the target dynamic object representing a virtual traffic dynamic object; the object attribute information of the target dynamic object is configured based on the real distribution information corresponding to the virtual traffic dynamic object. Based on at least one target dynamic object in each of the dynamic object sets, generate at least one simulation scene; Based on at least one simulation scenario, the autonomous driving algorithm is tested, and the test results are obtained. Based on the test results, the analysis results of the autonomous driving algorithm are determined; the analysis results indicate the analysis information of the autonomous driving algorithm in each attribute dimension of the at least one target dynamic object; The step of determining the analysis results of the autonomous driving algorithm based on the test results includes: Determine the attribute dimension of the at least one target dynamic object, wherein the attribute dimension corresponds to at least one attribute value; Determine at least one set of simulation scenarios, wherein the at least one set of simulation scenarios corresponds one-to-one with the at least one attribute value; Based on the test results, determine the test pass rate information of the autonomous driving algorithm in the at least one simulation scenario set; Determine the attribute weight corresponding to the at least one attribute value; Based on the test pass rate information of the autonomous driving algorithm in the at least one simulation scenario set and the attribute weights corresponding to the at least one attribute value, analysis information corresponding to the at least one attribute value is obtained; Based on the analysis information corresponding to the at least one attribute value, the analysis information of the autonomous driving algorithm in the corresponding attribute dimension is obtained.

2. The method of claim 1, wherein, The method further includes: The analysis information of the autonomous driving algorithm in each attribute dimension of the target dynamic object is added together to obtain the target analysis information of the autonomous driving algorithm, which indicates the comprehensive performance of the autonomous driving algorithm.

3. The method of claim 1, wherein, The process of obtaining at least one set of dynamic objects includes: Obtain a dynamic object library, wherein the dynamic object library includes at least one dynamic object; At least one dynamic object is selected from the dynamic object library as at least one target dynamic object, and the dynamic object set is obtained from the at least one target dynamic object.

4. The method of claim 1, wherein, The step of generating at least one simulation scene based on the at least one set of dynamic objects includes: Obtain an initial static simulation scene; the static simulation scene includes at least one target static object; At least one target dynamic object from the dynamic object set is added to the static simulation scene to obtain the simulation scene corresponding to the dynamic object set.

5. The method according to claim 1, characterized in that, The method further includes: The object attribute information includes quantity information, location information, or behavior information.

6. The method according to claim 1, characterized in that, The test, based on the at least one simulation scenario, is performed on the autonomous driving algorithm to obtain test results, including: Acquire vehicle dynamics models, onboard sensor models, and autonomous driving algorithms; The at least one simulation scenario, the vehicle dynamics model, the on-board sensor model, and the autonomous driving algorithm are integrated to obtain at least one test case; the test case corresponds one-to-one with the simulation scenario. Execute the at least one test case to obtain test results; the test results indicate the pass information of the autonomous driving algorithm in the at least one test case.

7. An autonomous driving test device based on a simulation scenario, characterized in that, The device includes: A dynamic object acquisition module is used to acquire at least one set of dynamic objects, the set of dynamic objects including at least one target dynamic object, the target dynamic object representing a virtual traffic dynamic object; the object attribute information of the target dynamic object is configured based on the real distribution information corresponding to the virtual traffic dynamic object. The simulation scene generation module is used to generate at least one simulation scene based on at least one target dynamic object in each of the dynamic object sets; The testing module is used to perform tests on the autonomous driving algorithm based on at least one simulation scenario and obtain test results. An analysis module is used to determine the analysis results of the autonomous driving algorithm based on the test results; the analysis results indicate the analysis information of the autonomous driving algorithm in each attribute dimension of the at least one target dynamic object; The analysis module includes: An attribute dimension determination unit is used to determine the attribute dimension of the at least one target dynamic object, wherein the attribute dimension corresponds to at least one attribute value; A simulation scene set determination unit is used to determine at least one simulation scene set, wherein the at least one simulation scene set corresponds one-to-one with the at least one attribute value; A ratio information determination unit is used to determine the test pass ratio information of the autonomous driving algorithm in the at least one simulation scenario set based on the test results. An attribute weight determination unit is used to determine the attribute weight corresponding to the at least one attribute value; The first analysis unit is used to obtain analysis information corresponding to the at least one attribute value based on the test pass rate information of the autonomous driving algorithm in the at least one simulation scenario set and the attribute weights corresponding to the at least one attribute value. The second analysis unit is used to obtain the analysis information of the autonomous driving algorithm in the corresponding attribute dimension based on the analysis information corresponding to the at least one attribute value.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement an autonomous driving test method based on a simulation scenario as described in any one of claims 1 to 6.

9. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or at least one program being loaded and executed by the processor to implement an autonomous driving test method based on a simulation scenario as described in any one of claims 1 to 6.

Images