Simulation testing methods, electronic devices and computer storage media

Abstract

This application provides a simulation testing method, electronic device, and computer storage medium. The simulation testing method includes: simulation playback testing based on road test data to identify key obstacles, wherein the key obstacles are traffic objects that affect the driving in the autonomous driving simulation and are the responsible party in the collision; determining multiple interaction strategies for the interaction between the key obstacles and the autonomous driving simulation device based on scene information corresponding to the driving impact; using the multiple interaction strategies to perform interaction deduction between the key obstacles and the autonomous driving simulation device; and obtaining simulation test results based on the interaction deduction results. According to this application embodiment, the overall efficiency of simulation testing using road test data is improved, and the participation of complex prediction models is not required, thus reducing the implementation cost of the solution.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a simulation testing method, electronic device, and computer storage medium. Background Technology

[0002] With the development of autonomous driving technology, more and more industries and fields are using equipment with autonomous driving capabilities (such as vehicles and aircraft) to perform corresponding tasks, in order to improve work efficiency and reduce the burden of manual labor. In order to ensure the driving safety of these devices in actual operation, autonomous driving testing has become an essential step.

[0003] In autonomous driving testing, proving the safety of autonomous vehicles through road tests is extremely costly, while simulation testing offers a lower-cost and safer approach. Simulation testing extracts key scenarios from road test data, effectively simulating autonomous driving scenarios and allowing for continuous iteration and updates to the simulation testing algorithm. However, due to differences in decision-making and planning between different algorithm versions, when the position of the simulated autonomous vehicle differs from that of the vehicle in the road test data, and the environmental obstacles in the road test playback data are still recreated in their original positions, unreasonable interactions between the simulated autonomous vehicle and the obstacles in the road test playback environment can easily occur. This leads to numerous false alarms such as unreasonable sudden braking and collisions, significantly reducing testing efficiency. To address this, one related solution uses collision prediction and unified obstacle control to handle this situation. However, this approach requires a highly accurate collision prediction model, and the unified control method is relatively simplistic, making it difficult to achieve diverse interactive tests and cover real-world testing scenarios.

[0004] Therefore, how to effectively utilize road test data for simulation testing and improve the efficiency of simulation testing has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, embodiments of this application provide a simulation testing scheme to at least partially solve the above-mentioned problems.

[0006] According to a first aspect of the embodiments of this application, a simulation testing method is provided, comprising: conducting simulation playback tests based on road test data to identify key obstacles, wherein the key obstacles are traffic objects that affect the driving of an autonomous driving simulation device and are responsible for the impact; determining multiple interaction strategies for the interaction between the key obstacles and the autonomous driving simulation device based on scene information corresponding to the driving impact; performing interaction simulation between the key obstacles and the autonomous driving simulation device using the multiple interaction strategies; and obtaining simulation test results based on the interaction simulation results.

[0007] According to a second aspect of the present application, an electronic device is provided, including: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus; the memory is used to store at least one executable instruction, which causes the processor to perform an operation corresponding to the method described in the first aspect.

[0008] According to a third aspect of the embodiments of this application, a computer storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0009] According to the solution provided in this application, during the simulation playback test using road test data, key obstacles that bear responsibility for the impact on driving are identified. Then, for the interaction between these key obstacles and the autonomous driving simulation device in this scenario, multiple interaction strategies are determined. Based on these multiple interaction strategies, corresponding interaction deductions are performed to achieve various interaction tests in this scenario. Through multiple interaction tests, on the one hand, the autonomous driving simulation device can be effectively tested multiple times from different interaction angles to determine its performance and expand the driving scenarios that can be covered by the test; on the other hand, the effective utilization of road test data is achieved; furthermore, the true cause of the driving impact on the autonomous driving simulation device can be determined based on the simulation test results, such as whether it is due to defects in the autonomous driving simulation device or false alarms caused by the simulation test algorithm version, thereby exposing potential simulation playback test problems. Therefore, the overall efficiency of simulation testing using road test data is improved, and the participation of complex prediction models is not required, reducing the implementation cost of the solution. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0011] Figure 1 A schematic diagram of an exemplary system to which the embodiments of this application are applicable;

[0012] Figure 2A This is a flowchart illustrating the steps of a simulation testing method according to an embodiment of this application.

[0013] Figure 2B for Figure 2A A schematic diagram of a scenario example in the illustrated embodiment;

[0014] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0015] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0016] The specific implementation of the embodiments of this application will be further described below with reference to the accompanying drawings.

[0017] Figure 1 An exemplary system applicable to embodiments of this application is shown. For example... Figure 1 As shown, the system 100 may include a cloud server 102, a communication network 104, and / or one or more user devices 106. Figure 1 The example uses multiple user devices. In this example, user device 106 can be implemented as a test terminal device for performing autonomous driving simulation tests.

[0018] The cloud server 102 can be any suitable device for storing information, data, programs, and / or any other suitable type of content, including but not limited to distributed storage system devices, server clusters, computing cloud server clusters, etc. In some embodiments, the cloud server 102 can store at least drive test data. However, it is not limited to this; other data, as well as process data and result data from simulation playback tests performed by user equipment 106, can also be stored in the cloud server 102.

[0019] In some embodiments, the communication network 104 can be any suitable combination of one or more wired and / or wireless networks. For example, the communication network 104 can include any one or more of the following: the Internet, an intranet, a wide area network (WAN), a local area network (LAN), a wireless network, a digital subscriber line (DSL) network, a frame relay network, an asynchronous transfer mode (ATM) network, a virtual private network (VPN), and / or any other suitable communication network. The user equipment 106 can be connected to the communication network 104 via one or more communication links (e.g., communication link 112), and the communication network 104 can be linked to the cloud server 102 via one or more communication links (e.g., communication link 114). The communication link can be any communication link suitable for transmitting data between the user equipment 106 and the cloud server 102, such as a network link, a dial-up link, a wireless link, a hardwired link, any other suitable communication link, or any suitable combination of such links.

[0020] User equipment 106 may include any one or more user equipments suitable for conducting autonomous driving simulation tests. In some embodiments, user equipment 106 may perform simulation playback tests based on road test data. Due to iterative updates of the test algorithm version formed by simulation playback tests based on road test data, unreasonable interactions may occur between the autonomous driving simulation vehicle and obstacles in the road test playback environment, leading to a large number of false alarms. Therefore, in some embodiments, user equipment 106 may identify key obstacles during simulation playback tests based on road test data; these are traffic objects (including but not limited to motor vehicles or non-motor vehicles) that interact with the autonomous driving simulation device and affect its driving, and are responsible for the impact. Then, based on the scenario corresponding to the driving impact, multiple interaction strategies are determined for the interaction between the key obstacles and the autonomous driving simulation device. Under these multiple interaction strategies, corresponding multiple interaction simulations are performed to identify problems with the autonomous driving simulation device or determine data false alarms based on the interaction simulation results. In some embodiments, user equipment 106 may include any suitable type of device. For example, in some embodiments, user equipment 106 may include mobile devices, tablet computers, laptop computers, desktop computers, vehicle platform systems and / or any other suitable type of user equipment.

[0021] Based on the above system, this application provides a simulation testing method, which will be described below through several embodiments.

[0022] Reference Figure 2A The flowchart illustrates the steps of a simulation testing method according to an embodiment of this application.

[0023] The simulation testing method in this embodiment includes the following steps:

[0024] Step S202: Based on the road test data, conduct a simulation playback test to identify key obstacles.

[0025] Among them, the key obstacle is a traffic object that interacts with the autonomous driving simulation equipment and affects the driving of the autonomous driving simulation equipment (such as causing a collision), and is the party responsible for the impact.

[0026] Road test data is generally obtained by sampling data from an autonomous vehicle in a real physical environment. This road test data can usually describe the physical environment in which the autonomous vehicle is located and its interaction with other traffic objects in that environment. In the embodiments of this application, there are no restrictions on the method of obtaining the road test data, the specific data format, or the physical environment it describes.

[0027] Based on the obtained road test data, simulation playback tests of autonomous driving simulation equipment can be conducted using any appropriate method or algorithm. In this simulation playback test, the autonomous driving simulation equipment will use the physical environment in which the autonomous driving data collection vehicle performed sampling as the simulation environment and simulate interactions with various traffic objects in that environment. However, as mentioned earlier, the simulation playback test algorithm will be continuously iterated and updated. In each iteration, the interaction strategy adopted by the autonomous driving simulation equipment when interacting with other traffic objects may be different, which may generate abnormal information.

[0028] If the generated abnormal information directly indicates that there is a problem or malfunction in the autonomous driving simulation equipment (such as a collision that occurred and the autonomous driving simulation equipment is at fault, or the autonomous driving simulation equipment has problems such as sudden braking or forced stopping), then the problem or malfunction, together with the scene information and driving information (including but not limited to interaction strategies and driving trajectories), will be sent to the preset analysis equipment so that the analysis equipment can analyze the problem or malfunction and then optimize the autonomous driving simulation equipment accordingly.

[0029] If the generated abnormal information indicates that the autonomous driving simulation device has interacted with other traffic objects, and this interaction has adversely affected the device's driving (e.g., a collision), and the party responsible for the collision is another traffic object, then it's possible that a version update has caused unreasonable interactions between the autonomous driving simulation device and other traffic objects. For example, the position of the autonomous driving simulation device might be inconsistent with the autonomous driving data collection vehicle in the road test data, while environmental obstacles in the road test playback data are still recreated in their original positions to test the autonomous driving system. This could lead to unreasonable false alarms such as sudden braking or collisions from the autonomous driving simulation device. In this case, further verification and processing of the driving impact, such as the collision, is required.

[0030] Therefore, in this embodiment of the application, key obstacles in the simulation playback test will be identified, that is, those traffic objects that interact with the autonomous driving simulation device and affect the driving of the autonomous driving simulation device, and that affect the responsible party, will be identified for further processing.

[0031] In one feasible approach, simulation playback tests can be performed based on road test data to identify key data frames that affect the driving of the autonomous driving simulation device (such as collisions). From these key data frames, key obstacles can be identified. By identifying key data frames first and then key obstacles, the identification of key obstacles becomes more efficient, and the system's data processing burden is reduced. For example, the key data frame could be the data frame corresponding to the moment a collision occurs, or multiple data frames related to the moment a collision occurs, etc. However, it is not limited to these; key data frames affecting driving can also be identified through intelligent annotation. Intelligent annotation is a machine learning-assisted technique that uses a machine learning model for assisted annotation. The annotator only needs to annotate the target object once, and the model can track and predict its position and driving situation in subsequent frames. Based on this, during simulation playback tests, one or more frames that affect the driving of the autonomous driving simulation device, such as collisions, can be identified as key data frames. Furthermore, multiple parties involved in a collision are identified from key data frames. For example, consider two parties: the autonomous driving simulation device and another traffic object. Further, responsibility among the parties can be determined based on a responsibility allocation model such as the RSS (Responsibility Sensitive Safety) model. Taking a collision as an example, after identifying the key data frame showing a collision between the autonomous driving simulation device and another traffic object, the responsibility of each party in that key data frame, such as the autonomous driving simulation device and the other traffic object, can be determined using the RSS model. If the responsibility lies with the other traffic object, it is identified as a critical obstacle. If the responsibility lies with the autonomous driving simulation device, as mentioned earlier, the relevant information will be directly sent to the backend analysis equipment for analysis.

[0032] To facilitate subsequent testing in the same scenario and fully expose any issues with the autonomous driving simulation equipment or test data, one feasible approach is to obtain, after acquiring the key data frames, the motion trajectories of key obstacles and the autonomous driving simulation equipment can be obtained based on these frames. Based on the obtained motion trajectories, the corresponding scenario information affecting driving can be determined, such as collision scenario information. This scenario information includes, but is not limited to: information on the driving routes of key obstacles and the autonomous driving simulation equipment, driving direction information, driving behavior information, driving speed information, relative position information, relative velocity information, surrounding environment and other traffic objects, etc.

[0033] Through the above process, key obstacles were identified and scene information was collected, providing conditions for subsequent interactive simulations.

[0034] Step S204: Based on the scene information corresponding to the driving impact, determine multiple interaction strategies for the interaction between key obstacles and autonomous driving simulation equipment.

[0035] For scenarios identified as being caused by critical obstacles, the occurrence of such scenarios can be due to various possibilities. For example, a false alarm might arise due to a discrepancy between the location of the autonomous driving simulation equipment and the location of the autonomous driving data collection vehicle in the road test data; or the driving abnormality, such as a collision, might genuinely be caused by the autonomous driving simulation equipment or the critical obstacle. Therefore, for this scenario, multiple tests with different interactions can be conducted to identify possible causes and potential problems.

[0036] In one feasible approach, multiple interaction strategies between key obstacles and autonomous driving simulation equipment can be determined through Monte Carlo tree search based on scene information corresponding to driving impacts. Monte Carlo tree search methods are mostly based on the current situation, simulating possible future scenarios, and identifying currently feasible actions from multiple actionable options. Specifically, in this application embodiment, taking a collision scenario as an example, multiple interaction strategies between key obstacles and autonomous driving simulation equipment can be determined through Monte Carlo tree search based on collision scenario information, including the relative position and speed between the key obstacle and the autonomous driving simulation equipment, its position and speed with other traffic objects, and the surrounding traffic environment (lanes, speed limits, traffic signals, etc.). It should be noted that the method is not limited to Monte Carlo tree search; other methods for determining interaction strategies, such as using a trained convolutional neural network model, are also applicable to the scheme of this application embodiment. However, the Monte Carlo tree search method can obtain more reasonable interaction strategies, thereby improving the effectiveness and efficiency of subsequent testing.

[0037] However, to improve the efficiency of interaction and testing, in one feasible approach, an interaction target can be set. Specifically, based on the scene information corresponding to the driving impact, an interaction target can be obtained for the interaction between the key obstacle and the autonomous driving simulation device in the scene indicated by the scene information. Based on the interaction target, multiple candidate interaction strategy sequences are generated. These candidate interaction strategy sequences are evaluated according to a preset cost evaluation function. Based on the evaluation results, the target interaction strategy sequence is determined. The interaction target indicates the desired outcome of the interaction. For example, the key obstacle may cut in front of the autonomous driving simulation device, or remain behind the autonomous driving simulation device, or change lanes to reach the side of the autonomous driving simulation device, etc. The interaction target can be appropriately set by those skilled in the art according to the actual situation. In this embodiment, the specific implementation of the interaction target is not limited.

[0038] Based on the interaction objective, multiple candidate interaction strategy sequences can be generated. For example, Monte Carlo tree search can be used to generate various candidate interaction strategy sequences. A sequence is a combination of a series of interactive behaviors, such as lane change-acceleration-cutting, which forms a candidate interaction strategy sequence. There may be multiple interaction strategy sequences to achieve a certain interaction objective. For example, cutting can be implemented as left lane change-acceleration-cutting, right lane change-acceleration-cutting, or following-lane change-cutting, etc. However, the purpose of further interaction and testing is to achieve reasonable and effective interaction while avoiding conflicts. Therefore, the generated multiple candidate interaction strategy sequences can be evaluated using a preset cost evaluation function to determine the optimal sequence, i.e., the target interaction strategy sequence.

[0039] Optionally, the cost evaluation function may include at least one of the following: a first cost evaluation function for evaluating the efficiency of achieving the interaction goal based on the candidate interaction strategy sequence; a second cost evaluation function for evaluating the smoothness of the motion trajectory of the key obstacle when it moves based on the candidate interaction strategy sequence; and a third cost evaluation function for evaluating the interaction difficulty between the key obstacle and the autonomous driving simulation device when it moves based on the candidate interaction strategy sequence. For driving decisions, evaluation can be conducted from multiple aspects such as comfort, safety, and stability. Based on this, this embodiment of the application, while fully considering the interaction goal, selects multiple dimensions such as the efficiency of achieving the interaction goal, the smoothness of the motion trajectory, and the interaction difficulty to evaluate whether the candidate interaction strategy sequence is reasonable and efficient.

[0040] The evaluation of multiple candidate interaction strategy sequences based on a preset cost evaluation function may include at least one of the following: evaluating the efficiency of achieving the interaction goal using a first cost evaluation function based on the time required to achieve the interaction goal; evaluating the smoothness of the key obstacle's trajectory using a second cost evaluation function based on the acceleration and average acceleration of the key obstacle when moving based on multiple candidate interaction strategy sequences; and evaluating the interaction difficulty between the key obstacle and the autonomous driving simulation device using a third cost evaluation function based on the shortest interaction distance between the key obstacle and the autonomous driving simulation device when moving based on multiple candidate interaction strategy sequences. It should be noted that the specific implementation of the first, second, and third cost evaluation functions in this embodiment is not limited. Those skilled in the art can implement the cost evaluation functions using any appropriate formula or algorithm, taking into account the factors considered in the aforementioned cost evaluation functions, to evaluate multiple candidate interaction strategy sequences, according to actual needs. For example, the first cost evaluation function can be evaluated by comparing the implementation time of various candidate interaction strategy sequences in the test, thereby determining the implementation efficiency level; or, the implementation time of various candidate interaction strategy sequences in the test can be compared with the average implementation time of the interaction target to compare the various candidate interaction strategy sequences and determine the implementation efficiency level, and so on. For example, the first cost evaluation function includes, but is not limited to, functions for calculating simulation duration, average simulation speed, simulation mileage, average simulation motion speed, number of stops, total parking time, and maximum parking time. The second and third cost evaluation functions can also adopt a similar approach. For example, the second cost evaluation function may include, but is not limited to: emergency braking evaluation function (statistically counting the number of frames in each frame where the acceleration x, y values ​​and the x, y values ​​in the emergency braking direction exceed a specified interval, and the number of frames exceeds a certain threshold), large steering (steering wheel angle exceeds a certain threshold) function, etc. The third cost evaluation function may include, but is not limited to: a function that calculates the distance based on TTC (Time-To-Collision), etc.

[0041] Through evaluation, one or more strategy sequences can be selected from a variety of candidate interaction strategy sequences to serve as the target interaction strategy sequence. For comprehensive testing, multiple target interaction strategy sequences are generally acceptable.

[0042] In addition, to ensure that the critical obstacle can subsequently perform reasonable driving behavior, one feasible approach is to add a strategy of maintaining its current lane after determining the target interaction strategy sequence. That is, after executing a series of interaction actions indicated by the target interaction strategy sequence, the critical obstacle will maintain its position in the lane where the last interaction action was completed to perform further operations and ensure driving stability.

[0043] Step S206: Use multiple interaction strategies to simulate the interaction between key obstacles and autonomous driving simulation equipment.

[0044] In this embodiment of the application, interactive simulation refers to the testing process in which key obstacles and autonomous driving simulation equipment drive according to a series of interactive actions indicated by various determined interactive strategies in the scenario corresponding to the aforementioned driving influence, such as the collision scenario indicated by the collision scenario information, and obtain the interactive results.

[0045] To ensure the smooth operation of this process, one feasible approach is to perform trajectory planning for multiple interaction strategies. Based on these strategies and their corresponding trajectory plans, the interaction between key obstacles and the autonomous driving simulation device is simulated. In practical implementation, the trajectory planning can be sampled according to the interaction target. For example, during lane-changing decisions, different lane offsets and lane-changing times are sampled on the target lane. Under constraints of acceleration and speed ranges, the fastest lane-changing trajectory is found. A fifth-order polynomial is used to solve for the trajectory of continuous motion from the current driving state to the target driving state, obtaining the position, speed, and acceleration state of the key obstacle, etc. For example, the Lattice trajectory planning algorithm can be used. The Lattice algorithm is a sampling-based motion planning algorithm. It transforms the vehicle coordinate system to the reference line coordinate system, i.e., the Frenet coordinate system. Then, it plans the d-axis and s-axis of the Frenet coordinate system separately to form the planned trajectory in the Frenet coordinate system. Finally, the trajectory in the Frenet coordinate system is synthesized and restored to the world coordinate system. One specific implementation of the Lattice planning algorithm is the use of LatticePlanner, a local trajectory planner for vehicle motion. The output trajectory is directly input to the controller, which then performs tracking control of the local trajectory. Therefore, the trajectory output by LatticePlanner is a smooth, collision-free, stable, and safe local trajectory that satisfies vehicle kinematic and velocity constraints. Its input mainly includes three parts: perception and obstacle information, reference line information, and positioning information; its output is a trajectory composed of a series of trajectory points with velocity information, ensuring the stability and safety of the vehicle controller during vehicle tracking control. However, this is not the only approach; other methods of implementing trajectory rules based on interaction strategies, such as using deep learning models, are also applicable to the solutions in this application.

[0046] Since there may be multiple interaction strategies to be simulated in the embodiments of this application, in order to improve the efficiency of interaction simulation, in one feasible approach, multiple interaction strategies can be used to perform multiple interaction simulations between key obstacles and autonomous driving simulation equipment in parallel, corresponding to multiple interaction strategies. This parallel approach can be implemented by those skilled in the art according to the actual situation using appropriate parallel methods. For example, multiple identical collision scenarios can be started through multiple processes, and different interaction strategies can be used in different collision scenarios to perform interaction simulations according to the trajectory rules corresponding to each interaction strategy; or, multiple identical collision scenarios can be presented on the same interface, but different interaction strategies can be used in different collision scenarios to perform interaction simulations according to the trajectory rules corresponding to each interaction strategy, etc., all of which are within the protection scope of the embodiments of this application.

[0047] Step S208: Obtain simulation test results based on the interactive deduction results.

[0048] After performing interaction simulations between key obstacles and the autonomous driving simulation device according to various interaction strategies, corresponding interaction simulation results will be obtained. These simulation results may indicate that the impact has been eliminated, such as eliminating the collision, suggesting that the original collision may have been a false alarm caused by road test data; however, they may also indicate that a collision occurred again. In this case, it is necessary to determine, based on the interaction simulation results, whether there is an interaction in the corresponding interaction simulation where the interaction between the key obstacle and the autonomous driving simulation device causes a driving impact, such as a collision, and the autonomous driving simulation device is the responsible party for the impact; if so, the interaction simulation information corresponding to that interaction is identified as auxiliary information for the unresolved issues of the autonomous driving simulation device. This information can then be sent to the backend analysis equipment for targeted analysis. This approach ensures the effectiveness of simulation testing while enriching the diversity of the testing process.

[0049] This embodiment identifies key obstacles responsible for impacting driving conditions during simulation playback testing using road test data. Then, for each scenario, multiple interaction strategies are determined between these key obstacles and the autonomous driving simulation device. Based on these strategies, corresponding interaction simulations are performed to achieve various interaction tests within that scenario. Through these multiple interaction tests, the autonomous driving simulation device can be effectively tested from different interaction angles to determine its performance and expand the driving scenarios covered by the tests. Furthermore, road test data is effectively utilized. Additionally, the simulation test results can determine the true cause of the impact on the autonomous driving simulation device's driving, such as whether it's due to a defect in the device or a false alarm caused by a different simulation algorithm version, thus exposing potential simulation playback test problems. Therefore, this approach improves the overall efficiency of simulation testing using road test data without requiring complex prediction models, reducing the implementation cost.

[0050] The following example, using a collision scenario, illustrates the above process. Figure 2B As shown.

[0051] like Figure 2B As shown, the process includes:

[0052] Step S302: Perform simulation playback test based on the road test data.

[0053] In this step, after obtaining the road test data and determining the autonomous driving algorithm to be tested, simulation playback testing can be performed. Optionally, in this example, scene data before and after key data frames will also be identified and collected as collision scene information.

[0054] When conducting simulation playback tests, the autonomous driving algorithm under test can be used. The position, speed, acceleration, heading angle, etc. of the road test obstacles in the road test data can be used for playback as the perception input of the autonomous driving algorithm under test, driving the prediction, decision-making and planning control module of the autonomous driving simulation vehicle to run.

[0055] When the autonomous driving simulation vehicle and the road test data collection vehicle make different decisions, such as acceleration / deceleration or lane changing, some road test obstacles will collide with the autonomous driving simulation vehicle due to the lack of interaction.

[0056] In this example, the data frame corresponding to the collision moment is used as the key data frame. However, as mentioned earlier, key data frames can also be identified through intelligent annotation and other means.

[0057] Based on key data frames, collision liability can be determined according to traffic regulations. In specific implementations, an RSS model can be used to determine collision liability. If a road test obstacle bears primary responsibility, such as in a rear-end collision with an autonomous driving simulation vehicle, then the road test obstacle will be identified as a key obstacle, and its behavior needs to be simulated and retested to determine the validity of the collision.

[0058] In this example, the movement trajectories of all road-tested obstacles and the autonomous driving simulation vehicle are recorded for 10 seconds before and after the collision. This provides scenario information for subsequent simulations of the effective interaction behavior of key obstacles. If the primary responsibility for the collision lies with the autonomous driving simulation vehicle—for example, if the vehicle shows no deceleration or other response when faced with another vehicle cutting in—this scenario is not simulated; instead, the simulation test is directly deemed a failure. The relevant data is then sent to backend analysis equipment for further analysis of the autonomous driving simulation vehicle's problems.

[0059] As can be seen, this step involves replaying the original road test data for simulation playback testing. Key obstacles are identified through methods such as obstacle collision detection and intelligent labeling. The motion trajectory data of the obstacles and the autonomous driving simulation device before and after the key data frames are collected and recorded. Thus, the initial simulation is completed, collision liability is determined, and relevant data input and scene information are provided for subsequent key obstacle interaction simulations.

[0060] Step S304: Determine the interaction strategy for the autonomous driving simulation device and key obstacles in the collision scenario.

[0061] First, by using the motion trajectories of road test obstacles, including key obstacles, before and after the collision, as well as the motion trajectory of the autonomous driving simulation equipment, i.e., the collision scene information, the real collision scene is reconstructed.

[0062] Based on this, Monte Carlo tree search can be used to determine the interaction strategy. Monte Carlo tree search combines the generality of stochastic simulation with the accuracy of tree search, providing a better decision, i.e., the interaction strategy.

[0063] In this example, the determination of the interaction strategy also takes into account the interaction objectives. For example, given the interaction objectives between a key obstacle and the autonomous driving simulation device, such as the key obstacle cutting in front of the autonomous driving simulation device, maintaining a following distance behind the autonomous driving simulation device, or changing lanes to reach the side of the autonomous driving simulation device, a better sequence of target interaction strategies is determined through multiple cost evaluation functions to achieve reasonable and effective interaction while avoiding collisions.

[0064] The cost evaluation function includes at least one of the following: a cost evaluation function for evaluating the efficiency of achieving the interaction goal (first cost evaluation function), which can be achieved by evaluating the time taken to achieve the interaction goal; a cost evaluation function for evaluating the comfort of the trajectory, i.e., a cost evaluation function for evaluating the smoothness of the motion trajectory (second cost evaluation function), which can be achieved by evaluating the acceleration of the key obstacle and the mean of the acceleration; and a cost evaluation function for evaluating the challenge of the interaction, i.e., a cost evaluation function for evaluating the difficulty of the interaction (third cost evaluation function), which can be achieved by evaluating the shortest interaction distance with the key obstacle.

[0065] Taking the interaction target of a key obstacle cutting into the autonomous driving simulation device during simulation playback testing as an example, a Monte Carlo tree search can be performed 5 seconds before the collision to attempt interaction targets cutting into the front of the autonomous driving simulation device, waiting for the autonomous driving simulation device to pass before cutting into the rear of the autonomous driving simulation device, and remaining to the side of the autonomous driving simulation device. Through the Monte Carlo tree search, the interaction strategy sequence of the key obstacle accelerating in its own lane and then changing lanes to the lane where the autonomous driving simulation device is located to cut into the front of the autonomous driving simulation device can be obtained, and this sequence can be used as the target interaction strategy sequence. Of course, other corresponding interaction strategy sequences can also achieve the interaction target and can also be used as target interaction strategy sequences.

[0066] Since this example mainly addresses the behavior planning of key interaction segments for critical obstacles in collision scenarios, it is also possible to set an interaction strategy sequence for the critical obstacle to maintain its lane after the target interaction strategy sequence is completed, so as to achieve reasonable subsequent behavior of the critical obstacle.

[0067] As can be seen, based on the interaction behavior between key obstacles and autonomous driving simulation equipment in collision scenarios, such as following and cutting in, a reasonable sequence of one or more target interaction strategies between key obstacles and autonomous driving simulation equipment is obtained through Monte Carlo tree search, and interaction simulation is performed. By exploring the possible interaction targets and corresponding interaction strategy sequences between key obstacles and autonomous driving simulation equipment based on collision scenarios, a method for simulating key obstacle interactions is provided for subsequent targeted testing, serving as a basis for the takeover method of key obstacles in new simulation replay test tasks.

[0068] Step S306: Conduct interactive simulation of key obstacles and autonomous driving simulation equipment.

[0069] In this example, the Lattice trajectory planning algorithm is used to realize the motion planning of key obstacles and autonomous driving simulation equipment corresponding to the target interaction strategy sequence.

[0070] For example, based on a determined target interaction strategy sequence, the corresponding planning state can be sampled. For instance, when making a lane change decision, different lane offsets and lane change times can be sampled on the target lane. Under the constraints of acceleration and speed range, the fastest lane change trajectory can be found. The trajectory from the current state to the target state to continuous motion can be solved using a fifth-order polynomial, thereby obtaining the position, speed, and acceleration states of key obstacles and autonomous driving simulation equipment.

[0071] For example, the trajectory output by Lattice Planner can be used as the initial value, and then Lattice Planner can be used to perform dense sampling near the initial value to obtain a collision-free trajectory that satisfies kinematic constraints to drive the interaction of key obstacles in the simulation playback test.

[0072] Furthermore, this example provides extensions to various trajectory planning methods. For instance, it supports a combination of learning-based planners (deep learning models) and Lattice Planners for trajectory planning, controlling key obstacles in simulation playback tests and enriching interactive testing methods. Through trajectory planning algorithms based on the combination of Lattice Planners and deep learning models, various types of interactive simulations between key obstacles and autonomous driving simulation equipment can be achieved.

[0073] As can be seen, by using the Lattice trajectory planning algorithm in this step to plan the motion trajectory of key obstacles, it is possible to conduct effective interactive testing while avoiding collisions with autonomous driving simulation equipment.

[0074] Step S308: Output simulation test results.

[0075] In this example, the target interaction strategy sequence can include multiple sequences, and there will also be multiple corresponding interaction simulations. These multiple interaction simulations can be executed in parallel to achieve multi-task parallel testing.

[0076] After all the interactive simulations are completed, corresponding simulation playback test results will be generated. Combining the autonomous driving simulation equipment with multiple evaluation dimensions such as collision, traffic rule evaluation, and driving comfort, that is, the degree of impact of different interactive simulations on the autonomous driving simulation equipment, test scores for different interactive simulations will be given.

[0077] Then, the interactive simulation results that have the greatest impact on the autonomous driving simulation equipment can be selected and presented to the user, while the remaining interactive simulation results can be folded below for easy viewing by the user.

[0078] This step enables the presentation of test results corresponding to interactive simulations. These results are based on various interactive simulations, such as accelerating and changing lanes to enter the autonomous driving simulation device when encountering a key obstacle, decelerating and changing lanes to yield to the autonomous driving simulation device, and maintaining the current lane. This not only ensures the effectiveness of the simulation test but also enriches the diversity of the test.

[0079] This example demonstrates a process that includes automatic identification of key obstacles, determination of interaction strategies for key obstacles, deduction of multiple interactions of key obstacles, and retesting by comparing test results. This process can effectively reduce collisions caused by unreasonable road test data and improve the effectiveness of autonomous driving algorithm testing.

[0080] Based on the above process, a test architecture was implemented consisting of road test data playback simulation testing – key obstacle deduction (interaction strategy determination and corresponding interaction deduction execution) – simulation retesting, improving the diversity and effectiveness of road test data testing. Specifically, by employing Monte Carlo tree search and Lattice trajectory planning, different interaction strategies and interaction deductions for key obstacles were implemented, obtaining optimal target interaction strategy sequences for simulation testing, thus improving the diversity of interaction testing. Furthermore, by using Monte Carlo tree search and Lattice trajectory planning, on the one hand, due to the deduction mechanism, trajectory planning can obtain the correct motion trajectory a priori, which helps to obtain better planning results; on the other hand, it is more controllable than the results of model training, and has the conditions for large-scale deployment.

[0081] After obtaining the results of the interactive simulation, this example will also combine the degree of impact of the results on the autonomous driving simulation equipment to give the interactive simulation results that are challenging for the autonomous driving simulation equipment.

[0082] In terms of the overall solution, compared to traditional methods such as Intersim, which determines whether to take over obstacles based on predicted interaction trajectories, this example makes decisions based on deterministic interaction results (such as collisions). The decision is not overly conservative or aggressive due to the predictive model, thus preserving boundary test conditions and avoiding model bias. Compared to Logsim, this solution addresses the lack of intelligence in obstacle interaction, significantly improving the effectiveness of road test data utilization and enhancing the testing value of autonomous driving road test data feedback. This plays a crucial role in accelerating autonomous driving testing and improving algorithm safety.

[0083] It should be noted that although this embodiment uses a collision scenario as an example, those skilled in the art should understand that other scenarios that affect the driving of autonomous driving simulation equipment (such as scenarios that cause the autonomous driving simulation equipment to brake suddenly or change lanes suddenly) can all be simulated and tested with reference to this embodiment, and all of them are within the protection scope of this application.

[0084] Reference Figure 3 This document illustrates a schematic diagram of an electronic device according to an embodiment of this application. The specific embodiments of this application do not limit the specific implementation of the electronic device.

[0085] like Figure 3 As shown, the electronic device may include: a processor 402, a communications interface 404, a memory 406, and a communications bus 408.

[0086] in:

[0087] The processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408.

[0088] Communication interface 404 is used to communicate with other electronic devices or servers.

[0089] The processor 402 is used to execute program 410, specifically to perform the relevant steps in the above simulation test method embodiment.

[0090] Specifically, program 410 may include program code that includes computer operation instructions.

[0091] Processor 402 may be a CPU, an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The smart device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0092] Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0093] Program 410 may include multiple computer instructions. Specifically, program 410 can use multiple computer instructions to cause processor 402 to execute the operation corresponding to the simulation test method described in the foregoing method embodiments.

[0094] The specific implementation of each step in procedure 410 can be found in the corresponding descriptions of the steps and units in the above method embodiments, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0095] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in any of the foregoing method embodiments. The computer storage medium includes, but is not limited to, compact disc read-only memory (CD-ROM), random access memory (RAM), floppy disk, hard disk, or magneto-optical disk.

[0096] This application also provides a computer program product, including computer instructions that instruct a computing device to perform the operation corresponding to the simulation test method in the above method embodiments.

[0097] Furthermore, it should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data used for training the model, user data in the collected road test data, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0098] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0099] The methods described in the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA)). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., Random Access Memory (RAM), Read-Only Memory (ROM), Flash Memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0100] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0101] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A simulation testing method, comprising: Based on the simulation playback test of road test data, key obstacles were identified. These key obstacles are traffic objects that affect the driving of the autonomous driving simulation equipment and are responsible for the driving. Based on the scene information corresponding to the driving impact, multiple interaction strategies are determined for the interaction between the key obstacles and the autonomous driving simulation device. Using multiple interaction strategies, various interaction simulations are performed in parallel between the key obstacle and the autonomous driving simulation device, corresponding to the multiple interaction strategies. Based on the interactive simulation results, the simulation test results are obtained.

2. The method according to claim 1, wherein, The simulation playback test based on road test data identified key obstacles, including: Simulation playback tests are conducted based on road test data to identify key data frames that affect the driving of the autonomous driving simulation device. The key obstacle is identified from the key data frame.

3. The method according to claim 2, wherein, The method further includes: Based on the key data frames, the motion trajectories of the key obstacles and the autonomous driving simulation device are obtained; Based on the obtained motion trajectory, determine the scene information corresponding to the driving impact.

4. The method according to any one of claims 1-3, wherein, Based on the scene information corresponding to the driving impact, the system determines multiple interaction strategies for the interaction between the key obstacles and the autonomous driving simulation device, including: Based on the scene information corresponding to the driving impact, multiple interaction strategies are determined for the interaction between the key obstacles and the autonomous driving simulation device through Monte Carlo tree search.

5. The method according to any one of claims 1-3, wherein, Based on the scene information corresponding to the driving impact, the system determines multiple interaction strategies for the interaction between the key obstacles and the autonomous driving simulation device, including: Based on the scene information corresponding to the driving impact, the interaction target between the key obstacle and the autonomous driving simulation device in the scene indicated by the scene information is obtained; Based on the interaction objective, generate multiple candidate interaction strategy sequences; The multiple candidate interaction strategy sequences are evaluated according to a preset cost evaluation function; Based on the evaluation results, determine the target interaction strategy sequence.

6. The method according to claim 5, wherein, The cost evaluation function includes at least one of the following: The first cost evaluation function is used to evaluate the efficiency of achieving the interaction goal based on the candidate interaction strategy sequence; A second cost evaluation function is used to evaluate the smoothness of the motion trajectory when the key obstacle moves based on a sequence of candidate interaction strategies; A third cost evaluation function is used to assess the difficulty of interaction between the key obstacle and the autonomous driving simulation device when moving based on a sequence of candidate interaction strategies.

7. The method according to claim 6, wherein, The evaluation of the multiple candidate interaction strategy sequences according to the preset cost evaluation function includes at least one of the following: Based on the time required to achieve the interaction goal, the efficiency of achieving the interaction goal using the first cost evaluation function is evaluated for multiple candidate interaction strategy sequences. Based on the acceleration and the mean acceleration of the key obstacle when it moves based on multiple candidate interaction strategy sequences, the smoothness of the trajectory of the key obstacle is evaluated using the second cost evaluation function. Based on the shortest interaction distance between the key obstacle and the autonomous driving simulation device when the key obstacle moves based on multiple candidate interaction strategy sequences, the interaction difficulty between the key obstacle and the autonomous driving simulation device is evaluated using the third cost evaluation function.

8. The method according to claim 5, wherein, After determining the target interaction strategy sequence based on the evaluation results, the method further includes: After determining the target interaction strategy sequence, a strategy to maintain driving in the current lane is added for the key obstacle.

9. The method according to any one of claims 1-3, wherein, The use of multiple interaction strategies to simulate the interaction between the key obstacles and the autonomous driving simulation device includes: Perform trajectory planning for each of the various interaction strategies described; Based on the various interaction strategies and their corresponding trajectory planning, the interaction between the key obstacles and the autonomous driving simulation device is simulated.

10. The method according to any one of claims 1-3, wherein, The process of obtaining simulation test results based on interactive deduction results includes: Based on the interactive simulation results, determine whether there is an interaction in the corresponding interactive simulation where the key obstacle interacts with the autonomous driving simulation device and causes a driving impact, and the party responsible for the impact is the autonomous driving simulation device. If it exists, the interaction deduction information corresponding to the interaction will be determined as auxiliary information for the problem to be processed by the autonomous driving simulation device.

11. An electronic device, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform an operation corresponding to the method as described in any one of claims 1-10.

12. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1-10.

13. A computer program product comprising computer instructions that instruct a computing device to perform an operation corresponding to any one of the methods described in claims 1-10.

Images