Automatic driving simulation test method, device and equipment and storage medium

Abstract

The present disclosure provides an automatic driving simulation test method, device, equipment and storage medium, relates to the technical field of automatic driving, in particular to the technical field of automatic driving simulation control. The specific implementation scheme is: acquiring automatic driving data collected by an automatic driving vehicle in a road test process; performing correction processing on the data transmission sequence and retransmission processing on the lost data of the automatic driving data to obtain processed data; using the processed data to reproduce a problem scenario in the road test process to determine a problem of an automatic driving algorithm module in the automatic driving vehicle in the problem scenario; optimizing the automatic driving algorithm module according to the problem of the automatic driving algorithm module in the road test to obtain an optimized algorithm module; verifying whether the optimized algorithm module can repair the problem in the problem scenario to obtain a verification result, which is used to represent whether the optimized algorithm module needs to be further optimized.

Description

Technical Field

[0001] This disclosure relates to the field of autonomous driving technology, specifically to the field of autonomous driving simulation and control technology. More particularly, it relates to an autonomous driving simulation testing method, apparatus, device, and storage medium. Background Technology

[0002] According to research, autonomous vehicles need at least 17 billion kilometers of driving data to achieve the same level of performance as human drivers.

[0003] In current technologies, using road testing to optimize autonomous driving algorithms in self-driving vehicles is prohibitively expensive for any company, both in terms of time and money. Furthermore, replicating extreme traffic conditions and dangerous scenarios through road testing raises concerns about safety.

[0004] Therefore, using autonomous driving simulation devices to simulate problems encountered during road testing has become a key solution to the lack of data and difficulty in reproducing extreme scenarios when using road testing methods for autonomous driving testing. Summary of the Invention

[0005] This disclosure provides an autonomous driving simulation testing method, apparatus, equipment, and storage medium.

[0006] According to a first aspect of this disclosure, an autonomous driving simulation testing method is provided, applied to an autonomous driving simulation device to perform an open-loop simulation testing task, the method comprising:

[0007] Acquire autonomous driving data collected by autonomous vehicles during road testing;

[0008] The autonomous driving data is processed by correcting the data transmission and reception order and retransmitting lost data to obtain processed data;

[0009] The processed data is used to reproduce the problem scenarios during the road test to determine the problems that occur in the autonomous driving algorithm module of the autonomous vehicle in the problem scenarios.

[0010] Based on the problems encountered by the autonomous driving algorithm module in the road test, the autonomous driving algorithm module is optimized to obtain an optimized algorithm module;

[0011] Verify whether the optimized algorithm module can fix the problem in the problem scenario, and obtain the verification result. The verification result is used to indicate whether it is necessary to continue to optimize the optimized algorithm module.

[0012] Furthermore, acquiring autonomous driving data collected by autonomous vehicles during road testing includes:

[0013] The autonomous driving vehicle acquires sensor data collected during the road test, as well as computational data obtained by the autonomous driving algorithm module based on the sensor data. The autonomous driving algorithm module includes a localization module, a perception module, a planning module, and a driving control module.

[0014] Furthermore, the autonomous driving data undergoes data transmission and reception order correction processing and lost data retransmission processing to obtain processed data, including:

[0015] Acquire delayed and lost data from the autonomous driving data;

[0016] The delayed data transmission and reception order is corrected, and the lost data is retransmitted to obtain the processed data.

[0017] Furthermore, the step of acquiring delayed and lost data in the autonomous driving data includes: acquiring the autonomous driving data to be sent to the autonomous driving test module, and the data transmission list at the current transmission time;

[0018] Based on the data sending list, listen for asynchronous events, wherein the autonomous driving test module is used to send an asynchronous event after receiving the autonomous driving data;

[0019] The asynchronous events are parsed and verified, and the sending and receiving status of the autonomous driving data is updated in the hash table based on the parsing and verification results;

[0020] The transmission and reception status is detected at a predetermined frequency to obtain delayed transmission and reception data and lost data in the autonomous driving data.

[0021] Furthermore, the autonomous driving simulation device includes: an autonomous driving test module, a data transmission module, a timer module, a monitoring module, and a simulation control module; the acquisition of autonomous driving data to be sent to the autonomous driving test module, and the data transmission list at the current transmission time, includes:

[0022] The data transmission module is used to parse the autonomous driving data to obtain the parsed autonomous driving data and the data transmission list at the current transmission time;

[0023] The data sending module is further configured to inform the monitoring module of the data sending list at the current sending time, send the parsed autonomous driving data to the autonomous driving test module, and register a callback function when sending the parsed autonomous driving data. The autonomous driving test module is configured to send an asynchronous event based on the callback function after receiving the parsed autonomous driving data and before processing the parsed autonomous driving data, so as to inform the monitoring module that the parsed autonomous driving data has been received normally.

[0024] Furthermore, the step of listening to asynchronous events based on the data sending list includes: using the monitoring module to listen to the asynchronous events based on the data sending list;

[0025] The step of parsing and verifying the asynchronous event to update the transmission and reception status of the autonomous driving data in the hash table based on the parsing and verification results includes: parsing and verifying the asynchronous event after it is detected, and updating the transmission and reception status in the hash table of the monitoring module according to the parsing and verification results.

[0026] Further, the step of detecting the transmission and reception status based on a predetermined frequency to obtain delayed transmission and reception data and lost data in the autonomous driving data includes:

[0027] The simulation control module, under the control of the timer module, detects the transmission and reception status recorded in the hash table of the monitoring module at a predetermined frequency to obtain the delayed transmission and reception data and lost data in the autonomous driving data.

[0028] Furthermore, the step of detecting the transmission and reception status recorded in the hash table of the monitoring module based on a predetermined frequency includes:

[0029] If the hash table of the monitoring module is checked and it is determined that all data at the current sending time has been successfully sent and received, then the next sending time will begin, until all autonomous driving data has been processed.

[0030] If the hash table of the monitoring module is checked and it is determined that all data at the current sending time has not been sent or has not been successfully received, then the delayed data and lost data in the autonomous driving data are identified, so as to trigger the data sending module to perform data transmission order correction processing on the delayed data and retransmission processing on the lost data to obtain the processed data.

[0031] Furthermore, in the process of reproducing the problem scenarios during the road test using the processed data, the method further includes:

[0032] The autonomous driving simulation device acquires different autonomous driving test modules and the dependencies between them, wherein the autonomous driving test module includes at least one of the following: a localization module, a perception module, a planning module, and a driving control module.

[0033] Based on the dependencies between different autonomous driving test modules, a sequence dependency relationship is constructed between the autonomous driving test data input to and the autonomous driving test data output by different autonomous driving test modules;

[0034] Based on the sequence dependency relationship, the out-of-order data between the different autonomous driving test modules is restored, and the data transmission rate between the different autonomous driving test modules is controlled.

[0035] Furthermore, in the process of reproducing the problem scenarios during the road test using the processed data, the method further includes:

[0036] Obtain the autonomous driving test data used by the autonomous driving test module during simulation testing, and the data frame sequence number of the autonomous driving test data used during simulation testing;

[0037] Based on the data frame sequence number of the autonomous driving test data used in the simulation test, the vehicle trajectory data in the problem scenario is filtered out;

[0038] Determine the sensor data used by the autonomous driving algorithm module in the autonomous vehicle when calculating the vehicle trajectory data;

[0039] Based on the sensor data, the autonomous driving data used in the autonomous driving test module is adjusted to be consistent with the autonomous driving data used by the autonomous driving algorithm module during road testing.

[0040] Furthermore, the sensor data includes: traffic light data, obstacle data, and vehicle position data, wherein the obstacle data is key data that triggers the calculation of the vehicle trajectory data; based on the sensor data, the autonomous driving data used in the autonomous driving test module is adjusted to be consistent with the autonomous driving data used by the autonomous driving algorithm module during road testing, including:

[0041] Based on the time of the obstacle data, the time of the traffic light data and the vehicle position data are adjusted so that the autonomous driving data used in the autonomous driving test module is consistent with the autonomous driving data used by the autonomous driving algorithm module during road testing.

[0042] Furthermore, the method also includes:

[0043] Based on the different autonomous driving test modules in the autonomous driving simulation device, a directed graph is abstracted.

[0044] Based on the autonomous driving test data input and output from different autonomous driving test modules, the directed graph is expanded to obtain a simulation cycle of the open-loop simulation test task.

[0045] After one simulation cycle ends, the next simulation cycle begins to ensure the integrity of each simulation cycle and the simulation results.

[0046] Furthermore, the optimized algorithm module is verified to fix the problems in the problem scenario, and the verification results are obtained, including:

[0047] Obtain the target obstacle in the problem scenario, wherein the target obstacle is an obstacle that affects the normal execution and verification of the open-loop simulation test task;

[0048] After cleaning and filtering the target obstacles, the optimized algorithm module is verified to determine whether it can fix the problems in the problem scenario, and the verification result is obtained.

[0049] According to a second aspect of this disclosure, an autonomous driving simulation testing apparatus is provided, the autonomous driving simulation testing apparatus comprising:

[0050] The acquisition unit is used to acquire autonomous driving data collected by autonomous vehicles during road testing.

[0051] The processing unit is used to correct the data transmission and reception order of the autonomous driving data and retransmit lost data to obtain processed data.

[0052] The reproduction unit is used to reproduce the problem scenarios during the road test using the processed data, so as to determine the problems that occur in the autonomous driving algorithm module in the autonomous vehicle in the problem scenarios.

[0053] An optimization unit is used to optimize the autonomous driving algorithm module based on the problems encountered by the autonomous driving algorithm module in the road test, so as to obtain an optimized algorithm module.

[0054] The verification unit is used to verify whether the optimized algorithm module can fix the problem in the problem scenario and obtain the verification result. The verification result is used to indicate whether the optimized algorithm module needs to be further optimized.

[0055] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0056] At least one processor; and

[0057] A memory communicatively connected to the at least one processor; wherein,

[0058] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described.

[0059] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method according to any one of the following methods.

[0060] According to a fifth aspect of this disclosure, a computer program product is provided, the computer program product comprising: a computer program stored in a readable storage medium, wherein at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the method described in the first aspect.

[0061] According to the technology disclosed herein, autonomous driving data collected by an autonomous vehicle during road testing is obtained; the autonomous driving data undergoes data transmission and reception sequence correction processing and lost data retransmission processing to obtain processed data; the processed data is used to reproduce the problem scenarios during the road testing process to determine the problems encountered by the autonomous driving algorithm module in the autonomous vehicle in the problem scenarios; based on the problems encountered by the autonomous driving algorithm module in the road testing, the autonomous driving algorithm module is optimized to obtain an optimized algorithm module; the optimized algorithm module is verified to determine whether it can fix the problems encountered in the problem scenarios, and a verification result is obtained, wherein the verification result is used to characterize whether further optimization processing of the optimized algorithm module is needed.

[0062] After acquiring autonomous driving data collected by the autonomous vehicle during road testing, the data transmission and reception order is corrected, and lost data is retransmitted to obtain processed data. This avoids message delays and loss during simulation testing. Based on this processed data, the problem scenarios encountered by the autonomous vehicle during road testing are reproduced in simulation tests, thereby identifying the problems of the autonomous driving algorithm module in these scenarios and achieving the purpose of simulation testing. Then, based on the problems encountered by the autonomous driving algorithm module in the aforementioned road tests, the module is optimized to obtain an optimized algorithm module. Finally, the optimized algorithm module is verified to solve the problems in the current problem scenario until the verification results indicate that the optimized algorithm module can solve the problems in the current problem scenario, at which point further optimization of the optimized algorithm module is unnecessary.

[0063] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0064] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0065] Figure 1 This is a flowchart of an autonomous driving simulation testing method provided according to an embodiment of the present disclosure;

[0066] Figure 2 This is a schematic diagram of an optional autonomous driving simulation device provided according to an embodiment of the present disclosure;

[0067] Figure 3a This is a schematic diagram of the storage time series of autonomous driving data provided according to an embodiment of the present disclosure;

[0068] Figure 3b This is a schematic diagram of autonomous driving data used in the calculation of another planning module according to an embodiment of this disclosure;

[0069] Figure 3c This is a schematic diagram of the storage time series of repaired autonomous driving data provided according to an embodiment of the present disclosure;

[0070] Figure 4 This is a flowchart of an autonomous driving simulation testing method provided according to an embodiment of the present disclosure;

[0071] Figure 5 This is a schematic diagram illustrating the dependency relationship between a planning module and a sensing module according to an embodiment of this disclosure;

[0072] Figure 6a This is a schematic diagram illustrating a collision between an autonomous vehicle and an obstacle vehicle A, according to an embodiment of this disclosure.

[0073] Figure 6b This is a schematic diagram illustrating an autonomous vehicle colliding with an obstacle vehicle B after braking, according to an embodiment of this disclosure.

[0074] Figure 6c This is a schematic diagram of a vehicle B after removing an unreasonable obstacle, according to an embodiment of this disclosure;

[0075] Figure 7 This is a schematic diagram of the frame of an autonomous driving simulation testing device provided according to an embodiment of the present disclosure;

[0076] Figure 8 This is a block diagram of an electronic device used to implement an autonomous driving simulation testing method according to an embodiment of the present disclosure. Detailed Implementation

[0077] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0078] A key performance indicator for autonomous driving simulation is reproducibility. Due to the unique nature of road testing, it is nearly impossible to reproduce an exact identical scenario in real-world road testing. Furthermore, for post-simulation verification, a common method is to recalculate using the repaired autonomous driving algorithm module based on the raw sensor data collected on the road, to determine if the problem can be resolved.

[0079] However, at the scene reproduction level, existing methods lack consideration for message delays and loss during simulation. The former will disrupt the actual order of message delivery, while the latter will directly lead to message loss, affecting the reproduction effect. At the verification level, existing methods verify whether the optimized autonomous driving algorithm module can solve the current problem by truncating the length of the scene. However, since the behavior of obstacle vehicles in autonomous driving data is immutable, unreasonable new collisions and other problems have occurred. Therefore, how to truncate a scene of appropriate length requires a lot of experimentation.

[0080] Currently, there are two main types of common autonomous driving simulation tasks: open-loop simulation test tasks and closed-loop simulation test tasks. Among them, closed-loop simulation test tasks are commonly simulation tasks based on manually constructed scenarios, while open-loop simulation test tasks are mainly simulation tasks based on scenarios constructed from road-collected data. This disclosure proposes an autonomous driving simulation test method and apparatus embodiment for the aforementioned open-loop simulation test tasks.

[0081] To address the aforementioned issues, this disclosure provides an autonomous driving simulation testing method, apparatus, device, and storage medium, applied in the field of autonomous driving simulation control technology, to reproduce problem scenarios in open-loop simulation testing tasks and to verify whether the optimized autonomous driving algorithm module can fix the problems that occur in the aforementioned problem scenarios.

[0082] Example 1

[0083] Figure 1 This is a flowchart of an autonomous driving simulation testing method provided according to embodiments of the present disclosure, such as... Figure 1 As shown, this disclosure provides an autonomous driving simulation testing method, applied to an autonomous driving simulation device, to perform open-loop simulation testing tasks. The method includes the following steps:

[0084] S101, acquire autonomous driving data collected by autonomous vehicles during road testing.

[0085] S102, the above-mentioned autonomous driving data is processed to correct the data transmission and reception order and retransmit lost data to obtain the processed data.

[0086] S103, using the processed data described above, reproduce the problem scenarios from the road test process to determine the problems that occur in the autonomous driving algorithm module of the autonomous vehicle in the aforementioned problem scenarios.

[0087] S104. Based on the problems encountered by the above-mentioned autonomous driving algorithm module in the above-mentioned road test, the above-mentioned autonomous driving algorithm module is optimized to obtain an optimized algorithm module.

[0088] S105, verify whether the optimized algorithm module can fix the problems in the above problem scenario, and obtain the verification result. The verification result is used to characterize whether the optimized algorithm module needs to be further optimized.

[0089] In one example, the above-mentioned open-loop simulation test task is a virtual test relative to road testing. Specifically, it is a simulation task based on driving scenarios constructed from data collected on simulated test roads. The collected data refers to the autonomous driving data collected by the autonomous vehicle during road testing. This autonomous driving data includes: sensor data stored by the autonomous vehicle during road testing, and calculation data obtained by the autonomous driving algorithm module in the autonomous vehicle based on the aforementioned sensor data.

[0090] In one example, the aforementioned autonomous vehicle is an unmanned intelligent car that relies on autonomous driving technology, artificial intelligence technology, visual computing technology, etc. The autonomous vehicle can be any type and any size, and this disclosure does not specifically limit it.

[0091] Optionally, the aforementioned autonomous driving algorithm modules include: a localization module, a perception module, a planning module, and a driving control module.

[0092] In another example, the aforementioned road testing process verifies the rationality, safety, and stability of autonomous driving algorithm modules in terms of functionality, performance, safety, and stability before autonomous vehicles are truly put into commercial production. It verifies the maturity of the data generation, transmission, and interaction technologies between autonomous vehicles and roads, providing autonomous driving data for subsequent virtual simulation tests. At the same time, it can also establish dynamic adaptive testing of vehicles through test scenarios. This is a crucial step for whether autonomous vehicles can truly be put into commercial production and will undoubtedly play a positive role in promoting it.

[0093] In one example, the autonomous driving simulation device is an autonomous driving simulation software. In the simulation test, it obtains autonomous driving data from the road data collection data packet during the road test process and reproduces the problem scenarios in the road test process. Due to the differences in the software and hardware environment, message delays and loss may occur, and the actual reproduction effect is not good.

[0094] In this embodiment of the disclosure, in order to reproduce as accurately as possible the problem scenarios encountered by autonomous vehicles during road testing in simulation testing, and thus identify problems in the autonomous driving algorithm module, the simulation testing aims to achieve its purpose. After acquiring the autonomous driving data collected by the autonomous vehicle during road testing, this embodiment of the disclosure corrects the data transmission and reception order and retransmits lost data to obtain processed data, thereby avoiding message delays and loss during simulation testing.

[0095] For example, in one instance, delayed and lost data from the aforementioned autonomous driving data are acquired; the delayed data is processed to correct the data transmission order, and the lost data is retransmitted to obtain the processed data. Furthermore, in simulation testing, the problem scenarios encountered by the autonomous vehicle during road testing can be reproduced based on the processed data, thereby identifying problems in the autonomous driving algorithm module and achieving the purpose of simulation testing.

[0096] Optionally, in one example, the above-mentioned problem scenario is a scenario in the autonomous driving scenario during road testing where driving problems occur. For example, the problem scenario may include one or more problems such as vehicle collision, running a red light, or not driving according to the planned vehicle trajectory.

[0097] In one example, based on the problems encountered by the autonomous driving algorithm module in the aforementioned road test, such as vehicle collision problems, the algorithm modules related to vehicle collision algorithms (such as the localization module, perception module, planning module, and driving control module, all of which may be related to vehicle collision problems) within the autonomous driving algorithm module are optimized. For example, the optimization process can involve performing one or more optimizations on the autonomous driving algorithm module, such as environmental perception, path planning, behavior decision-making, execution control, and sensitivity adjustment, according to the functional requirements of the autonomous driving algorithm module, to obtain an optimized algorithm module. In this embodiment of the disclosure, the specific optimization processing method used for one or more autonomous driving algorithm modules may also be, but is not limited to, any one or more optimization methods in the prior art, and this embodiment of the disclosure does not specifically limit this.

[0098] Therefore, this embodiment of the present disclosure, after acquiring autonomous driving data collected by the autonomous vehicle during road testing, corrects the data transmission and reception order and retransmits lost data to obtain processed data. This avoids message delays and loss during simulation testing. Based on the processed data, the problem scenarios encountered by the autonomous vehicle during road testing are reproduced in simulation testing, thereby identifying the problems of the autonomous driving algorithm module in the autonomous vehicle within these scenarios and achieving the purpose of simulation testing. Subsequently, based on the problems encountered by the autonomous driving algorithm module in the aforementioned road tests, the autonomous driving algorithm module is optimized to obtain an optimized algorithm module. Finally, the optimized algorithm module is verified to solve the problems in the current problem scenario until the verification results indicate that the optimized algorithm module can solve the problems in the current problem scenario, at which point further optimization of the optimized algorithm module is unnecessary.

[0099] Example 2

[0100] Based on the above embodiment 1, in this embodiment, considering that data may be delayed and lost during the simulation test, if the data is replayed and the process is advanced without monitoring, it will not only disrupt the actual data transmission and reception order, but also directly lead to data loss and affect the problem reproduction effect of the simulation test. Therefore, this embodiment specifically adopts the following method to obtain the delayed transmission and reception data and lost data in the above autonomous driving data.

[0101] In one optional implementation, the autonomous driving data to be sent to the autonomous driving test module and the data transmission list at the current transmission time are obtained; asynchronous events are monitored based on the data transmission list, wherein the autonomous driving test module is used to send an asynchronous event after receiving the autonomous driving data; the asynchronous events are parsed and verified, and the transmission and reception status of the autonomous driving data is updated in a hash table based on the parsing and verification results; the transmission and reception status is detected at a predetermined frequency to obtain delayed transmission and reception data and lost data in the autonomous driving data.

[0102] In one example, such as Figure 2 As shown, the above-mentioned autonomous driving simulation device includes: an autonomous driving test module, a data transmission module, a timer module, a monitoring module, and a simulation control module.

[0103] In another optional implementation, obtaining delayed transmission and reception data and lost data from the aforementioned autonomous driving data specifically includes the following method steps:

[0104] S201, the above-mentioned data transmission module is used to parse the above-mentioned autonomous driving data to obtain the data transmission list at the current transmission time.

[0105] S202, after the data sending module informs the monitoring module of the data sending list at the current sending time, it sends the parsed autonomous driving data to the autonomous driving test module and registers a callback function when sending the parsed autonomous driving data.

[0106] In step S202 above, the autonomous driving test module is used to send an asynchronous event based on the callback function after receiving the parsed autonomous driving data and before processing the parsed autonomous driving data, so as to inform the monitoring module that the parsed autonomous driving data has been received normally.

[0107] S203, after the above-mentioned monitoring module detects the above-mentioned asynchronous event, it updates the sending and receiving status of the parsed autonomous driving data in the hash table according to the parsing and verification results of the above-mentioned asynchronous event.

[0108] S204, the simulation control module, under the control of the timer module, detects the transmission and reception status recorded in the hash table of the monitoring module at a predetermined frequency to obtain the delayed transmission and reception data and lost data in the autonomous driving data.

[0109] In this embodiment of the disclosure, considering that data may be delayed or lost during the simulation test, if the data is replayed and the process is advanced without monitoring, it will not only disrupt the actual data transmission and reception order, but also directly lead to data loss and affect the problem reproduction effect of the simulation test. Therefore, in this embodiment of the disclosure, a data transmission module, a timer module, a monitoring module and a simulation control module are used in the autonomous driving simulation device to construct an asynchronous event data transmission and reception monitoring mechanism to ensure that autonomous driving data arrives at the autonomous driving test module in order and to enable the retransmission of lost data.

[0110] In one example, such as Figure 2 The data transmission module shown can retrieve autonomous driving data during road testing from the road data collection packet based on the timestamp of the autonomous driving data. However, instead of directly sending the autonomous driving data to the autonomous driving test module without monitoring, the data transmission module first parses the autonomous driving data to obtain a data transmission list for the current transmission time. This data transmission module then informs the monitoring module of this current transmission time's data transmission list before sending the parsed autonomous driving data to the autonomous driving test module.

[0111] In one example, the data sending module also registers a callback function when sending the parsed autonomous driving data. The autonomous driving test module, upon receiving the parsed autonomous driving data but before processing it, sends an asynchronous event based on the callback function to inform the monitoring module that the parsed autonomous driving data has been received normally. Since this asynchronous event is detected by the monitoring module, it further parses and verifies the event upon receipt, updating the transmission and reception status of the parsed autonomous driving data in the hash table.

[0112] In another example, in this embodiment, the simulation control module is controlled by a timer module. Under the control of the timer module, the simulation control module will detect the transmission and reception status recorded in the hash table of the monitoring module at a predetermined frequency to determine the delayed transmission and reception data and lost data in the above-mentioned autonomous driving data.

[0113] In one example, the above-mentioned detection of the transmission and reception status recorded in the hash table of the monitoring module based on a predetermined frequency includes:

[0114] S301, if the hash table of the monitoring module is checked and it is determined that all data at the current sending time has been successfully sent and received, then proceed to the next sending time until all autonomous driving data has been processed.

[0115] S302, if the hash table of the monitoring module is checked and it is determined that all data at the current sending time has not been sent or has not been successfully received, then the delayed data and lost data in the autonomous driving data are determined, so as to trigger the data sending module to perform data transmission order correction processing on the delayed data and retransmission processing on the lost data, so as to obtain the processed data.

[0116] In one example, by detecting the transmission and reception status recorded in the hash table of the monitoring module based on a predetermined frequency, it is determined that all data at the current transmission time has been successfully transmitted and received. Then, the next transmission time begins, the system goes into sleep mode and waits for the next wake-up. That is, the data transmission module is used to start a new round of autonomous driving data acquisition, parsing and transmission process until all autonomous driving data is processed and the entire autonomous driving scenario test is completed.

[0117] In another example, if the monitoring module detects the transmission and reception status recorded in its hash table and determines that all data at the current transmission time has not been successfully transmitted or received—for example, if the current transmission time is detected more than a certain number of times, indicating that autonomous driving data or asynchronous events have been lost, or that there is delayed transmission and reception of autonomous driving data—then the data transmission module (only the data transmission module reads and parses autonomous driving data from the road data packets; other modules receive the parsed autonomous driving data sent by the data transmission module for calculation and testing by the autonomous driving algorithm module) will be triggered to retransmit the outdated data, i.e., the lost data and the delayed transmission and reception data. By monitoring the retransmission and retransmission of delayed transmission and reception data, the transmission and reception order of the delayed transmission and reception data can be corrected until all parsed autonomous driving data is successfully received.

[0118] Example 3

[0119] Based on Embodiment 1 above, in this embodiment, after controlling the message sending and receiving order, the autonomous driving simulation device of this disclosure can guarantee the data sending and receiving order and its arrival at the autonomous driving test module in sequence. However, there are still other problems that need to be solved. First, the autonomous driving data stored in the road data collection package may also be inconsistent with the autonomous driving data stored in the road data collection package due to differences in storage time. This can lead to inconsistencies between the autonomous driving data actually used by the autonomous driving test module during calculation (e.g., when the planning module calculates the autonomous driving vehicle trajectory calculation result after receiving vehicle position data, traffic light data, and obstacle data) and the autonomous driving data stored in the road data collection package. Second, the planning module needs to wait until the obstacle data, traffic light data, and vehicle position data are received before starting vehicle trajectory planning. Vehicle trajectory planning is often a computationally intensive process, which is affected by the simulation machine resources. If the sending rate of autonomous driving data in the simulation test is not controlled, for example, if data is input and output in sequence without control in the simulation, it is very likely that the sensor data required for the next planning will be received before the current planning result is calculated, resulting in the sensor data being discarded after reception.

[0120] To address the aforementioned issues, this disclosure proposes the concept of frame sequence dependency between data and, based on the analytical framework of directed graph topology, analyzes and obtains the frame sequence dependency of key data in the autonomous driving algorithm module.

[0121] Taking a common autonomous driving scenario as an example, to test the planning module's calculation process in a simulation system, a common approach is to construct a simulation scenario based on the autonomous driving data collected during road testing, according to its storage time series, and then replay the autonomous driving data sequentially based on the storage time series. The frequencies of vehicle position data, traffic light data, and obstacle data are all different. Figure 3a The black vertical dashed lines shown represent the three calculation trigger points of the planning module. Here, based on the stored event sequence, a system is constructed as follows: Figure 3a The simulation timing sequence shown indicates the autonomous driving data used when the planning module is triggered to perform calculations based on this sequence. Figure 3b As shown, the first two calculations in the simulation process met expectations, but in the third calculation, since there was no suitable traffic light data for the current moment, the traffic light data of the 6th frame of the historical moment could only be used for the calculation of the current moment (if the traffic light data of the 7th frame was not issued at the current moment).

[0122] However, in actual road testing, the calculations are performed using the traffic light data from the 7th frame. But due to a data storage delay, the time is later. If the simulation is performed without modification, the simulated scenario will inevitably differ from the actual road test, and the calculation results will also differ, failing to achieve the desired simulation reproduction. Therefore, to solve the above problems, this disclosure proposes the following solution:

[0123] In one example, the above method further includes:

[0124] S401, Based on the different autonomous driving test modules in the above-mentioned autonomous driving simulation device, a directed graph is abstracted.

[0125] S402, based on the autonomous driving test data input and output from the different autonomous driving test modules, the directed graph is extended to obtain a simulation cycle for the open-loop simulation test task.

[0126] S403, wait for the end of one of the above simulation cycles before starting the next simulation cycle, to ensure the integrity of each simulation cycle and simulation results.

[0127] Optionally, the autonomous driving test module is set in the autonomous driving simulation device, and the autonomous driving algorithm module is set in the autonomous driving vehicle. The autonomous driving algorithm module includes a localization module, a perception module, a planning module, and a driving control module. It should be noted that, in this embodiment of the disclosure, one or more necessary test modules are selected based on which algorithm modules in the autonomous driving algorithm module are tested. Therefore, the above-mentioned autonomous driving test module includes at least one of the following: a localization module, a perception module, a planning module, and a driving control module.

[0128] Because there are interdependencies between the different autonomous driving test modules mentioned above, and between the autonomous driving test data input to and output by these modules. For example... Figure 4 As shown, in the autonomous driving test module, the positioning module acquires electronic map data, meaning the positioning module depends on electronic map data; the perception module depends on various sensor data and the positioning module; the planning module depends on the perception module and the positioning module; and the driving control module depends on the planning module and the positioning module. Therefore, a directed graph can be obtained based on the dependencies between the autonomous driving test modules. Then, the directed graph can be further expanded based on the dependencies between the input and output autonomous driving test data of the autonomous driving test modules to obtain one simulation cycle (i.e., computation cycle) of the aforementioned open-loop simulation test task.

[0129] In one example, a simulation cycle in this embodiment of the present disclosure typically begins with acquiring vehicle position data, traffic light data, obstacle data, and continues until the current vehicle control result (braking, acceleration, steering, etc.) is obtained.

[0130] In this embodiment, the transmission rate of autonomous driving data during simulation testing is controlled. By waiting for the completion of one simulation cycle before proceeding to the next, the integrity of each simulation cycle and its results is ensured. Furthermore, this embodiment avoids situations where the planning results of one simulation cycle have not been fully calculated before the sensor data required by the planning module for the next simulation cycle is received, thus preventing the sensor data from being discarded after reception.

[0131] In one example, during the process of reproducing the problem scenario in the road test using the processed data described above, the method further includes:

[0132] S401, obtain the autonomous driving test data used by the above-mentioned autonomous driving test module during simulation testing, and the data frame sequence number of the autonomous driving test data used during the above-mentioned simulation testing.

[0133] S402, based on the data frame sequence number of the autonomous driving test data used in the above simulation test, filter out the vehicle trajectory data in the above problem scenario.

[0134] S403, determine the sensor data used by the autonomous driving algorithm module in the aforementioned autonomous vehicle when calculating the aforementioned vehicle trajectory data.

[0135] S404, Based on the aforementioned sensor data, adjust the autonomous driving data used in the aforementioned autonomous driving test module to be consistent with the autonomous driving data used by the aforementioned autonomous driving algorithm module during road testing.

[0136] The aforementioned sensor data includes traffic light data, obstacle data, and vehicle position data. The obstacle data is the key data that triggers the calculation of the vehicle trajectory data.

[0137] In one example, based on the aforementioned sensor data, the autonomous driving data used in the aforementioned autonomous driving test module is adjusted to be consistent with the autonomous driving data used by the aforementioned autonomous driving algorithm module during road testing. Specifically, this can be achieved by adjusting the timing of the aforementioned traffic light data and the aforementioned vehicle position data based on the timing of the aforementioned obstacle data, so that the autonomous driving data used in the aforementioned autonomous driving test module is consistent with the autonomous driving data used by the aforementioned autonomous driving algorithm module during road testing.

[0138] For example, taking the autonomous driving test module, which includes a perception module and a planning module, as an example, the frequency of data calculation for each path is not exactly the same in a simulation cycle. Therefore, the frequency of generating traffic light data, obstacle data, and image data (similar to snapshots, representing the situation at a moment; for example, when obstacle data during road testing is stored in the road acquisition data packet at a predetermined frequency, an "obstacle snapshot" is formed, and the car in the "obstacle snapshot" is the obstacle car. By continuously playing the "obstacle snapshots" to form video-like image data, the behavior of the obstacle car can be interpreted as the actions that differ between two "obstacle snapshots," such as acceleration, changing direction, braking, etc.), radar data, and vehicle position data is uncertain.

[0139] Therefore, in this embodiment of the disclosure, there is also an optional embodiment in which the data frame sequence used in the calculation of the autonomous driving algorithm module is recorded, that is, the frame number of the data used by the autonomous driving algorithm module in the road test is recorded, so as to support the reproduction of problems in the simulation test based on the dependency relationship between the frame numbers, that is, the sequence dependency relationship between the autonomous driving test data input by the autonomous driving algorithm module and the autonomous driving test data output.

[0140] Based on the data frame sequence number of the autonomous driving test data used in the above simulation test, vehicle trajectory data (i.e., vehicle trajectory calculation results) is filtered out from the problem scenario. Based on this vehicle trajectory data, the frame numbers of traffic light data, obstacle data, and vehicle position data used in its calculation are deduced, such as... Figure 5 The dependency relationship between the planning module and the perception module is shown. Taking the vehicle trajectory calculation result of the 3rd frame where the problem occurred as an example, its calculation depends on the obstacle data of the 3rd frame, the traffic light data of the 7th frame, and the vehicle position data of the 14th frame. Among them, the obstacle data is the key data that triggers the calculation.

[0141] Therefore, in the process of constructing the simulation task, the timing of the traffic light data in frame 7 and the vehicle position data in frame 14 can be adjusted and placed after the obstacle data in frame 3. Furthermore, the timing of the traffic light data in frame 8 and the vehicle position data in frame 15 cannot be earlier than the timing of the obstacle data in frame 3. This ensures that the vehicle position data, traffic light data, and obstacle data used in the third trajectory calculation are consistent with the autonomous driving data used by the aforementioned autonomous driving algorithm module during road testing.

[0142] In one example, during the process of reproducing the problem scenario in the road test using the processed data described above, the method further includes:

[0143] S501, Obtain the different autonomous driving test modules in the above-mentioned autonomous driving simulation device, as well as the dependencies between the different autonomous driving test modules.

[0144] S502, based on the different dependencies between the aforementioned autonomous driving test modules, construct the sequence dependency relationship between the input autonomous driving test data and the output autonomous driving test data of the different autonomous driving test modules.

[0145] S503, based on the above sequence dependency relationship, restore the out-of-order data between the different autonomous driving test modules, and control the data transmission rate between the different autonomous driving test modules.

[0146] In this embodiment of the disclosure, the dependencies between different autonomous driving algorithm modules can be determined based on the dependencies between different autonomous driving algorithm modules and the sequence dependencies between the autonomous driving test data input to and output by the autonomous driving algorithm modules. In addition, the sequence dependencies between the autonomous driving test data input to and output by the different autonomous driving test modules can also be determined.

[0147] Still Figure 5 As shown, the calculation and analysis of traffic light data and obstacle data are similar. By analogy, the sequence dependencies (frame dependencies) between image data 1-5 and radar data 1-3 corresponding to the autonomous driving test data are obtained. Finally, the sequence dependencies between all sensor data in a single vehicle trajectory calculation process are obtained. Based on this sequence dependency, combined with the time of autonomous driving data used by the autonomous driving algorithm module during road testing, the sequence dependencies between data when each vehicle trajectory calculation is triggered, as well as the sequence dependencies when each traffic light data and obstacle data calculation is triggered, are constructed. Based on this, the disordered data is restored, for example, by adjusting the storage time of the disordered data to form a sequence as shown in the figure. Figure 3c The repaired simulation time series shown is to ensure that the autonomous driving data used in the autonomous driving test module is consistent with the autonomous driving data used by the above-mentioned autonomous driving algorithm module during road testing, so as to achieve a better effect of reproducing the problem scenario.

[0148] Furthermore, during the simulation test calculation process, the embodiments of this disclosure can also construct a sequence dependency relationship between the autonomous driving test data input and output of different autonomous driving test modules based on the different dependencies between the aforementioned autonomous driving test modules, so as to control the data transmission rate between the different autonomous driving test modules based on the sequence dependency relationship.

[0149] For example, during the planning module's calculations, it primarily relies on obstacle data input from the perception module. However, due to the large computational load in the planning process, ensuring the simulation results are unaffected by machine factors and guaranteeing consistency is crucial. To address this, after replaying the obstacles in the simulation task test, the device does not immediately begin the next round (next simulation cycle). Instead, it waits for the planning module to complete its calculations for the current round (current simulation cycle), meaning the next round of simulation only begins after the planning module issues its planning results. This solves the problem of starting the next round before the previous round is completed, while simultaneously ensuring the consistency of the simulation results. Furthermore, the optimization of message control in the open-loop simulation task further improves the reproducibility of problem scenarios in the open-loop simulation task, ensuring the consistency of the simulation results.

[0150] Example 4

[0151] Based on Embodiment 1 above, this embodiment verifies whether the optimized algorithm module can fix the problems in the above-mentioned problem scenario and obtains the verification results. The specific method and steps are as follows:

[0152] S701, Obtain the target obstacle in the above-mentioned problem scenario, wherein the target obstacle is an obstacle that affects the normal execution and verification of the above-mentioned open-loop simulation test task.

[0153] S702, after cleaning and filtering the target obstacles, verify whether the optimized algorithm module can fix the problems in the above problem scenario, and obtain the above verification results.

[0154] In this embodiment of the disclosure, after reproducing the problem scenario, the open-loop simulation test task can also verify the optimized algorithm module. A common verification method is to use the same autonomous driving data as before optimization and the optimized algorithm module to re-run the simulation task. This method can generally meet the requirements. However, the following situations often occur:

[0155] Take a rear-end collision scenario as an example, such as Figure 6a As shown, in the autonomous driving scenarios (which are Scenario 1, Scenario 2, and Scenario 3 as they progress over time, with Scenario 3 being the scenario where a vehicle collision occurs), the autonomous vehicle (i.e., the test autonomous vehicle, such as...) Figure 6a The main vehicle (as shown) collided with obstacle vehicle A. The optimized algorithm module can avoid the collision by changing lanes or braking. This embodiment uses braking as an example; before the collision is imminent, the optimized algorithm module will brake the autonomous vehicle to avoid the collision. However, the behavior of other obstacle vehicles (obstacle vehicle B) in the road data acquisition packet will not be affected by the optimized algorithm module. Figure 6bAs shown, theoretically, after the autonomous vehicle brakes, obstacle vehicle B should also brake. However, since the road data acquisition data is generated during the road test, the obstacle vehicle will still operate in the way it was in the road test, which eventually leads to a collision between obstacle vehicle B and the autonomous vehicle, i.e., the obstacle vehicle rear-ends the autonomous vehicle, ultimately causing the simulation task to fail verification.

[0156] Therefore, to address the aforementioned issues, existing methods, in order to quickly locate and resolve problems in autonomous driving algorithm modules, typically extract data from the moments when problems occur during road testing for simulation reproduction and verification. However, determining a suitable scene length requires extensive experimentation, and this method may result in insufficient verification. Based on this problem, the autonomous driving simulation device in this embodiment employs a collision avoidance algorithm model. This involves cleaning and filtering obstacles in the current frame, analyzing obstacle information for each frame before and after the collision point, filtering out unreasonable target obstacles, and avoiding the generation of unreasonable problems. Figure 6c In the scenario shown, that is, in the simulation task, unreasonable obstacle car B (gray) is removed to ensure the smooth progress of the verification task of the optimized algorithm module.

[0157] The autonomous driving simulation testing method proposed in this disclosure improves upon current technology from two perspectives: reproducing problem scenarios and verifying optimized algorithm modules. At the reproduction level, this disclosure implements an asynchronous event-based data transmission and reception monitoring mechanism to ensure data arrives in order and retransmits lost data. Furthermore, based on the analyzed sequence dependencies between the input and output data of the autonomous driving algorithm module, it restores out-of-order data and controls the data transmission rate. At the verification level, this disclosure cleans and filters obstacles in the current frame, processing obstacles that affect normal simulation verification and ensuring the smooth progress of the verification task.

[0158] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0159] Example 5

[0160] According to embodiments of this disclosure, Figure 7 This is a schematic diagram of the framework of an autonomous driving simulation testing device provided according to an embodiment of the present disclosure, such as... Figure 7 As shown, this disclosure also provides an autonomous driving simulation test device 700, which includes:

[0161] The acquisition unit 701 is used to acquire autonomous driving data collected by autonomous vehicles during road testing.

[0162] The processing unit 702 is used to correct the data transmission and reception order of the above-mentioned autonomous driving data and to retransmit lost data to obtain processed data.

[0163] The reproduction unit 703 is used to reproduce the problem scenarios in the road test process using the processed data to determine the problems that occur in the autonomous driving algorithm module in the autonomous vehicle in the problem scenarios.

[0164] The optimization unit 704 is used to optimize the autonomous driving algorithm module based on the problems that occurred in the road test, so as to obtain an optimized algorithm module.

[0165] The verification unit 705 is used to verify whether the optimized algorithm module can fix the problems in the above-mentioned problem scenario and obtain the verification result. The verification result is used to indicate whether the optimized algorithm module needs to be further optimized.

[0166] In one example, obtaining a unit includes:

[0167] The first acquisition module is used to acquire sensor data collected by the autonomous vehicle during the road test, as well as calculation data obtained by the autonomous driving algorithm module in the autonomous vehicle based on the sensor data. The autonomous driving algorithm module includes a localization module, a perception module, a planning module, and a driving control module.

[0168] In one example, the processing unit includes:

[0169] The second acquisition module is used to acquire delayed transmission and reception data and lost data in the above-mentioned autonomous driving data;

[0170] The processing module is used to correct the data transmission order of the aforementioned delayed data and to retransmit the aforementioned lost data to obtain the processed data.

[0171] In one example, the second acquisition module mentioned above includes:

[0172] The acquisition subunit is used to acquire the autonomous driving data to be sent to the autonomous driving test module, as well as the data transmission list at the current transmission time;

[0173] The listening subunit is used to send a list of asynchronous events based on the above data, wherein the above autonomous driving test module is used to send an asynchronous event after receiving the above autonomous driving data;

[0174] The parsing and verification subunit is used to parse and verify the above asynchronous events, and update the sending and receiving status of the above autonomous driving data in the hash table based on the parsing and verification results;

[0175] The detection subunit is used to detect the above-mentioned transmission and reception status based on a predetermined frequency in order to obtain the delayed transmission and reception data and lost data in the above-mentioned autonomous driving data.

[0176] In one example, the aforementioned autonomous driving simulation device includes: an autonomous driving test module, a data transmission module, a timer module, a monitoring module, and a simulation control module; the aforementioned acquisition subunit is specifically used for:

[0177] The aforementioned data transmission module is used to parse the aforementioned autonomous driving data to obtain the parsed autonomous driving data and the aforementioned data transmission list at the current transmission time;

[0178] The data sending module is further configured to inform the monitoring module of the data sending list at the current sending time, send the parsed autonomous driving data to the autonomous driving test module, and register a callback function when sending the parsed autonomous driving data. The autonomous driving test module is configured to send an asynchronous event based on the callback function after receiving the parsed autonomous driving data and before processing the parsed autonomous driving data, so as to inform the monitoring module that the parsed autonomous driving data has been received normally.

[0179] In one example, the aforementioned listening subunit is specifically used to: use the aforementioned monitoring module to listen for the aforementioned asynchronous events based on the aforementioned data sending list; the aforementioned parsing and verification subunit is specifically used to: parse and verify the aforementioned asynchronous events after they are detected, and update the aforementioned sending and receiving status in the hash table of the aforementioned monitoring module according to the aforementioned parsing and verification results.

[0180] In one example, the detection subunit is specifically used to: use the simulation control module under the control of the timer module to detect the transmission and reception status recorded in the hash table of the monitoring module at a predetermined frequency, so as to obtain the delayed transmission and reception data and lost data in the autonomous driving data.

[0181] In one example, the aforementioned parsing and verification subunit is further configured to: if the hash table of the monitoring module is checked and it is determined that all data at the current sending time has been successfully sent and received, then proceed to the next sending time until all autonomous driving data has been processed; if the hash table of the monitoring module is checked and it is determined that all data at the current sending time has not been successfully sent or received, then determine the delayed and lost data in the autonomous driving data, thereby triggering the data sending module to perform data transmission order correction processing on the delayed data and retransmission processing on the lost data, to obtain the processed data.

[0182] In one example, the above-mentioned device further includes:

[0183] The relationship acquisition unit is used to acquire different autonomous driving test modules in the above-mentioned autonomous driving simulation device, as well as the dependencies between the different autonomous driving test modules, wherein the above-mentioned autonomous driving test modules include at least one of the following: a localization module, a perception module, a planning module, and a driving control module;

[0184] The relation building unit is used to build sequence dependency relationships between the input autonomous driving test data and the output autonomous driving test data of different autonomous driving test modules based on the different dependency relationships between the aforementioned autonomous driving test modules.

[0185] The data processing unit is used to restore the out-of-order data between the different autonomous driving test modules according to the above-mentioned sequence dependency relationship, and to control the data transmission rate between the different autonomous driving test modules.

[0186] In one example, the above-mentioned device further includes:

[0187] The frame sequence number acquisition unit is used to acquire the autonomous driving test data used by the above-mentioned autonomous driving test module during simulation testing, as well as the data frame sequence number of the autonomous driving test data used during the above-mentioned simulation testing.

[0188] The data filtering unit is used to filter out vehicle trajectory data in the above-mentioned problem scenarios based on the data frame sequence number of the autonomous driving test data used in the above simulation test.

[0189] The data determination unit is used to determine the sensor data used by the autonomous driving algorithm module in the aforementioned autonomous vehicle when calculating the aforementioned vehicle trajectory data;

[0190] The data adjustment unit is used to adjust the autonomous driving data used in the autonomous driving test module to be consistent with the autonomous driving data used by the autonomous driving algorithm module during road testing, based on the aforementioned sensor data.

[0191] In one example, the aforementioned sensor data includes traffic light data, obstacle data, and vehicle position data, wherein the obstacle data is key data that triggers the calculation of the vehicle trajectory data; the aforementioned data adjustment unit is further configured to: adjust the timing of the aforementioned traffic light data and the aforementioned vehicle position data according to the timing of the aforementioned obstacle data, so that the autonomous driving data used in the aforementioned autonomous driving test module is adjusted to be consistent with the autonomous driving data used by the aforementioned autonomous driving algorithm module during road testing.

[0192] In one example, the above-mentioned device further includes:

[0193] The directed graph determination unit is used to abstract a directed graph based on the different autonomous driving test modules in the above-mentioned autonomous driving simulation device.

[0194] The simulation cycle determination unit is used to expand the directed graph based on the autonomous driving test data input and output from different autonomous driving test modules to obtain a simulation cycle for the open-loop simulation test task.

[0195] The simulation cycle control unit is used to wait for the completion of one of the above simulation cycles before starting the next simulation cycle, so as to ensure the integrity of each simulation cycle and simulation results.

[0196] In one example, the above verification unit includes:

[0197] The obstacle acquisition module is used to acquire target obstacles in the above-mentioned problem scenario, wherein the target obstacles are obstacles that affect the normal execution and verification of the above-mentioned open-loop simulation test task;

[0198] The obstacle removal module is used to clean and filter the target obstacles, and then verify whether the optimized algorithm module can fix the problems in the above-mentioned problem scenario, and obtain the above verification results.

[0199] Example 6

[0200] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0201] According to embodiments of the present disclosure, the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method according to any one of the above claims.

[0202] According to embodiments of the present disclosure, a computer program product is provided, comprising: a computer program stored in a readable storage medium, at least one processor of an electronic device being able to read the computer program from the readable storage medium, and the at least one processor executing the computer program causing the electronic device to perform the scheme provided in any of the above embodiments.

[0203] According to embodiments of this disclosure, this disclosure also provides an electronic device. Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0204] like Figure 8 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0205] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0206] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as an autonomous driving simulation testing method. For example, in some embodiments, the autonomous driving simulation testing method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the autonomous driving simulation testing method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the autonomous driving simulation testing method by any other suitable means (e.g., by means of firmware).

[0207] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0208] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0209] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0210] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0211] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0212] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0213] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0214] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An autonomous driving simulation testing method, applied to an autonomous driving simulation device to perform an open-loop simulation testing task, the method comprising: Acquire autonomous driving data collected by autonomous vehicles during road testing; The delayed data transmission and reception order in the autonomous driving data is corrected, and the lost data in the autonomous driving data is retransmitted to obtain the processed data. The processed data is used to reproduce the problem scenarios during the road test to determine the problems that occur in the autonomous driving algorithm module of the autonomous vehicle in the problem scenarios. Based on the problems encountered by the autonomous driving algorithm module in the road test, the autonomous driving algorithm module is optimized to obtain an optimized algorithm module; Verify whether the optimized algorithm module can fix the problem in the problem scenario, and obtain the verification result, wherein the verification result is used to characterize whether the optimized algorithm module needs to be further optimized; In the process of reproducing the problem scenario during the road test using the processed data, the method further includes: The autonomous driving simulation device acquires different autonomous driving test modules and the dependencies between them, wherein the autonomous driving test module includes at least one of the following: a localization module, a perception module, a planning module, and a driving control module. Based on the dependencies between different autonomous driving test modules, a sequence dependency relationship is constructed between the autonomous driving test data input to and the autonomous driving test data output by different autonomous driving test modules; Based on the sequence dependency relationship, the out-of-order data between the different autonomous driving test modules is restored, and the data transmission rate between the different autonomous driving test modules is controlled.

2. The method according to claim 1, wherein, Acquire autonomous driving data collected during road testing of autonomous vehicles, including: The autonomous driving vehicle acquires sensor data collected during the road test, as well as computational data obtained by the autonomous driving algorithm module based on the sensor data. The autonomous driving algorithm module includes a localization module, a perception module, a planning module, and a driving control module.

3. The method according to claim 1, further comprising: Acquire delayed and lost data from the autonomous driving data.

4. The method according to claim 3, wherein, The acquisition of delayed and lost data from the autonomous driving data includes: Obtain the autonomous driving data to be sent to the autonomous driving test module, as well as the data transmission list at the current transmission time; Based on the data sending list, listen for asynchronous events, wherein the autonomous driving test module is used to send an asynchronous event after receiving the autonomous driving data; The asynchronous events are parsed and verified, and the sending and receiving status of the autonomous driving data is updated in the hash table based on the parsing and verification results; The transmission and reception status is detected based on a predetermined frequency to obtain delayed transmission and reception data and lost data in the autonomous driving data.

5. The method according to claim 4, wherein, The autonomous driving simulation device includes: an autonomous driving test module, a data transmission module, a timer module, a monitoring module, and a simulation control module; the acquisition of autonomous driving data to be sent to the autonomous driving test module, and the data transmission list at the current transmission time, includes: The data transmission module is used to parse the autonomous driving data to obtain the parsed autonomous driving data and the data transmission list at the current transmission time; The data sending module is further configured to inform the monitoring module of the data sending list at the current sending time, send the parsed autonomous driving data to the autonomous driving test module, and register a callback function when sending the parsed autonomous driving data. The autonomous driving test module is configured to send an asynchronous event based on the callback function after receiving the parsed autonomous driving data and before processing the parsed autonomous driving data, so as to inform the monitoring module that the parsed autonomous driving data has been received normally.

6. The method according to claim 5, wherein, The step of listening for asynchronous events based on the data sending list includes: using the monitoring module to listen for the asynchronous events based on the data sending list; The step of parsing and verifying the asynchronous event to update the transmission and reception status of the autonomous driving data in the hash table based on the parsing and verification results includes: parsing and verifying the asynchronous event after it is detected, and updating the transmission and reception status in the hash table of the monitoring module according to the parsing and verification results.

7. The method according to claim 6, wherein, The step of detecting the transmission and reception status based on a predetermined frequency to obtain delayed transmission and reception data and lost data in the autonomous driving data includes: The simulation control module, under the control of the timer module, detects the transmission and reception status recorded in the hash table of the monitoring module at a predetermined frequency to obtain the delayed transmission and reception data and lost data in the autonomous driving data.

8. The method according to claim 7, wherein, The step of detecting the transmission and reception status recorded in the hash table of the monitoring module based on a predetermined frequency includes: If the hash table of the monitoring module is checked and it is determined that all data at the current sending time has been successfully sent and received, then the next sending time will begin, until all autonomous driving data has been processed. If the hash table of the monitoring module is checked and it is determined that all data at the current sending time has not been sent or has not been successfully received, then the delayed data and lost data in the autonomous driving data are identified, so as to trigger the data sending module to perform data transmission order correction processing on the delayed data and retransmission processing on the lost data to obtain the processed data.

9. The method according to any one of claims 1 to 8, wherein, In the process of reproducing the problem scenario during the road test using the processed data, the method further includes: Obtain the autonomous driving test data used by the autonomous driving test module during simulation testing, and the data frame sequence number of the autonomous driving test data used during simulation testing; Based on the data frame sequence number of the autonomous driving test data used in the simulation test, the vehicle trajectory data in the problem scenario is filtered out; Determine the sensor data used by the autonomous driving algorithm module in the autonomous vehicle when calculating the vehicle trajectory data; Based on the sensor data, the autonomous driving data used in the autonomous driving test module is adjusted to be consistent with the autonomous driving data used by the autonomous driving algorithm module during road testing.

10. The method according to claim 9, wherein, The sensor data includes traffic light data, obstacle data, and vehicle position data, wherein the obstacle data is key data for triggering the calculation of the vehicle trajectory data; based on the sensor data, the autonomous driving data used in the autonomous driving test module is adjusted to be consistent with the autonomous driving data used by the autonomous driving algorithm module during road testing, including: Based on the time of the obstacle data, the time of the traffic light data and the vehicle position data are adjusted so that the autonomous driving data used in the autonomous driving test module is consistent with the autonomous driving data used by the autonomous driving algorithm module during road testing.

11. The method according to claim 9, wherein, The method further includes: Based on the different autonomous driving test modules in the autonomous driving simulation device, a directed graph is abstracted. Based on the autonomous driving test data input and output from different autonomous driving test modules, the directed graph is expanded to obtain a simulation cycle of the open-loop simulation test task. After one simulation cycle ends, the next simulation cycle begins to ensure the integrity of each simulation cycle and the simulation results.

12. The method according to claim 1, wherein, Verify whether the optimized algorithm module can fix the problems in the problem scenario, and obtain the verification results, including: Obtain the target obstacle in the problem scenario, wherein the target obstacle is an obstacle that affects the normal execution and verification of the open-loop simulation test task; After cleaning and filtering the target obstacles, the optimized algorithm module is verified to determine whether it can fix the problems in the problem scenario, and the verification result is obtained.

13. An autonomous driving simulation testing device, the device comprising: The acquisition unit is used to acquire autonomous driving data collected by autonomous vehicles during road testing. The processing unit is used to correct the data transmission order of delayed data in the autonomous driving data and to retransmit lost data in the autonomous driving data to obtain processed data. The reproduction unit is used to reproduce the problem scenarios during the road test using the processed data, so as to determine the problems that occur in the autonomous driving algorithm module in the autonomous vehicle in the problem scenarios. An optimization unit is used to optimize the autonomous driving algorithm module based on the problems encountered by the autonomous driving algorithm module in the road test, so as to obtain an optimized algorithm module. A verification unit is used to verify whether the optimized algorithm module can fix the problem in the problem scenario and obtain a verification result, wherein the verification result is used to indicate whether the optimized algorithm module needs to be further optimized. The relationship acquisition unit is used to acquire different autonomous driving test modules in the autonomous driving simulation test device, as well as the dependencies between the different autonomous driving test modules, wherein the autonomous driving test module includes at least one of the following: a localization module, a perception module, a planning module, and a driving control module. The relationship building unit is used to build a sequence dependency relationship between the input autonomous driving test data and the output autonomous driving test data of different autonomous driving test modules based on the dependency relationship between the different autonomous driving test modules. The data processing unit is used to restore the out-of-order data between the different autonomous driving test modules according to the sequence dependency relationship, and to control the data transmission rate between the different autonomous driving test modules.

14. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-12.

15. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-12.

16. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-12.

Images