Test method, device, medium and product for vehicle automatic driving planning algorithm

Abstract

The application provides a test method, device, medium and product of a vehicle automatic driving planning algorithm, relates to the technical field of vehicles, and comprises the following steps: acquiring real driving data of a vehicle; performing neural rendering processing on static scene data in the real driving data of the vehicle to obtain reconstructed target static scene data; injecting semantic information corresponding to the real driving data of the vehicle into the target static scene data to obtain test simulation scene data; and performing simulation testing on a vehicle automatic driving planning algorithm based on the test simulation scene data. The application can realize reliable closed-loop evaluation of the vehicle automatic driving planning algorithm.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a testing method, device, medium, and product for a vehicle autonomous driving planning algorithm. Background Technology

[0002] In related technologies, the mainstream closed-loop testing method for autonomous driving planning algorithms mainly involves constructing test scenarios in game engines (such as the autonomous driving simulation simulator CARLA) or traffic simulators (such as the traffic flow simulation platform SUMO) to conduct algorithm testing. However, this type of testing method suffers from the inherent gap between simulation and reality: the simulation environment is a simplification and simulation of real traffic scenarios and cannot fully reproduce the complex road conditions, weather changes, sudden interference, and other dynamic factors in the real world. This leads to autonomous driving planning algorithms that perform well in simulation environments experiencing significant performance degradation when applied to real road environments, such as decision lag and unreasonable planning. In other words, the aforementioned mainstream testing methods cannot truly reflect the reliability of autonomous driving planning algorithms in real-world scenarios. Summary of the Invention

[0003] The main purpose of this application is to propose a testing method, equipment, medium, and product for vehicle autonomous driving planning algorithms, aiming to achieve reliable closed-loop evaluation of vehicle autonomous driving planning algorithms.

[0004] To achieve the above objectives, the first aspect of this application proposes a testing method for a vehicle autonomous driving planning algorithm, the method comprising:

[0005] Obtain real-time vehicle driving data;

[0006] The static scene data in the real driving data of the vehicle is subjected to neural rendering processing to obtain the reconstructed target static scene data.

[0007] Semantic information corresponding to the real driving data of the vehicle is injected into the target static scene data to obtain test simulation scene data;

[0008] The vehicle autonomous driving planning algorithm was simulated and tested based on the test simulation scenario data.

[0009] In some embodiments, the simulation test of the vehicle autonomous driving planning algorithm based on the test simulation scenario data includes:

[0010] Obtain virtual perception data corresponding to the test simulation scenario data;

[0011] The virtual perception data is input into the vehicle autonomous driving planning algorithm for trajectory planning processing to obtain the vehicle autonomous driving control command output by the vehicle autonomous driving planning algorithm.

[0012] Based on the vehicle autonomous driving control commands and the traffic agent behavior data corresponding to the test simulation scenario data, state deduction is performed to obtain world state information.

[0013] In some embodiments, the method further includes at least one of the following:

[0014] Traffic agent behavior data corresponding to the test simulation scenario data is generated based on a preset generative artificial intelligence model; the generative artificial intelligence model is obtained by learning and training the behavior of traffic participants on real vehicle driving data;

[0015] Based on the behavioral parameter information of dynamic objects in the real driving data of the vehicle, traffic agent behavior data corresponding to the test simulation scenario data is generated.

[0016] In some embodiments, acquiring virtual perception data corresponding to the test simulation scenario data includes:

[0017] When the static scene data is processed by neural rendering based on a preset neural rendering model, virtual perception data corresponding to the test simulation scene data is obtained by sampling the intermediate representation generated by the neural rendering model.

[0018] In some embodiments, the method further includes:

[0019] Acquire interactive data for simulation testing of the vehicle autonomous driving planning algorithm; the interactive data includes the vehicle autonomous driving control commands, the traffic agent behavior data, and the world state information;

[0020] Based on the vehicle autonomous driving control commands, the traffic agent behavior data, and the world state information, multi-dimensional quantitative indicators of the vehicle autonomous driving planning algorithm are calculated; the multi-dimensional quantitative indicators include safety indicators, interactivity indicators, and intelligence indicators.

[0021] In some embodiments, performing neural rendering processing on the static scene data in the real driving data of the vehicle includes:

[0022] The vehicle's real driving data is subjected to dynamic object segmentation processing to obtain static scene data after dynamic object segmentation in the vehicle's real driving data.

[0023] The static scene data is input into a preset neural rendering model for neural rendering processing.

[0024] In some embodiments, the real-world driving data of the vehicle includes image sequences of real-world driving scenarios of the vehicle, and the semantic information includes map vector elements corresponding to the image sequences;

[0025] The step of injecting semantic information corresponding to the vehicle's real driving data into the target static scene data includes:

[0026] Vectorized map construction is performed based on the image sequence to obtain map vector elements corresponding to the image sequence; spatial semantic registration is performed between the map vector elements and the target static scene data to inject the map vector elements into the target static scene data.

[0027] In some embodiments, the method further includes:

[0028] The simulation scenario modification data corresponding to the edge test cases is obtained based on the preset scenario editing interface;

[0029] Based on the simulation scenario modification data, the test simulation scenario data is adjusted to obtain edge test scenario data corresponding to the edge test cases;

[0030] The simulation test of the vehicle autonomous driving planning algorithm based on the test simulation scenario data includes:

[0031] The vehicle autonomous driving planning algorithm was simulated and tested based on the edge test scenario data.

[0032] To achieve the above objectives, a second aspect of this application proposes a testing apparatus for a vehicle autonomous driving planning algorithm, the apparatus comprising:

[0033] The acquisition module is used to acquire real-time driving data of the vehicle.

[0034] The neural rendering module is used to perform neural rendering processing on the static scene data in the real driving data of the vehicle to obtain the reconstructed target static scene data.

[0035] The scene reconstruction module is used to inject semantic information corresponding to the real driving data of the vehicle into the target static scene data to obtain test simulation scene data;

[0036] The closed-loop testing module is used to perform simulation testing on the vehicle autonomous driving planning algorithm based on the test simulation scenario data.

[0037] To achieve the above objectives, a third aspect of this application provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the test method for the vehicle autonomous driving planning algorithm described in the first aspect.

[0038] To achieve the above objectives, a fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a test method for the vehicle autonomous driving planning algorithm described in the first aspect.

[0039] To achieve the above objectives, the fifth aspect of this application provides a computer program product, which includes a computer program that, when executed by a processor, implements a test method for the vehicle autonomous driving planning algorithm provided in the first aspect above.

[0040] The testing method, apparatus, electronic device, computer-readable storage medium, and computer program product for the vehicle autonomous driving planning algorithm proposed in this application involve: acquiring real vehicle driving data; performing neural rendering processing on static scene data in the real vehicle driving data to obtain reconstructed target static scene data; injecting semantic information corresponding to the real vehicle driving data into the target static scene data to obtain test simulation scene data; and performing simulation testing on the vehicle autonomous driving planning algorithm based on the test simulation scene data.

[0041] Compared to mainstream closed-loop testing methods that construct test scenarios in autonomous driving simulators like CARLA or traffic flow simulation platforms like SUMO to conduct algorithm testing, this application uses neural rendering to process static scene data from real vehicle driving data to obtain reconstructed target static scene data. Then, semantic information corresponding to the real vehicle driving data is injected into this target static scene data to obtain test simulation scene data. Based on this test simulation scene data, the autonomous driving planning algorithm can be simulated and tested. When conducting closed-loop simulation testing of the autonomous driving planning algorithm, neural rendering provides photorealistic visual fidelity, greatly reducing the visual gap between simulation and reality. This solves the problems of insufficient realism and the Sim2Real gap, enabling reliable closed-loop evaluation of the autonomous driving planning algorithm. Attached Figure Description

[0042] Figure 1 A flowchart illustrating the steps of the test method for the vehicle autonomous driving planning algorithm provided in this application in some embodiments;

[0043] Figure 2 for Figure 1A detailed flowchart of step S102;

[0044] Figure 3 for Figure 1 A detailed flowchart of step S103;

[0045] Figure 4 A flowchart illustrating the steps of the test method for the vehicle autonomous driving planning algorithm provided in this application in some other embodiments;

[0046] Figure 5 for Figure 1 A detailed flowchart of step S104;

[0047] Figure 6 A flowchart illustrating the steps of the test method for the vehicle autonomous driving planning algorithm provided in this application in some of the embodiments;

[0048] Figure 7 A complete system flowchart of the test method for the vehicle autonomous driving planning algorithm provided in this application;

[0049] Figure 8 A schematic diagram of the structure of the test device for the vehicle autonomous driving planning algorithm provided in this application;

[0050] Figure 9 A schematic diagram of the hardware structure of the electronic device provided in this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] It should be noted that although functional modules are divided in the device / system schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device / system or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this application.

[0054] First, the overall concept of the testing method for the vehicle autonomous driving planning algorithm provided in this application will be explained.

[0055] In recent years, autonomous driving technology has developed rapidly. As a core component of autonomous driving, the decision-making and planning system directly determines its safety and reliability. Therefore, conducting thorough, safe, and efficient testing and verification of the decision-making and planning system (i.e., autonomous driving planning algorithm) has become a key challenge for the industry. However, mainstream testing methods for autonomous driving planning algorithms all have certain shortcomings:

[0056] Open-loop evaluation: This method uses pre-recorded real-world datasets (such as sensor log playback) for testing. The planning algorithm under evaluation generates a trajectory based on historical data and compares it with the actual trajectory of a human driver (e.g., L2 distance, speed error, etc.). Its fatal flaw is that it is a static evaluation that relies on hindsight, completely ignoring the dynamic feedback and impact of the vehicle's behavior on the surrounding environment. A trajectory that performs well in open-loop evaluation may trigger other vehicles to brake suddenly or swerve in actual execution, leading to dangerous accidents; therefore, the evaluation results are unreliable.

[0057] Real-world road testing is extremely costly, risky, and time-consuming, and it is difficult to cover critical moments (Corner Cases), thus failing to meet the needs of rapid algorithm iteration development.

[0058] Traditional Closed-loop Simulation: This method builds test scenarios within game engines (such as CARLA) or traffic simulators (such as SUMO). While it considers interaction, it has two major problems:

[0059] Insufficient realism: The appearance, texture, and lighting of virtual scenes, vehicles, and pedestrians differ significantly from the real world, i.e., the "Sim2Real" gap, which causes the performance of algorithms that perform well in simulation to degrade when migrated to the real world.

[0060] Traffic agents (also known as interactive intelligent agents) have simple behaviors: surrounding traffic participants are usually controlled by simple rule models (such as intelligent driver models IDM), whose behavior is too idealized, patterned and predictable, and cannot simulate the complex, variable, and sometimes even unreasonable game behavior of human drivers. This makes the test scenarios too simple and cannot effectively verify the performance of the algorithm in real complex traffic environments.

[0061] Therefore, there is an urgent need for a closed-loop evaluation scheme for vehicle autonomous driving planning algorithms that can reflect real-world interactive feedback, provide high-fidelity visual scenes, and are safe and efficient.

[0062] To address the aforementioned issues, this application proposes a testing method, apparatus, electronic device, computer-readable storage medium, and computer program product for vehicle autonomous driving planning algorithms. The aim is to overcome the shortcomings of existing technologies, construct a visually high-fidelity, behaviorally human-like, and flexibly editable testing environment, and achieve comprehensive, reliable, and efficient closed-loop evaluation of autonomous driving planning algorithms across multiple dimensions, including safety, efficiency, comfort, and intelligence.

[0063] It should be noted that the aforementioned vehicle autonomous driving planning algorithm can also be called a vehicle planning algorithm or an autonomous driving planning algorithm. It is a core component of the "Perception-Localization-Planning-Control" (PLPC) technology chain in vehicle autonomous driving systems. Based on the acquisition of perception data such as environmental obstacles, road structure, and traffic signals, the vehicle's own position / attitude localization information, navigation routes, and vehicle dynamic constraints (such as speed, steering angle, and acceleration limits), the vehicle autonomous driving planning algorithm determines the vehicle's driving behavior (such as following, overtaking, turning, and parking) and generates safe, efficient, and compliant continuous driving trajectories and control commands (speed and steering commands), ultimately guiding the vehicle smoothly from its current state to the target state.

[0064] In the industry, autonomous driving planning algorithms are typically categorized into three types based on their planning hierarchy: Global Path Planning, Behavior Planning, and Local Trajectory Planning. Global Path Planning is a long-distance path planning module based on high-precision maps / navigation maps. It doesn't consider real-time dynamic obstacles, focusing only on the shortest, least energy-consuming, and least time-consuming route from the starting point to the destination, aligning with navigation intent. Behavior Planning is a mid-level decision-making module in autonomous driving. Based on real-time environmental perception results (dynamic obstacle behavior, traffic signals, intersection status), it determines the vehicle's driving behavior (such as following, overtaking, merging, yielding, stopping, etc.) to ensure compliant and risk-controllable behavior. It primarily addresses the "when to do what" decision problem (e.g., "If the vehicle ahead slows down, should I overtake / follow?"). Local trajectory planning is a low-level execution-level planning method. Based on behavioral decision instructions, real-time obstacle status, and vehicle dynamics constraints, it generates continuous and smooth trajectories within a short time window (e.g., 1-5 seconds). It is safe (collision-free), smooth (compliant with vehicle dynamics), and has high tracking accuracy. It mainly solves the "how to do" execution problem, and the control instructions it generates can be directly input into the vehicle control system for execution.

[0065] The testing method, apparatus, electronic device, computer-readable storage medium, and computer program product for the vehicle autonomous driving planning algorithm proposed in this application involve: acquiring real vehicle driving data; performing neural rendering processing on static scene data in the real vehicle driving data to obtain reconstructed target static scene data; injecting semantic information corresponding to the real vehicle driving data into the target static scene data to obtain test simulation scene data; and performing simulation testing on the vehicle autonomous driving planning algorithm based on the test simulation scene data.

[0066] Compared to mainstream closed-loop testing methods that construct test scenarios in autonomous driving simulators like CARLA or traffic flow simulation platforms like SUMO to conduct algorithm testing, this application uses neural rendering to process static scene data from real vehicle driving data to obtain reconstructed target static scene data. Then, semantic information corresponding to the real vehicle driving data is injected into this target static scene data to obtain test simulation scene data. Based on this test simulation scene data, the autonomous driving planning algorithm can be simulated and tested. When conducting closed-loop simulation testing of the autonomous driving planning algorithm, neural rendering provides photorealistic visual fidelity, greatly reducing the visual gap between simulation and reality. This solves the problems of insufficient realism and the Sim2Real gap, enabling reliable closed-loop evaluation of the autonomous driving planning algorithm.

[0067] Next, the testing method, apparatus, electronic device, computer-readable storage medium, and computer program product of the vehicle autonomous driving planning algorithm provided in this application will be specifically described through the following embodiments, and firstly, the various detailed embodiments of the testing method of the vehicle autonomous driving planning algorithm provided in this application will be described in detail.

[0068] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent will be obtained first. Furthermore, the collection, use, and processing of this data will comply with relevant laws, regulations, and standards. In addition, when this application needs to obtain sensitive personal information of users, separate permission or consent from the user will be obtained through pop-ups or redirects to confirmation pages. Only after obtaining the user's separate permission or consent will the necessary user-related data for the normal operation of this application be obtained.

[0069] It should be noted that the testing method for the vehicle autonomous driving planning algorithm provided in this application relates to the field of vehicle technology. The testing method for the vehicle autonomous driving planning algorithm provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be an in-vehicle terminal, or an electronic device such as a smartphone, tablet, laptop, or desktop computer that is associated with the vehicle and can communicate and interact with it via a network. The server can be a backend server terminal device of the terminal, which can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The software can be an application implementing the testing method for the vehicle autonomous driving planning algorithm, a computer program, and a storage medium carrying the computer program. It should be understood that, based on different design needs of practical applications, the terminal, server, and software of the test method for the vehicle autonomous driving planning algorithm provided in this application may also be other forms not listed here, depending on the different feasible embodiments. The test method for the vehicle autonomous driving planning algorithm provided in this application does not specifically limit these.

[0070] Furthermore, this application can also be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: vehicle terminals, personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, personal computers (PCs), minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via communication networks. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0071] For ease of understanding and explanation, the following text uses the test method for applying the vehicle autonomous driving planning algorithm provided in this application to a terminal device as an example to describe the various specific embodiments of this application in detail. The terminal device can use the test method for applying the vehicle autonomous driving planning algorithm provided in this application to perform closed-loop simulation testing of the vehicle autonomous driving planning algorithm. In some descriptions, the terminal device may be simply referred to as the terminal. The implementation of the test method for applying the vehicle autonomous driving planning algorithm provided in this application to any of the above-described subject matter can refer to the process of applying the test method for applying the vehicle autonomous driving planning algorithm to a terminal device as described below.

[0072] Please refer to Figure 1 , Figure 1 The flowchart illustrates the steps of the testing method for the vehicle autonomous driving planning algorithm provided in this application in some embodiments. It should be understood that, although... Figure 1 The flowcharts illustrating subsequent steps show the execution order of some method steps. However, based on different design needs in practical applications, the testing method for the vehicle autonomous driving planning algorithm provided in this application can, of course, employ a different execution order of method steps than shown in the figures. That is, Figure 1 The order of the steps shown does not constitute a limitation on the execution logic order of the test method for the vehicle autonomous driving planning algorithm provided in this application. Any other method based on... Figure 1 Reasonable changes to the sequence of steps shown should be included within the protection scope of the test method for the vehicle autonomous driving planning algorithm provided in this application.

[0073] like Figure 1 As shown, in some embodiments, the testing method for the vehicle autonomous driving planning algorithm provided in this application may include steps S101 to S104 as shown below.

[0074] Step S101: Obtain real driving data of the vehicle.

[0075] It should be noted that real-world vehicle driving data can be multimodal data collected from real-world vehicles, including multi-camera videos, LiDAR point clouds, pose information, and so on.

[0076] Terminal devices can acquire real-world driving data of vehicles when performing closed-loop simulation tests on autonomous driving planning algorithms.

[0077] In some embodiments, the terminal device may acquire real-time vehicle driving data stored locally.

[0078] In other embodiments, the terminal device can also obtain real driving data of the vehicle provided by the staff through a human machine interface (HMI).

[0079] Step S102: Perform neural rendering processing on the static scene data in the real driving data of the vehicle to obtain the reconstructed target static scene data.

[0080] It should be noted that the static scene data in real vehicle driving data can be a clean, high-fidelity static background model obtained after separating and removing dynamic objects from the real vehicle driving data.

[0081] After acquiring the vehicle's real driving data, the terminal device further performs high-fidelity simulation test scene reconstruction based on the vehicle's real driving data. First, it performs neural rendering processing on the static scene data in the vehicle's real driving data to obtain the reconstructed target static scene data.

[0082] In some embodiments, the terminal device may use neural rendering technologies such as Neural Radiation Field (NeRF) and 3D Gaussian Sputtering (3DGS) to perform neural rendering processing on static scene data in real vehicle driving data.

[0083] Step S103: Inject semantic information corresponding to the real driving data of the vehicle into the target static scene data to obtain test simulation scene data.

[0084] It should be noted that the semantic information corresponding to the actual driving data of a vehicle can be physical and semantic information such as collision objects, drivable areas, lane topology, and traffic rules.

[0085] During the process of reconstructing a high-fidelity simulation test scenario based on real vehicle driving data, the terminal device, after obtaining the reconstructed target static scene data, further injects semantic information corresponding to the real vehicle driving data into the target static scene data, thereby obtaining the reconstructed test simulation scene data.

[0086] In some embodiments, when a terminal device reconstructs a high-fidelity simulation test scene based on real vehicle driving data, it can employ neural rendering technologies such as Neural Radiation Field (NeRF) and 3D Gaussian Sputtering (3DGS). Through an automated process, dynamic objects in the original data are first separated and removed to obtain a clean, high-fidelity static background model. Subsequently, physical and semantic information (such as collision objects, drivable areas, lane topology, and traffic rules) is automatically injected into the model, transforming it from a "visual model" into an "interactive simulation world." This completes the high-fidelity reconstruction of the simulation test scene based on real vehicle driving data, resulting in reconstructed test simulation scene data.

[0087] In some embodiments, the terminal device can use the high-fidelity editable scene reconstruction module in the autonomous driving trajectory closed-loop evaluation system to process multimodal data (including multi-camera video, LiDAR point clouds, and pose information) collected in the real world, and automatically reconstruct a photorealistic dynamic 3D scene based on neural radiation field (NeRF) or 3D Gaussian sputtering (3DGS) neural rendering technologies. Specifically, after acquiring real vehicle driving data, the terminal device uses the high-fidelity editable scene reconstruction module to separate dynamic objects to reconstruct a static scene, obtaining target static scene data. Then, the high-fidelity editable scene reconstruction module injects semantic and physical information into the target static scene data to obtain reconstructed test simulation scene data.

[0088] It should be noted that the core of the high-fidelity editable scene reconstruction module lies in its ability to separate dynamic objects and inject semantics. It can identify, segment and remove dynamic objects in the original data (i.e., real vehicle driving data) to obtain a clean static background model (i.e., target static scene data), and automatically generate corresponding physical and semantic information such as collision bodies, drivable areas, lane topology and traffic rules for the model.

[0089] In some embodiments, the high-fidelity editable scene reconstruction module also provides a scene editing interface, which supports controllable modification of weather, lighting, traffic density and road topology of the reconstructed scene (i.e. test simulation scene data).

[0090] Step S104: Perform simulation testing on the vehicle autonomous driving planning algorithm based on the test simulation scenario data.

[0091] After reconstructing the test simulation scenario data, the terminal device further conducts closed-loop simulation tests on the vehicle autonomous driving planning algorithm based on the test simulation scenario data.

[0092] In some embodiments, when the terminal device performs closed-loop simulation testing on the vehicle autonomous driving planning algorithm based on the test simulation scenario data, it can remove the original dynamic objects in the test simulation scenario data according to the test case requirements, and generate and configure the surrounding traffic agents and their behavior parameters. Then, it sets the initial simulation state: loading the scenario (test simulation scenario data), agents (traffic agents and their behavior parameters), and the planning algorithm to be tested (vehicle autonomous driving planning algorithm), and initializes the asynchronous synchronous framework. Finally, in each simulation step, it generates perception data and inputs it into the planning algorithm. The algorithm outputs control commands, and the simulation engine integrates the vehicle commands and agent behaviors to update the world state at the next moment.

[0093] In this embodiment, when the terminal device performs a closed-loop simulation test on the vehicle autonomous driving planning algorithm, it acquires real vehicle driving data and reconstructs a high-fidelity simulation test scene from the real vehicle driving data. First, it performs neural rendering processing on the static scene data in the real vehicle driving data to obtain the reconstructed target static scene data. Then, it injects semantic information corresponding to the real vehicle driving data into the target static scene data to obtain the reconstructed test simulation scene data. Finally, based on the test simulation scene data, it performs a closed-loop simulation test on the vehicle autonomous driving planning algorithm.

[0094] Therefore, compared to the mainstream closed-loop testing method of constructing test scenarios in autonomous driving simulation simulators like CARLA or traffic flow simulation platforms like SUMO to conduct algorithm testing, this application uses neural rendering to process static scene data from real vehicle driving data to obtain reconstructed target static scene data. Then, semantic information corresponding to real vehicle driving data is injected into this target static scene data to obtain test simulation scene data. Based on this test simulation scene data, the autonomous driving planning algorithm can be simulated and tested. When conducting closed-loop simulation testing of the autonomous driving planning algorithm, neural rendering provides photorealistic visual fidelity, greatly reducing the visual gap between simulation and reality. This solves the problems of insufficient realism and the Sim2Real gap, enabling reliable closed-loop evaluation of the autonomous driving planning algorithm.

[0095] Please refer to Figure 2 , Figure 2 for Figure 1 A detailed flowchart of step S102.

[0096] like Figure 2 As shown, in some embodiments, the step S102 above, "performing neural rendering processing on the static scene data in the real driving data of the vehicle", may include steps S201 and S202 as shown below.

[0097] Step S201: Perform dynamic object segmentation processing on the real driving data of the vehicle to obtain static scene data after segmenting dynamic objects from the real driving data of the vehicle.

[0098] When performing neural rendering on static scene data in real vehicle driving data, the terminal device can first perform dynamic object segmentation on the real vehicle driving data to obtain static scene data after segmenting dynamic objects from the real vehicle driving data.

[0099] In some embodiments, the terminal device can employ a dynamic object segmentation method based on joint detection of LiDAR point clouds and images to segment dynamic objects in the vehicle's real driving data. For example, a pre-trained PointRCNN model is used to perform 3D object detection on the point cloud of each frame, obtaining bounding boxes for all vehicles, pedestrians, and other objects. Subsequently, SLAM (such as ORB-SLAM3) technology is used to calculate the precise pose of the camera in each frame, and the bounding boxes of all detected dynamic objects are projected back to the global coordinate system according to their motion state (obtained through Kalman filtering tracking), forming a spatio-temporal volume occupied by the dynamic object. All LiDAR points and image pixels within this volume are marked as dynamic regions and excluded from background reconstruction.

[0100] Step S202: Input the static scene data into a preset neural rendering model for neural rendering processing.

[0101] It should be noted that neural rendering models can include neural radiation field models based on neural radiation field (NeRF) technology and 3D Gaussian sputtering models based on 3D Gaussian sputtering (3DGS) technology.

[0102] When performing neural rendering on static scene data from real-world vehicle driving data, terminal devices can use Neural Radiation Field (NeRF) technology to input the segmented static scene data (after dynamic objects are processed) into a Neural Radiation Field model, and then use that model for neural rendering. Alternatively, terminal devices can use 3D Gaussian Sputtering (3DGS) neural rendering technology to input the static scene data into a 3D Gaussian Sputtering model, and then use that model for neural rendering.

[0103] In some embodiments, when a terminal device uses a neural radiation field model or a 3D Gaussian sputtering model to perform neural rendering processing on static scene data, the model parameters can be set according to specific needs to facilitate the neural rendering processing of the static scene data and obtain the reconstructed target static scene data output by the model. For example, the terminal device inputs static scene data (static point cloud and corresponding camera image) after removing dynamic objects into a 3D Gaussian Splatting model. The model parameters are set as follows: the initial point cloud is provided by a static LiDAR point cloud, the upper limit of the number of Gaussians is set to 500,000, and the number of iterations is 30,000. A stochastic gradient descent (SGD) optimizer is used for training to maximize the structural similarity (SSIM) between the reconstructed image and the real image and minimize the L1 photometric error. In this way, the terminal device can obtain the reconstructed target static scene data output by the 3D Gaussian sputtering model after it has completed the neural rendering processing of the static scene data.

[0104] In some embodiments, real-world driving data includes image sequences of real-world driving scenarios. For example, real-world driving data may be an image sequence containing an intersection from the publicly available dataset KITTI (such as 2011_09_26_drive_0005).

[0105] In this case, the semantic information corresponding to the vehicle's actual driving data can include the map vector elements corresponding to the image sequence.

[0106] It should be noted that map vector elements can include lane lines, lane center lines, curbs, traffic signs, etc.

[0107] Please refer to Figure 3 , Figure 3 for Figure 1 A detailed flowchart of step S103.

[0108] like Figure 3 As shown, in some embodiments, the step of "injecting semantic information corresponding to the real driving data of the vehicle into the target static scene data" in step S103 above may include steps S301 and S302 as shown below.

[0109] Step S301: Perform vectorized map construction processing based on the image sequence to obtain the map vector elements corresponding to the image sequence.

[0110] When reconstructing a high-fidelity simulation test scene from real vehicle driving data, the terminal device can, if the real driving data is an image sequence of a real driving scene, infer the image sequence for vector map construction, thereby constructing map vector elements corresponding to the image sequence. Then, the terminal device can inject these map vector elements as semantic information into the target static scene data reconstructed based on the image sequence, thus obtaining the reconstructed high-fidelity test simulation scene data.

[0111] In some embodiments, when the real-world driving data is an image sequence of a real-world driving scene, the terminal device can perform data preparation and preprocessing operations on the image sequence before performing high-fidelity simulation test scene reconstruction on the image sequence, and then perform subsequent inference to construct the map vector elements corresponding to the image sequence. For example, when the real-world driving data is an image sequence containing an intersection, 2011_09_26_drive_0005, from the publicly available KITTI dataset, the terminal device can perform data preparation and preprocessing on the image sequence, downloading synchronized left and right color camera images (0.5MP), Velodyne HDL-64E LiDAR point cloud data, and OXTS pose information provided by IMU / GPS. In this way, the terminal device can obtain the preprocessed image sequence for subsequent processing such as simulation test scene reconstruction.

[0112] Step S302: Perform spatial semantic registration processing on the map vector elements and the target static scene data to inject the map vector elements into the target static scene data.

[0113] When injecting map vector elements into target static scene data, the terminal device can first perform spatial semantic registration between the map vector elements and the target static scene data, and then convert the registered map vector elements according to the specific format requirements of the simulation engine (if any). For example, the registered map vector elements can be converted into a format that the simulation engine can recognize (such as road network files in OpenDRIVE format and semantic maps in OSM format). Then, the simulation engine can load the map vector elements and inject them into the target static scene data to obtain the reconstructed test simulation scene data.

[0114] In some embodiments, when the terminal device reconstructs a high-fidelity simulation test scene from real vehicle driving data (image sequences of real vehicle driving scenes) using a high-fidelity editable scene reconstruction module, after the module reconstructs the target static scene data using a 3D Gaussian sputtering model (obtained by neural rendering of static point clouds and corresponding camera images after removing dynamic objects from the image sequence), it uses another pre-trained network (such as HDMapNet) to infer high-precision map vector elements from the original image sequence, including lane lines, lane center lines, curbs, traffic signs, etc. Then, the module registers these map vector elements with the reconstructed 3D Gaussian scene (target static scene data) (e.g., through GPS coordinates or feature point matching). Subsequently, the registered map vector elements are converted into a format recognizable by the simulation engine (such as OpenDRIVE format road network files and OSM format semantic maps), thereby endowing the target static scene data with semantic information such as drivable areas, lane topology relationships, and traffic rules.

[0115] In this embodiment, the terminal device performs dynamic object segmentation on the vehicle's real driving data to obtain static scene data after dynamic object segmentation. Then, using neural rendering techniques based on Neural Radiation Field (NeRF) or 3D Gaussian Sputtering (3DGS), the segmented static scene data is input into the corresponding model. This model then performs neural rendering on the static scene data to reconstruct the target static scene data. Subsequently, physical and semantic information (such as collision objects, drivable areas, lane topology, traffic rules, etc.) is automatically injected into the target static scene data to obtain the reconstructed test simulation scene data.

[0116] Thus, this embodiment implements a high-fidelity editable scene reconstruction method based on dynamic separation and semantic injection: Neural Radiance Field (NeRF) and 3D Gaussian Sputtering (3DGS) neural rendering technologies are used not only for video generation, but also through an automated process to first separate and remove dynamic objects from the original data, resulting in a clean, high-fidelity static background model. Subsequently, physical and semantic information (such as collision objects, drivable areas, lane topology, and traffic rules) is automatically injected into this model, transforming it from a "visual model" into an "interactive simulation world." This effectively solves the problems of "insufficient realism" and "Sim2Real gap" in traditional mainstream closed-loop testing methods. Neural rendering provides photorealistic visual fidelity, far exceeding traditional game engines, greatly narrowing the visual gap between simulation and reality. Furthermore, it enables "interactive" simulation scene reconstruction: the separation of dynamic objects creates the prerequisite for replacing them with data-driven interactive agents, and the injection of semantic information allows the planning algorithm to understand the road structure and make decisions as if in the real world, rather than operating in a "visual shell."

[0117] In some embodiments, when the terminal device reconstructs a high-fidelity simulation test scene based on real vehicle driving data using the high-fidelity editable scene reconstruction module, it can also support controllable modification of the weather, lighting, traffic density, and road topology of the reconstructed scene based on the scene editing interface provided by the module.

[0118] Please refer to Figure 4 , Figure 4 The following are schematic flowcharts of the steps in some embodiments of the test method for the vehicle autonomous driving planning algorithm provided in this application.

[0119] like Figure 4 As shown, in some embodiments, the testing method for the vehicle autonomous driving planning algorithm provided in this application may further include steps S401 and S402 as shown below.

[0120] Step S401: Obtain the simulation scene modification data corresponding to the edge test cases based on the preset scene editing interface.

[0121] When the terminal device reconstructs a high-fidelity simulation test scene from real vehicle driving data using the high-fidelity editable scene reconstruction module, the module provides a scene editing interface that supports controllable modifications to the reconstructed scene, including weather, lighting, traffic density, and road topology. The terminal device can then proceed through an automated process: first, separating and removing dynamic objects from the original real vehicle driving data to obtain a clean, high-fidelity static background model (target static scene data); then, automatically injecting physical and semantic information into the model to obtain the reconstructed test simulation scene data; and finally, through the scene editing interface provided by the module, receiving simulation scene modification data (used to modify weather, lighting, or road conditions, etc.) input by staff based on the testing requirements of edge test cases.

[0122] Step S402: Adjust the test simulation scenario data based on the simulation scenario modification data to obtain edge test scenario data corresponding to the edge test cases.

[0123] When the terminal device reconstructs a high-fidelity simulation test scenario based on real vehicle driving data through the high-fidelity editable scene reconstruction module, after receiving the simulation scene modification data corresponding to the edge test case based on the scene editing interface, it further adjusts the test simulation scene data reconstructed by the current automated process according to the simulation scene modification data, thereby obtaining the edge test scene data corresponding to the edge test case.

[0124] In some embodiments, the terminal device can utilize the scene editing interface provided by the high-fidelity editable scene reconstruction module, allowing operators to modify the lighting and weather of the reconstructed test simulation scene data using built-in shaders. For example, operators can adjust the ambient occultation map (HDRi) through the scene editing interface to change a daytime scene to dusk. Alternatively, operators can use the scene editing interface to add post-processing effects such as raindrop textures, water reflections, and fog effects to simulate rainy conditions. Furthermore, operators can directly edit OpenDRIVE format road network files or OSM format semantic maps through the scene editing interface to create edge test scenario data corresponding to edge test cases (such as temporarily closing a lane to simulate a construction scenario, thereby creating edge test cases, etc.).

[0125] In this case, step S104 above: conducting simulation tests on the vehicle autonomous driving planning algorithm based on the test simulation scenario data, may further include the following steps:

[0126] The vehicle autonomous driving planning algorithm was simulated and tested based on the edge test scenario data.

[0127] After obtaining the edge test scenario data corresponding to the edge test cases, the terminal device can further conduct closed-loop simulation tests on the vehicle's autonomous driving planning algorithm based on the test simulation scenario data. For example, according to the requirements corresponding to the edge test cases, the original dynamic objects are removed from the edge test simulation scenario data, and surrounding traffic agents and their behavior parameters are generated and configured. Then, the initial simulation state is set: the edge test simulation scenario data, traffic agents and their behavior parameters, and the autonomous driving planning algorithm to be tested are loaded, and the asynchronous-synchronous framework is initialized. Finally, in each simulation step, perception data is generated and input into the planning algorithm. The algorithm outputs control commands, and the simulation engine integrates the vehicle commands and agent behaviors to update the world state.

[0128] In this embodiment, after obtaining the reconstructed test simulation scenario data through the terminal device, the device further receives simulation scenario modification data input by staff based on the test requirements of edge test cases via a scenario editing interface. The device then adjusts the test simulation scenario data reconstructed in the current automated process according to this modification data to obtain edge test scenario data corresponding to the edge test case. Based on this test simulation scenario data, closed-loop simulation testing is performed on the vehicle autonomous driving planning algorithm. Thus, this embodiment can construct a visually high-fidelity, behaviorally human-like, and flexibly editable test environment, enabling comprehensive, reliable, and efficient closed-loop evaluation of the vehicle autonomous driving planning algorithm.

[0129] Please refer to Figure 5 , Figure 5 for Figure 1 A detailed flowchart of step S104.

[0130] like Figure 5 As shown, in some embodiments, step S104 above: performing simulation tests on the vehicle autonomous driving planning algorithm based on the test simulation scenario data may include steps S501 to S503 as shown below.

[0131] Step S501: Obtain virtual perception data corresponding to the test simulation scenario data.

[0132] When the terminal device performs closed-loop simulation testing on the vehicle autonomous driving planning algorithm based on the test simulation scenario data, it can acquire virtual perception data corresponding to the test simulation scenario data in each simulation step.

[0133] In some embodiments, the terminal device can generate virtual perception data corresponding to the test simulation scenario data in each simulation step when performing closed-loop simulation testing of the vehicle autonomous driving planning algorithm based on the test simulation scenario data through the multi-rate synchronous simulation engine module in the autonomous driving planning trajectory closed-loop evaluation system.

[0134] It should be noted that the multi-rate synchronous simulation engine module can be used to coordinate the operation of an autonomous driving trajectory planning closed-loop evaluation system. For example, this module can decouple and synchronize computationally expensive neural rendering (low frame rate) with real-time-critical planning algorithms and physical simulation (high frame rate) through an evaluation-oriented "perception-planning-simulation" multi-rate closed-loop synchronous framework, thereby solving the speed mismatch problem between neural rendering (low frame rate) and vehicle dynamics and planning algorithms (high frame rate). Furthermore, this module / framework includes a virtual sensor model that can directly sample from intermediate representations of neural rendering (such as Gaussian point clouds from 3DGS or radiation fields from NeRF) to generate noisy simulated sensor data (such as LiDAR point clouds / camera images) for the algorithm under test to perceive. In this case, the simulated sensor data is the virtual perception data corresponding to the simulated scenario data.

[0135] In this case, step S501 above may include the following steps:

[0136] When the static scene data is processed by neural rendering based on a preset neural rendering model, virtual perception data corresponding to the test simulation scene data is obtained by sampling the intermediate representation generated by the neural rendering model.

[0137] When the terminal device performs neural rendering processing on static scene data using the aforementioned neural rendering model, it can perform closed-loop simulation testing of the vehicle autonomous driving planning algorithm based on the test simulation scene data in the multi-rate synchronous simulation engine module. In each simulation step, the virtual sensor model within the module directly samples from the intermediate representation (such as Gaussian point cloud of 3DGS or radiation field of NeRF) generated by the neural rendering model to generate noisy simulated sensor data (such as LiDAR point cloud / camera image). This simulated sensor data is then used as virtual perception data corresponding to the test simulation scene data.

[0138] Step S502: Input the virtual perception data into the vehicle autonomous driving planning algorithm for trajectory planning processing to obtain the vehicle autonomous driving control command output by the vehicle autonomous driving planning algorithm.

[0139] After acquiring virtual perception data corresponding to the test simulation scenario data, the terminal device further inputs this virtual perception data into a pre-loaded vehicle autonomous driving planning algorithm. This algorithm then performs autonomous driving trajectory planning and other processing on the virtual perception data, generating vehicle autonomous driving control commands. In this way, the terminal device can obtain the vehicle autonomous driving control commands output by the vehicle autonomous driving planning algorithm.

[0140] Step S503: Based on the vehicle autonomous driving control command and the traffic agent behavior data corresponding to the test simulation scenario data, perform state deduction to obtain world state information.

[0141] After receiving the vehicle autonomous driving control command output by the vehicle autonomous driving planning algorithm, the terminal device further performs state deduction on the vehicle autonomous driving control command and the traffic agent behavior data corresponding to the pre-generated and loaded test simulation scenario data to obtain the world state information at the next moment.

[0142] In some embodiments, during closed-loop simulation operation, the terminal device can input the vehicle autonomous driving control commands output by the vehicle autonomous driving planning algorithm and traffic agent behavior data into a high-precision dynamic model, thereby using the model to deduce the world state information at the next moment.

[0143] In this embodiment, when the terminal device performs closed-loop simulation testing on the vehicle autonomous driving planning algorithm based on the test simulation scenario data, it acquires virtual perception data corresponding to the test simulation scenario data and inputs the virtual perception data into the pre-loaded vehicle autonomous driving planning algorithm. The vehicle autonomous driving planning algorithm performs autonomous driving trajectory planning and other processing on the virtual perception data to generate vehicle autonomous driving control commands. Finally, the vehicle autonomous driving control commands and the traffic agent behavior data corresponding to the pre-generated and loaded test simulation scenario data are used to perform state deduction to obtain the world state information at the next moment.

[0144] This approach effectively addresses the issue of poor input realism in traditional closed-loop simulation testing methods by using a terminal device to perform neural rendering on static scene data based on a preset neural rendering model. This involves sampling the intermediate representation generated by the neural rendering model to obtain virtual perception data corresponding to the test simulation scene data. Specifically, the virtual sensor model provides the planning algorithm under test with raw perception input highly similar to that of a real vehicle, rather than rendered images. This allows the evaluation to cover the entire "perception-decision-control" chain, making the evaluation results more reliable. Furthermore, by adopting a multi-rate closed-loop synchronous framework for "perception-planning-simulation" oriented towards evaluation, the feasibility of closed-loop evaluation can be addressed. This framework solves the engineering challenges of computationally intensive neural rendering, making it difficult to directly apply to simulations, thus enabling high-fidelity closed-loop evaluation in practice.

[0145] In some embodiments, the testing method for the vehicle autonomous driving planning algorithm provided in this application can generate traffic agent behavior data corresponding to the test simulation scenario data through any one of the following methods one and two.

[0146] Method 1: Traffic agent behavior data corresponding to the test simulation scenario data is generated based on a preset generative artificial intelligence model; the generative artificial intelligence model is obtained by learning and training the behavior of traffic participants on real vehicle driving data.

[0147] It should be noted that generative artificial intelligence models can include Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), or Diffusion Models. These models can be pre-trained on large-scale real-world driving data to learn the behavior of traffic participants (such as vehicles and pedestrians).

[0148] After obtaining the reconstructed test simulation scenario data, the terminal device can generate traffic agent behavior data corresponding to the test simulation scenario data through a generative artificial intelligence model.

[0149] In some embodiments, the terminal device can generate and configure surrounding traffic agents and their behavioral parameters through a data-driven interactive agent module in an autonomous driving trajectory planning closed-loop evaluation system. That is, this interactive agent module abandons simple rule models (such as IDM) and uses generative artificial intelligence models trained on large-scale real-world driving data to learn and reproduce the complex, variable, and even irrational behavioral patterns of human drivers. Furthermore, configurable "personality parameters" (such as aggression, caution, and inattention) are introduced for each agent, thereby simulating a continuous spectrum of behaviors from "polite" to "road rage."

[0150] It should be noted that the interactive agent module is primarily used to generate the behavior of surrounding traffic participants, which can then serve as traffic agent behavior data corresponding to the test simulation scenario. This module uses a generative artificial intelligence model, trained on large-scale real driving data, to learn and generate highly human-like and diverse behaviors. Furthermore, this module assigns configurable personality parameters to each agent to simulate various driving styles, ranging from conservative to aggressive, and can be dynamically adjusted according to testing needs.

[0151] Method 2: Based on the behavioral parameter information of dynamic objects in the real driving data of the vehicle, generate traffic agent behavior data corresponding to the test simulation scenario data.

[0152] After obtaining the reconstructed test simulation scenario data, the terminal device can also generate traffic agent behavior data corresponding to the test simulation scenario data based on the behavior parameter information of dynamic objects in the real driving data of the vehicle, according to the test requirements of the test cases.

[0153] In some embodiments, when the terminal device removes original dynamic objects from the test simulation scenario data according to test case requirements, the behavioral parameter information of the original dynamic objects can be retained in a 1:1 ratio according to the test requirements. In this case, the retained behavioral parameter information can be used as the traffic agent behavior data corresponding to the test simulation scenario data.

[0154] In other embodiments, when the terminal device removes the original dynamic objects in the test simulation scenario data according to the test case requirements, it can also delete all dynamic objects and their behavior parameters, and then regenerate and configure new surrounding agents and their behavior parameters according to the test case requirements. For example, it can generate traffic agent behavior data corresponding to the test simulation scenario data based on the above-mentioned generative artificial intelligence model.

[0155] In this embodiment, traffic agent behavior data corresponding to test simulation scenario data is generated by the terminal device based on a generative artificial intelligence model. This realizes interactive traffic agent based on generative artificial intelligence and parameterized persona. The data-driven generative agent can generate rich, unpredictable, and highly human-like behaviors, which can transform the interactive environment faced by autonomous driving algorithms from "idealistic" to "realistic," thereby solving the problem of simple interactive agent behavior in traditional closed-loop simulation testing methods. Furthermore, by generating traffic agent behavior data based on the generative artificial intelligence model by the terminal device, it is possible to actively and efficiently generate a large number of corner cases by adjusting the "personality parameters" of surrounding vehicles, such as testing the vehicle's ability to cope with behaviors such as "forced lane cutting" and "intersection rushing," thereby achieving a systematic evaluation of the robustness of the planning algorithm.

[0156] Furthermore, in this embodiment, the terminal device can retain the behavioral parameter information of the original dynamic objects in the test simulation scenario data at a 1:1 ratio according to the test requirements of the test cases, or delete all dynamic objects and their behavioral parameters and then regenerate and configure new surrounding agents and their behavioral parameters. That is, based on the requirements, the original dynamic vehicle information can be completely retained, partially retained, or completely deleted, or the system can be added and generated according to the requirements, thereby improving the flexibility of closed-loop simulation testing of planning algorithms on test simulation scenario data.

[0157] Please refer to Figure 6 , Figure 6 A flowchart illustrating the steps of the test method for the vehicle autonomous driving planning algorithm provided in this application in some other embodiments.

[0158] like Figure 6 As shown, in some embodiments, the testing method for the vehicle autonomous driving planning algorithm provided in this application may further include steps S601 and S602 as shown below.

[0159] Step S601: Obtain interactive data for simulation testing of the vehicle autonomous driving planning algorithm; the interactive data includes the vehicle autonomous driving control command, the traffic agent behavior data, and the world state information.

[0160] When the terminal device performs simulation testing on the vehicle autonomous driving planning algorithm based on the test simulation scenario data, it can also obtain all the interaction data in the entire closed-loop simulation process, namely, the vehicle autonomous driving control commands generated by the vehicle autonomous driving planning algorithm in each simulation step, traffic agent behavior data, and world state information updated by the integrated commands and agent behavior.

[0161] Step S602: Based on the vehicle autonomous driving control command, the traffic agent behavior data, and the world state information, calculate the multi-dimensional quantitative indicators of the vehicle autonomous driving planning algorithm; the multi-dimensional quantitative indicators include safety indicators, interactivity indicators, and intelligence indicators.

[0162] It should be noted that safety indicators may include collision indicators, TTC indicators, etc. Interactivity indicators may include the number of times other vehicles brake suddenly, merging success rate, comfort level (Jerk), etc. Intelligence indicators may include traffic efficiency, decision compliance and social etiquette compliance, etc.

[0163] After acquiring all the interactive data for simulating and testing the vehicle autonomous driving planning algorithm, the terminal device further calculates the multi-dimensional quantitative indicators of the vehicle autonomous driving planning algorithm, including safety indicators, interactivity indicators, and intelligence indicators, based on the vehicle autonomous driving control commands, traffic agent behavior data, and world state information.

[0164] In some embodiments, the terminal device can use the loop-in-the-loop evaluation and quantitative analysis module of the planning algorithm in the autonomous driving planning trajectory closed-loop evaluation system to perform closed-loop testing on the vehicle's autonomous driving planning algorithm using test simulation scenario data and output the evaluation results. Specifically, this module starts from a certain moment in the real log, handing over control of the vehicle to the autonomous driving planning algorithm under evaluation. The planning algorithm makes decisions based on perception data input from virtual sensors and outputs autonomous driving control commands. These commands, along with the behavior of the interactive agent (traffic agent behavior data), are input into a high-precision vehicle dynamics model to predict the world state at the next moment. Furthermore, this module defines and calculates a multi-dimensional quantitative indicator system, including safety indicators (such as the number of collisions and minimum safe distance (TTC), interactivity indicators (such as the number of times other vehicles brake suddenly, lane change success rate, and acceleration change rate Jerk), and intelligence indicators (such as traffic efficiency and compliance with traffic rules and social etiquette).

[0165] It should be noted that the multidimensional quantitative indicator system can be a multidimensional quantitative indicator system that integrates security, efficiency and social etiquette. This indicator system defines a set of closed-loop quantitative indicators that go beyond traditional open-loop errors (such as L2 distance) and are specifically used to evaluate interaction performance, including the aforementioned security indicators, interactivity indicators and intelligence indicators.

[0166] In this embodiment, interactive data for simulating and testing the vehicle autonomous driving planning algorithm is acquired through a terminal device. Based on this interactive data, multi-dimensional quantitative indicators of the vehicle autonomous driving planning algorithm are calculated: safety indicators, interactivity indicators, and intelligence indicators. Thus, by integrating a multi-dimensional quantitative indicator system that considers safety, efficiency, and social etiquette, the dynamic impact of the vehicle's behavior on the surrounding environment can be quantitatively evaluated. This accurately reveals algorithmic defects in some planning algorithms, such as perfect trajectories that lead to traffic conflicts, solving the problem of traditional open-loop evaluation methods neglecting interactive feedback. Furthermore, by evaluating the social etiquette compliance of the planning algorithm, it can guide the algorithm not only to pursue safety and efficiency but also to learn from human driving tacit understanding, improving the comfort and acceptance of passengers and other traffic participants. In other words, it can guide the vehicle autonomous driving planning algorithm towards a more human-centered development.

[0167] Next, a complete embodiment of the testing method for the vehicle autonomous driving planning algorithm provided in this application is presented.

[0168] In a complete embodiment of the testing method for the vehicle autonomous driving planning algorithm provided in this application, the testing method for the vehicle autonomous driving planning algorithm provided in this application can be executed by the aforementioned autonomous driving planning trajectory closed-loop evaluation system, thereby constructing a visually high-fidelity, behaviorally humanized, and flexibly editable testing environment, and thus realizing a comprehensive, reliable, and efficient closed-loop evaluation of the vehicle autonomous driving planning algorithm in multiple dimensions such as safety, efficiency, comfort, and intelligence.

[0169] It should be noted that the autonomous driving trajectory planning closed-loop evaluation system can be a system that integrates neural rendering and interactive agents, and includes the following functional modules:

[0170] High-fidelity editable scene reconstruction module: This module processes multimodal data collected from the real world (including multi-camera video, LiDAR point clouds, and pose information). Using neural rendering techniques based on Neural Radiation Field (NeRF) or 3D Gaussian Sputtering (3DGS), it automatically reconstructs photorealistic dynamic 3D scenes. The core of this module lies in its dynamic object separation and semantic injection capabilities. It can identify, segment, and remove dynamic objects from the original data to obtain a clean static background model, and automatically generate corresponding physical and semantic information for this model, such as colliders, drivable areas, lane topology, and traffic rules. Simultaneously, this module provides a scene editing interface, supporting controllable modifications to the reconstructed scene's weather, lighting, traffic density, and road topology.

[0171] The data-driven interactive agent module is used to generate the behavior of surrounding traffic participants (vehicles, pedestrians). This module uses generative artificial intelligence models (such as Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), or Diffusion Models) trained on large-scale real driving data to learn and generate highly human-like and diverse behaviors. Each agent is assigned configurable personality parameters (including aggression, caution, and attention level) to simulate various driving styles, ranging from conservative to aggressive, and can be dynamically adjusted according to testing needs.

[0172] Multi-rate synchronous simulation engine module: Used to coordinate the operation of the entire system. This module designs an asynchronous scheduling framework to solve the speed mismatch problem between neural rendering (low frame rate) and vehicle dynamics and planning algorithms (high frame rate); it includes a virtual sensor model that can directly sample from intermediate representations of neural rendering (such as Gaussian point clouds of 3DGS or radiation fields of NeRF) to generate simulated, noisy camera images and LiDAR point cloud data as perceptual inputs to the planning algorithm to be evaluated.

[0173] The planning algorithm in the environmental assessment and quantitative analysis module is used to perform closed-loop testing and output evaluation results. This module starts from a specific moment in the real log and hands over control of the vehicle to the planning algorithm under evaluation. The planning algorithm makes decisions based on virtual sensor inputs and outputs control commands. These commands, along with the behavior of the interactive agent, are input into a high-precision vehicle dynamics model to predict the world state at the next moment. This module defines and calculates a multi-dimensional quantitative index system, including safety indicators (such as the number of collisions and minimum safe distance (TTC), interactivity indicators (such as the number of times other vehicles brake suddenly, lane change success rate, and acceleration change rate Jerk), and intelligence indicators (such as traffic efficiency and compliance with traffic rules and social etiquette).

[0174] Please refer to Figure 7 , Figure 7 The overall system flowchart involved in a complete embodiment of the test method for the vehicle autonomous driving planning algorithm provided in this application.

[0175] like Figure 7 As shown, in some embodiments, when the autonomous driving planning trajectory closed-loop evaluation system applies the test method of the vehicle autonomous driving planning algorithm provided in this application, it may include the following steps S1 to S6.

[0176] Step S1: Data preprocessing and scene reconstruction: Obtain a real-world driving dataset, and use the high-fidelity editable scene reconstruction module to separate dynamic objects, reconstruct static scenes, and inject semantic and physical information.

[0177] Step S2: Interactive Agent Configuration and Scene Editing: Based on the test case requirements, remove the original dynamic objects in the reconstructed scene, and generate and configure the surrounding traffic agents and their behavior parameters through the data-driven interactive agent module.

[0178] In some embodiments, the system can modify weather, lighting, or road conditions through a scene editing interface.

[0179] Step S3: Initialization and Synchronization: Set the initial state of the simulation. The multi-rate synchronous simulation engine module loads the scene, agent and planning algorithm to be tested, and initializes the asynchronous synchronization framework.

[0180] Step S4: Closed-loop simulation operation: In each simulation step, the virtual sensor model generates perception data and inputs it into the planning algorithm. The algorithm outputs control commands, and the simulation engine integrates the vehicle commands and agent behavior to update the world state.

[0181] Step S5: Data Recording and Index Calculation: During the simulation, the planning algorithm records all interactive data in real time in the environmental assessment and quantitative analysis module, and calculates multi-dimensional quantitative indicators after the test is completed.

[0182] Step S6: Results Analysis and Test Reproduction: Generate an evaluation report and provide complete test logs for accurate reproduction and in-depth analysis of any failure cases.

[0183] In this embodiment, an autonomous driving trajectory planning closed-loop evaluation system integrating neural rendering and interactive agents is constructed to create a visually high-fidelity, behaviorally human-like, and flexibly editable testing environment. This environment enables comprehensive, reliable, and efficient closed-loop evaluation of vehicle autonomous driving planning algorithms across multiple dimensions, including safety, efficiency, comfort, and intelligence. It successfully transforms cutting-edge neural rendering and generative AI technologies from demos into engineering tools serving the core objective of autonomous driving testing. Through a series of system-level innovative designs (dynamic separation, semantic injection, multi-rate synchronization, and virtual sensing), an evaluation infrastructure is built that possesses visual effects that are infinitely close to reality, high intelligence, and variable behavioral interaction, while also being able to operate efficiently and on a large scale. Furthermore, compared to the aforementioned mainstream testing methods, this embodiment provides a completely new algorithm evaluation paradigm, capable of assessing the intelligence and emotional intelligence of autonomous driving systems with unprecedented efficiency, fidelity, and depth. This is a key guarantee for promoting the maturity and implementation of advanced autonomous driving technologies.

[0184] Please refer to Figure 8 This application also provides a testing device for a vehicle autonomous driving planning algorithm, which can implement the above-mentioned testing method for the vehicle autonomous driving planning algorithm.

[0185] like Figure 8 As shown, the testing apparatus for the vehicle autonomous driving planning algorithm provided in this application may include:

[0186] The acquisition module is used to acquire real-time driving data of the vehicle.

[0187] The neural rendering module is used to perform neural rendering processing on the static scene data in the real driving data of the vehicle to obtain the reconstructed target static scene data.

[0188] The scene reconstruction module is used to inject semantic information corresponding to the real driving data of the vehicle into the target static scene data to obtain test simulation scene data;

[0189] The closed-loop testing module is used to perform simulation testing on the vehicle autonomous driving planning algorithm based on the test simulation scenario data.

[0190] In some embodiments, the closed-loop testing module is further configured to acquire virtual perception data corresponding to the test simulation scenario data; input the virtual perception data into a vehicle autonomous driving planning algorithm for trajectory planning processing to obtain the vehicle autonomous driving control command output by the vehicle autonomous driving planning algorithm; and perform state deduction based on the vehicle autonomous driving control command and the traffic agent behavior data corresponding to the test simulation scenario data to obtain world state information.

[0191] In some embodiments, the testing apparatus for the vehicle autonomous driving planning algorithm provided in this application may further include:

[0192] The agent module is used to generate traffic agent behavior data corresponding to the test simulation scenario data based on a preset generative artificial intelligence model; the generative artificial intelligence model is obtained by learning and training the behavior of traffic participants on real vehicle driving data; and, based on the behavior parameter information of dynamic objects in the real vehicle driving data, the traffic agent behavior data corresponding to the test simulation scenario data is generated.

[0193] In some embodiments, the closed-loop testing module is further configured to perform sampling processing on the intermediate representation generated by the neural rendering model to obtain virtual perception data corresponding to the test simulation scene data when the static scene data is processed by neural rendering based on a preset neural rendering model.

[0194] In some embodiments, the closed-loop testing module is further configured to acquire interactive data for simulating and testing the vehicle autonomous driving planning algorithm; the interactive data includes the vehicle autonomous driving control command, the traffic agent behavior data, and the world state information; and, based on the vehicle autonomous driving control command, the traffic agent behavior data, and the world state information, calculate multi-dimensional quantitative indicators of the vehicle autonomous driving planning algorithm; the multi-dimensional quantitative indicators include safety indicators, interactivity indicators, and intelligence indicators.

[0195] In some embodiments, the neural rendering module is further configured to perform dynamic object segmentation processing on the real vehicle driving data to obtain static scene data after segmentation of dynamic objects in the real vehicle driving data; and to input the static scene data into a preset neural rendering model for neural rendering processing.

[0196] In some embodiments, the real-world driving data of the vehicle includes image sequences of real-world driving scenarios of the vehicle, and the semantic information includes map vector elements corresponding to the image sequences;

[0197] The scene reconstruction module is further configured to perform vectorized map construction processing based on the image sequence to obtain map vector elements corresponding to the image sequence; and to perform spatial semantic registration processing between the map vector elements and the target static scene data to inject the map vector elements into the target static scene data.

[0198] In some embodiments, the scene reconstruction module is further configured to obtain simulation scene modification data corresponding to the edge test case based on a preset scene editing interface; and to adjust the test simulation scene data based on the simulation scene modification data to obtain edge test scene data corresponding to the edge test case.

[0199] The scene reconstruction module is also used to perform simulation tests on the vehicle autonomous driving planning algorithm based on the edge test scene data.

[0200] It should be noted that the specific implementation of the test device for the vehicle autonomous driving planning algorithm provided in this application is basically the same as the specific implementation of the test method for the vehicle autonomous driving planning algorithm described above, and will not be repeated here.

[0201] Please see Figure 9 This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-mentioned vehicle autonomous driving planning algorithm.

[0202] In some embodiments, the electronic device may be any smart terminal such as an in-vehicle terminal, an in-vehicle hardware platform (e.g., an in-vehicle computer), a tablet computer, a smartphone, or a wearable device; or, the electronic device may be a vehicle including a memory and a processor.

[0203] like Figure 9 As shown, the electronic device provided in this application may include:

[0204] The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to achieve the technical solution provided in this application.

[0205] The memory 902 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called by the processor 901 to execute the test method for the vehicle autonomous driving planning algorithm of this application.

[0206] The input / output interface 903 is used to implement information input and output;

[0207] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0208] Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904);

[0209] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0210] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described vehicle autonomous driving planning algorithm as a test method.

[0211] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0212] This application also provides a computer program product, including a computer program, the steps of which, when executed by a processor, are basically the same as the specific embodiments of the test method for the above-described vehicle autonomous driving planning algorithm, and will not be repeated here.

[0213] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of this application, and do not constitute a limitation on the technical solutions provided in this application. Those skilled in the art will know that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0214] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0215] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0216] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0217] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0218] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0219] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0220] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0221] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0222] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0223] The preferred embodiments of this application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and spirit of this application shall be within the scope of the claims.

Claims

1. A testing method for a vehicle autonomous driving planning algorithm, characterized in that, The method includes: Acquire real-world driving data of the vehicle; the real-world driving data of the vehicle includes image sequences of real-world driving scenarios of the vehicle. The vehicle's real driving data is subjected to dynamic object segmentation processing to obtain static scene data after segmentation of dynamic objects in the vehicle's real driving data. The static scene data is then input into a preset neural rendering model for neural rendering processing to obtain reconstructed target static scene data. The neural rendering model includes a neural radiation field model and a 3D Gaussian sputtering model. The image sequence is inferred and then processed into a vectorized map to obtain the map vector elements corresponding to the image sequence. Spatial semantic registration is performed between the map vector elements and the target static scene data to inject the map vector elements into the target static scene data, thereby obtaining test simulation scene data. The vehicle autonomous driving planning algorithm was simulated and tested based on the test simulation scenario data.

2. The method according to claim 1, characterized in that, The simulation test of the vehicle autonomous driving planning algorithm based on the test simulation scenario data includes: Obtain virtual perception data corresponding to the test simulation scenario data; The virtual perception data is input into the vehicle autonomous driving planning algorithm for trajectory planning processing to obtain the vehicle autonomous driving control command output by the vehicle autonomous driving planning algorithm. Based on the vehicle autonomous driving control commands and the traffic agent behavior data corresponding to the test simulation scenario data, state deduction is performed to obtain world state information.

3. The method according to claim 2, characterized in that, The method further includes at least one of the following: Traffic agent behavior data corresponding to the test simulation scenario data is generated based on a preset generative artificial intelligence model; the generative artificial intelligence model is obtained by learning and training the behavior of traffic participants on real vehicle driving data; Based on the behavioral parameter information of dynamic objects in the real driving data of the vehicle, traffic agent behavior data corresponding to the test simulation scenario data is generated.

4. The method according to claim 2, characterized in that, The acquisition of virtual perception data corresponding to the test simulation scenario data includes: When the static scene data is processed by neural rendering based on a preset neural rendering model, virtual perception data corresponding to the test simulation scene data is obtained by sampling the intermediate representation generated by the neural rendering model.

5. The method according to claim 2, characterized in that, The method further includes: Acquire interactive data for simulation testing of the vehicle autonomous driving planning algorithm; the interactive data includes the vehicle autonomous driving control commands, the traffic agent behavior data, and the world state information; Based on the vehicle autonomous driving control commands, the traffic agent behavior data, and the world state information, multi-dimensional quantitative indicators of the vehicle autonomous driving planning algorithm are calculated; the multi-dimensional quantitative indicators include safety indicators, interactivity indicators, and intelligence indicators.

6. The method according to claim 1, characterized in that, The method further includes: The simulation scenario modification data corresponding to the edge test cases is obtained based on the preset scenario editing interface; Based on the simulation scenario modification data, the test simulation scenario data is adjusted to obtain edge test scenario data corresponding to the edge test cases; The simulation test of the vehicle autonomous driving planning algorithm based on the test simulation scenario data includes: The vehicle autonomous driving planning algorithm was simulated and tested based on the edge test scenario data.

7. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the test method of the vehicle autonomous driving planning algorithm according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements a test method for the vehicle autonomous driving planning algorithm as described in any one of claims 1 to 6.

9. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements a test method for the vehicle autonomous driving planning algorithm as described in any one of claims 1 to 6.

Images