Simulation testing methods for intelligent driving algorithms

By combining real data and simulation data generated from a 3D Gaussian model, a sensor perception perspective is constructed, which solves the problem of consistent perception input in intelligent driving simulation testing, realizes the synchronous update of perception input and vehicle status, and improves the realism and consistency of the test.

CN123067196APending Publication Date: 2026-11-10CHERY AUTOMOBILE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611406809.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-10
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

In existing intelligent driving simulation tests, it is difficult to maintain the consistency of perception input after changes in vehicle behavior, which leads to deviation between perception results and actual pose, affecting the confidence of test results and the effectiveness of closed-loop verification.

Method used

By combining real data with simulation data generated from a 3D Gaussian model, a sensor perception perspective is constructed. Sensor data is updated through control commands, forming a closed-loop simulation test process to ensure that the perception input is synchronized with the vehicle status.

Benefits of technology

It improves the realism, real-time performance, and interactive consistency of intelligent driving algorithm simulation tests, and enhances the effectiveness of test reports and the accuracy of algorithm performance evaluation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123067196A_ABST
    Figure CN123067196A_ABST
Patent Text Reader

Abstract

This application relates to the field of intelligent driving simulation testing, and in particular to a simulation testing method for intelligent driving algorithms. Addressing the problems of existing methods that struggle to adapt to changes in perspective after vehicle control during real-world data playback, and the difficulty in balancing realism and real-time performance in scene reconstruction, leading to inconsistencies between perceived input and vehicle state, and distortion in closed-loop verification, this application uses a scene representation capable of generating multi-sensor perceived perspective data as a carrier. It acquires sensor data including real road data and / or simulation data generated based on a 3D Gaussian model of the road segment, inputs this data into the algorithm under test to obtain control commands, and continuously updates the sensor data according to the control commands. The test is executed cyclically until a termination condition is met, at which point a test report is output. This method unifies the input links of real-world data acquisition and simulation, improves scene realism, sensor perspective update capabilities, and closed-loop test consistency, mitigates data lag issues after changes in vehicle behavior, and enhances the accuracy and reusability of test results.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of intelligent driving simulation testing technology, and in particular to a simulation testing method for an intelligent driving algorithm. Background Technology

[0002] In the development of intelligent driving systems, the recharge simulation typically uses the playback of actual sensor data for algorithm verification.

[0003] However, the playback of real-world data is mostly open-loop verification under fixed trajectories. After the vehicle's behavior changes, the input data is difficult to update synchronously, which can easily cause the perception results to deviate from the actual position and pose.

[0004] Therefore, how to maintain the consistency of perception input after changes in vehicle behavior in intelligent driving simulation has become an urgent technical problem to be solved. Summary of the Invention

[0005] This application provides a simulation testing method for intelligent driving algorithms to maintain the consistency of perception input after changes in vehicle behavior.

[0006] In a first aspect, embodiments of this application provide a simulation testing method for an intelligent driving algorithm, comprising:

[0007] S1. Acquire vehicle sensor data. Sensor data includes at least one of real data and simulation data. Real data is collected by multiple sensors when the vehicle is in a real road environment. Simulation data is obtained by simulating the perception perspective of multiple sensors based on a three-dimensional Gaussian model of the road segment where the vehicle is located.

[0008] S2. Input the sensor data into the intelligent driving algorithm to be tested to obtain control commands;

[0009] S3. Update sensor data based on control commands;

[0010] S4. If the preset test termination condition is not triggered, return to execute S2;

[0011] S5. Generate and output a test report for the intelligent driving algorithm to be tested.

[0012] In one possible embodiment, acquiring vehicle sensor data includes:

[0013] Obtain real data;

[0014] Based on real data, a three-dimensional Gaussian model of the road segment where the vehicle is located is constructed;

[0015] Based on the three-dimensional Gaussian model, the simulation data is determined.

[0016] In one possible implementation, a three-dimensional Gaussian model of the road segment where the vehicle is located is constructed based on real data, including:

[0017] Build an initial model based on real data;

[0018] Based on the preset loss function, the initial model is trained until convergence, resulting in a three-dimensional Gaussian model of the road segment where the vehicle is located.

[0019] In one possible implementation, an initial model is constructed based on real data, including:

[0020] Based on real data, multiple Gaussian points and image planes were determined;

[0021] Multiple Gaussian points are projected onto the image plane and rendered to obtain the initial model.

[0022] In one possible embodiment, the Gaussian point includes at least one parameter among location, covariance matrix, opacity, and spherical harmonic coefficients.

[0023] In one possible embodiment, the loss function As shown in the formula below:

[0024]

[0025] in, , , These represent the weight parameters of the three loss terms. To predict the image, For real images, To calculate the absolute value of the differences between all pixels, It is a structural similarity index. This is a regularization term for opacity.

[0026] In one possible embodiment, updating sensor data based on control commands includes:

[0027] Based on control commands, determine the vehicle's position and orientation information;

[0028] The sensor data is updated based on pose information and a 3D Gaussian model.

[0029] In one possible embodiment, updating sensor data based on pose information includes:

[0030] The pose information is input into the three-dimensional Gaussian model to generate the corresponding sensor image simulation data;

[0031] Update sensor data based on sensor image simulation data.

[0032] In one possible embodiment, the test termination conditions include at least one of the following: reaching a preset test duration, a vehicle collision event, and the vehicle deviating from the road.

[0033] In one possible embodiment, it also includes:

[0034] Extract the dynamic trajectory of dynamic objects from real data;

[0035] Construct a three-dimensional dynamic model based on dynamic trajectory;

[0036] The three-dimensional dynamic model is superimposed onto the three-dimensional Gaussian model.

[0037] In one possible embodiment, it also includes:

[0038] Transfer the surface texture features of dynamic objects in real data to a 3D dynamic model.

[0039] In one possible embodiment, the test report includes at least one of trajectory tracking error, safety metrics, comfort metrics, and perceived consistency.

[0040] In one possible embodiment, before inputting the pose information into the 3D Gaussian model, the method further includes:

[0041] Obtain supplementary data, which consists of actual data collected by multiple sensors of the vehicle during the test.

[0042] The 3D Gaussian model is updated based on the supplementary data.

[0043] In one possible embodiment, the sensor includes at least one of a camera, a lidar, and a millimeter-wave radar, and the sensor data includes at least one of image data, point cloud data, and radar data.

[0044] In one possible embodiment, the simulation data also includes anomalous data generated when at least one of the multiple sensors is in a faulty state or when preset noise is introduced.

[0045] In one possible embodiment, it also includes:

[0046] Based on the test report, a graphical interface is generated and output, which includes the vehicle's simulated driving screen, driving trajectory, and quantitative curves of control commands.

[0047] Secondly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0048] The memory stores the instructions that the computer executes;

[0049] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0050] Thirdly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0051] By acquiring sensor data, including real data and / or sensor data obtained from simulations based on 3D Gaussian models of the road sections where the vehicle is located, and inputting the sensor data into the intelligent driving algorithm under test to obtain control commands, and then updating the sensor data based on the control commands and cyclically executing the test, the algorithm under test can always obtain perceptual input corresponding to the current control behavior as the vehicle state continues to change. This improves the realism, real-time performance, and interaction consistency of intelligent driving algorithm simulation tests, and enhances the effectiveness of the test report in evaluating algorithm performance. Attached Figure Description

[0052] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0053] Figure 1 This is a simulation test diagram of the intelligent driving algorithm provided in the embodiments of this application;

[0054] Figure 2 A flowchart illustrating the simulation testing method for the intelligent driving algorithm provided in this application embodiment. Figure 1 ;

[0055] Figure 3 A flowchart illustrating the simulation testing method for the intelligent driving algorithm provided in this application embodiment. Figure 2 ;

[0056] Figure 4 A schematic diagram of the structure of a simulation testing device for an intelligent driving algorithm provided in an embodiment of this application;

[0057] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0058] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0059] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0060] Testing of intelligent driving algorithms involves the joint verification of vehicle perception, decision-making, and control, and is commonly found in the backfeed simulation platform during the development phase. Such platforms typically access data from multiple sources of sensors, such as cameras, LiDAR, and positioning units, and combine them with vehicle trajectory, scene models, and the algorithm under test to form a simulation test chain for cyclical operation.

[0061] Figure 1 This is a simulation test diagram of the intelligent driving algorithm provided in the embodiments of this application, such as... Figure 1 As shown, it includes:

[0062] 10. Regarding the decisions made by intelligent driving algorithms, in existing technologies, image frame data from collected real-world data is input into the intelligent driving algorithm under test to obtain the decisions made by the algorithm, such as... Figure 1 As shown by the dashed line, the vehicle changed lanes to the other side of the path following the aforementioned decision.

[0063] 11. Decision-making based on real data, such as Figure 1 As shown in the solid line vehicle image, the data in the next frame is collected from multiple sensors on the solid line vehicle.

[0064] Therefore, inputting real data into the autonomous driving algorithm under test during the continued testing process will disrupt the decision-making chain during the testing process, further leading to a lower confidence level in the test results.

[0065] Current intelligent driving simulations often involve directly playing back images, point clouds, and pose information collected in real road environments to the algorithm under test, and then evaluating the results after the algorithm outputs control commands.

[0066] It is mainly applicable to open-loop testing under fixed trajectory. Once the vehicle changes lanes, avoids obstacles, or adjusts its steering due to the control result of the algorithm under test, the subsequent input sensor data still remains at the perspective corresponding to the original acquisition trajectory, and the perception result gradually deviates from the current position of the vehicle.

[0067] In the above situation, the environmental information obtained by the perception module is inconsistent with the vehicle state after the control is executed. The decision chain in the test process is broken, and collision judgment, trajectory evaluation and comfort analysis will be affected. The same piece of real-world data is also difficult to use repeatedly for closed-loop verification of different algorithm versions.

[0068] To address the aforementioned technical challenges, a scene representation capable of generating multi-sensor perception data is employed as the test platform. Real-world road data and simulation-generated data are integrated into the sensor data input chain, and sensor data is synchronously updated after the algorithm under test outputs control commands. This ensures the algorithm continuously receives perception inputs corresponding to the current vehicle state during cyclic testing. The simulation test is organized around a process of acquiring sensor data, inputting it into the algorithm under test, updating sensor data according to control commands, repeating the test until termination conditions are met, and outputting a test report. This approach effectively bridges the gap between scene realism, real-time update capability, and closed-loop consistency, improving the issue of input data lag after changes in vehicle behavior.

[0069] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0070] Figure 2 A flowchart illustrating the simulation testing method for the intelligent driving algorithm provided in this application embodiment. Figure 1 The methods described above in the embodiments of this application can be applied to any electronic device. For example... Figure 2 As shown, the method includes:

[0071] S1. Acquire vehicle sensor data, which includes at least one of real data and simulation data. Real data is collected by multiple sensors when the vehicle is in a real road environment. Simulation data is obtained by simulating the perception perspective of multiple sensors based on a three-dimensional Gaussian model of the road segment where the vehicle is located.

[0072] Sensor data serves as the input data set for the intelligent driving algorithm under test, covering at least one data source from both real-world acquisition and simulation generation. Real-world data corresponds to scene observations collected by multiple sensors when the vehicle is operating in a real road environment. Simulation data is obtained by simulating the scene on a 3D Gaussian model according to the perception perspective of each sensor. The 3D Gaussian model is used to characterize the spatial scene structure and appearance information of the road segment where the vehicle is located, while the perception perspective characterizes the observation direction, field of view, and sampling time of each sensor relative to the vehicle and the scene.

[0073] In this application, the executing entity can be a test server, an onboard computing unit, or a test system composed of both deployed in the recharge simulation platform. Specifically, a data index corresponding to the test scenario is first established, recording at least the original acquisition timeline, vehicle reference trajectory, and scenario road segment identifiers. For the real data link, the system reads the scenario observation results corresponding to the current test moment from the pre-stored dataset and completes data alignment based on a unified time reference. When different data have different sampling frequencies, interpolation, alignment, or resampling methods can be used to generate input data packets for the same simulation moment. For the simulation data link, a three-dimensional Gaussian model corresponding to the road segment where the vehicle is located can be obtained or constructed in advance, and the perception perspectives of multiple sensors can be simulated according to the vehicle state at the start of the test to obtain the corresponding simulation data. If this round of testing uses a mixture of real and simulation data input, the still valid sensor data can be directly used, while data that can no longer correspond to the current state can be switched to simulation data generated based on the three-dimensional Gaussian model, and a unified time label and coordinate reference are retained in the input data packet. Based on the above analysis, this step integrates the real road data collection results with the scene representation that can regenerate perspective data as the state changes into the input, providing a data foundation related to the current state of the vehicle for subsequent closed-loop testing, so that the test starting point is no longer limited to a single observation sequence on a fixed collection trajectory.

[0074] S2. Input the sensor data into the intelligent driving algorithm under test to obtain control commands. The intelligent driving algorithm is the algorithm object being tested and verified. Its input is the sensor data packet formed in S101, and its output is the control commands acting on the vehicle's motion state. The control commands may include information characterizing the vehicle operation results given by the algorithm under test under the current environmental perception results.

[0075] In this application, after receiving the sensor data generated by S101, the test platform can perform adaptation processing according to the interface specification of the intelligent driving algorithm under test, and send the processed data to the algorithm under test. The algorithm under test outputs corresponding control commands based on the current input data. To ensure the timing consistency of the closed-loop iteration, this step is usually run with a fixed simulation step size, with each simulation step corresponding to one sensor input and one control output. When the algorithm under test outputs multiple candidate control results, the platform can select the currently effective control command according to preset rules; when the algorithm does not output a valid control result, it can record the abnormal state and write it into the subsequent test report. Based on the above analysis, this step directly couples the scene input to the algorithm processing link, so that each round of control results is based on the current perception input, forming the driving source for subsequent state updates.

[0076] S3. Update sensor data based on control commands. This step feeds back the control results output by the algorithm under test at the current moment to the test environment, and generates the next round of sensor inputs corresponding to the new vehicle state, thus forming a closed-loop simulation. When updating sensor data, the relevant states in the test environment can be advanced according to the control commands, and at least one of the real data and simulation data can be updated accordingly.

[0077] In practice, the testing platform updates the vehicle state corresponding to the current test cycle based on control commands and determines the acquisition method of sensor data at the next moment after the state update. If the vehicle state at that moment still matches the real data, the corresponding data can continue to be read from the real data cache; if the vehicle state changes and the real acquisition result no longer represents the current state, the simulation data generation link corresponding to that road segment is called to update the relevant sensor data. If the input in this round adopts a mixed form of real data and simulation data, the mismatched sensor channels can be replaced with simulation data, while other real channels that still match the current state are retained, so as to maintain closed-loop consistency while ensuring the realism of the scenario. The updated sensor data is rewritten into the input buffer, with the current cycle number, vehicle state quantity, and data source identifier appended, to prepare for the next round of algorithm calls. Based on the above analysis, this step solves the problem that the input data does not change when the control result changes in the fixed trajectory playback method by feeding back the control output as a new test input, so that perception, decision-making, and control maintain the correspondence between the previous and subsequent steps in the test cycle.

[0078] S4. If the preset test termination condition is not triggered, return to execute S2. The test termination condition is used to limit the end time of the closed-loop simulation run, and can be related to at least the test duration, number of loops, vehicle status, task completion, or test result event. Returning to execute S2 means that the sensor data updated in S3 will continue to be input into the intelligent driving algorithm under test, and the control output and scene update will be repeatedly executed until the termination criterion is met. This loop mechanism allows the same road segment scenario to be reused under different algorithm versions, different control strategies, and different disturbance conditions, and continuously outputs perception input consistent with the current state of the vehicle.

[0079] In this application, the platform executes termination condition determination logic after each round of S3. During the determination, the platform reads the vehicle status log, event detection results, and algorithm output status codes. When the determination result indicates that the termination condition has not been triggered, the system maintains the current scene context and directly calls the input interface of S2, using the sensor data generated and cached in the previous round of S3 as the input for the next round. When multiple sensors operate at different frequencies, the platform can advance according to the master clock and perform asynchronous frame interpolation in each channel, ensuring that the algorithm under test receives complete input in each control cycle. During the loop, the platform continuously records the input data summary, control commands, vehicle status, key event flags, and data source identifiers for each round, forming a complete test process log. Based on the above analysis, this step maintains the repetitive execution relationship between S2 and S3, preventing the test from remaining at a single frame or single segment open-loop evaluation, but rather forming a continuous closed-loop verification process under a unified timing sequence.

[0080] S5. Generate and output a test report for the intelligent driving algorithm to be tested.

[0081] The test report is used to summarize and output the algorithm's performance throughout the closed-loop simulation test process. Its content is generated based on multiple rounds of sensor inputs, control commands, vehicle state update results, and termination events.

[0082] In practice, the platform stops loops S2 through S4 after triggering the termination condition and extracts complete runtime data from the test log. First, it reconstructs the vehicle trajectory, speed curve, control sequence, and key event sequence of the test according to the timeline. Then, it performs frame-by-frame retrieval and statistics on relevant events. If the output of the algorithm under test contains intermediate results, the platform can also correlate these intermediate results with the input data at the corresponding time points to generate diagnostic segments for locating problems. Finally, the platform organizes the statistical results, event list, keyframe index, and evaluation conclusions into a structured test report and outputs it as a file, database record, or interface display. Based on the above analysis, this step unifies the runtime results of each round of closed-loop testing into a traceable result set, allowing comparisons between the same road segment, the same algorithm version, or different algorithm versions based on the same indicator system.

[0083] Based on the above analysis, this application provides a simulation testing method for intelligent driving algorithms, including acquiring vehicle sensor data, inputting the sensor data into the intelligent driving algorithm under test to obtain control commands, updating sensor data based on the control commands, returning to the sensor data input step when a preset test termination condition is not triggered, and generating and outputting a test report for the intelligent driving algorithm under test. In this application, real data and simulation data generated based on a three-dimensional Gaussian model are uniformly connected to the data input link of the algorithm under test. After the algorithm outputs control commands, subsequent sensor data is updated based on the control results, ensuring that each subsequent round of algorithm execution corresponds to the vehicle state after the previous round of control results. Therefore, the perception input, algorithm decision-making, and vehicle state update in the test link maintain a continuous correspondence, the input mismatch problem after fixed trajectory playback and vehicle state changes is addressed, and scene realism and closed-loop iterability are balanced within the same test process. It should be understood that the above examples are merely illustrative and not limiting. Any implementation based on the technical features defined in the original claims and combined with equivalent variations thereof should fall within the protection scope of this application.

[0084] Based on the foregoing embodiments, further acquiring vehicle sensor data includes:

[0085] Obtain real data;

[0086] Based on real data, a three-dimensional Gaussian model of the road segment where the vehicle is located is constructed;

[0087] Based on the three-dimensional Gaussian model, the simulation data is determined.

[0088] In practical implementation, when acquiring real data, raw data from multiple sensors, vehicle pose data, and timestamp information can be collected simultaneously during vehicle movement. Time alignment and spatial calibration of different sensor data are performed to ensure that the data used in subsequent modeling is in a unified coordinate system. When constructing a 3D Gaussian model based on real data, static backgrounds, road boundaries, traffic signs, and visible targets within the road segment can be used as modeling objects. These can be represented as multiple 3D Gaussian primitives with center position, covariance parameters, color features, and opacity parameters. By optimizing the parameters of each primitive, the model can fit the appearance distribution of the real scene. This model can be segmented and stored in blocks to facilitate the retrieval of model data for the corresponding area when the vehicle travels to different locations. In practical applications, other model representation methods can also be selected, which are not limited in this application.

[0089] When determining simulation data based on a 3D Gaussian model, the corresponding perception viewpoint can be determined according to the vehicle's current position, orientation, and sensor installation extrinsic parameters. The model then renders the scene from this perception viewpoint, outputting image data or point cloud data consistent with the imaging patterns of real sensors. When the vehicle's position changes, the model synchronously updates the perception viewpoint parameters and regenerates the corresponding simulation data, ensuring that the output remains consistent with the current road conditions. In this way, the simulation data can be directly used as sensor input in subsequent closed-loop testing, forming the vehicle's perception data source together with real data.

[0090] This implementation constructs a 3D Gaussian model using real data, and then generates simulation data from the model. This allows the scene information of the road segment where the vehicle is located to be continuously expressed in a computable manner, and to quickly output matching perception results as the vehicle's perception perspective changes. Thus, the perception input can remain synchronized with the vehicle's current state, providing a continuous and updatable sensor data foundation for subsequent algorithm testing.

[0091] Based on the aforementioned embodiments, a three-dimensional Gaussian model of the road segment where the vehicle is located is further constructed based on real data, including: constructing an initial model based on real data; training the initial model until convergence based on a preset loss function to obtain a three-dimensional Gaussian model of the road segment where the vehicle is located.

[0092] In the actual construction process, the road boundaries, lane areas, static obstacles, and surrounding building outlines are first spatially initialized based on multi-view images and point cloud data from real data. The initialization results are then mapped to a scene representation composed of multiple 3D Gaussian points, each containing at least spatial location, scale, color, and opacity parameters. Subsequently, this scene representation is rendered and compared with the real image under the corresponding perception viewpoint to obtain the error between the predicted image and the real image. The error and opacity constraint are jointly input into the loss function. During training, the location, scale, color, and opacity parameters of the Gaussian points are iteratively updated through backpropagation, so that the rendering result gradually approaches the real observation data. Convergence is determined when the change in the loss function is less than a preset threshold or a preset number of iterations is reached, thus outputting a 3D Gaussian model consistent with the real road environment.

[0093] This 3D Gaussian model serves as the basis for scene representation during test runs. It can generate sensor data from the corresponding perception perspective based on the vehicle's current position and form a closed-loop linkage with the control output of the intelligent driving algorithm under test. Since the model is initialized with real data and trained and converged through a loss function, its geometric structure, appearance details, and occlusion relationships are consistent with real road sections, providing continuous and stable scene support for subsequent simulation perception inputs.

[0094] After adopting the above method, the three-dimensional Gaussian model can accurately preserve the spatial layout and visual features of the real road section. After the model converges, it can be directly used to generate multi-sensory view sensor data, reduce scene reconstruction deviation, and ensure that the perception input after the vehicle control command is updated is consistent with the current vehicle state, thereby improving scene consistency and data availability in closed-loop testing.

[0095] Based on the aforementioned embodiments, further, an initial model is constructed based on real data, including: determining multiple Gaussian points and an image plane based on real data; projecting the multiple Gaussian points onto the image plane and rendering them to obtain the initial model.

[0096] In practical implementation, multiple Gaussian points can be initialized according to the spatial coordinate distribution in real data, and each Gaussian point is assigned scale and color parameters consistent with real observations, so that it can approximately represent the local features of targets such as roads, vehicles, pedestrians, traffic signs, and background buildings in three-dimensional space. When projecting multiple Gaussian points onto the image plane, a mapping relationship from spatial coordinates to two-dimensional pixel coordinates can be established based on camera intrinsic and extrinsic parameters. Then, the projection area is integrated or sampled according to the kernel function of the Gaussian distribution to obtain the corresponding pixel coverage. Subsequently, when rendering the projection result, the Gaussian points can be superimposed and synthesized based on their position, opacity, and color contribution on the image plane to output an initial image representation corresponding to the real data, thus forming an initial model.

[0097] The aforementioned initial model serves as the initial carrier for subsequent 3D Gaussian model training. It constrains the spatial location and appearance attributes of Gaussian points using real data and establishes a two-dimensional representation that can directly participate in optimization through projection rendering on the image plane. The initial model constructed in this way can retain the local geometric and appearance information of the real scene and provide a stable starting point for subsequent convergence training based on the loss function, thereby enabling the final model to maintain high consistency and usability when reconstructing the sensor's perceived viewpoint.

[0098] Building upon the foregoing embodiments, the Gaussian point further includes at least one of the following parameters: position, covariance matrix, opacity, and spherical harmonic coefficients.

[0099] In the actual construction of a 3D Gaussian model, the position parameter corresponds to a spatial sampling point on the target surface in the scene, the covariance matrix is ​​used to control the unfolding shape of the sampling point on the rendering plane, the opacity is used to participate in color synthesis, and the spherical harmonic function coefficients are used to generate the corresponding appearance color under a given perception viewpoint. By fitting point clouds, image features, and pose information from real data, at least one parameter can be assigned to each Gaussian point, thereby forming a scene representation that can express the appearance of roads, vehicles, roadside facilities, and dynamic targets. For the specific parameter form, floating-point vectors can be used to store the position, matrices or their decomposition forms can be used to store the covariance matrix, single-channel scalars can be used to store the opacity, and multi-order coefficient arrays can be used to store the spherical harmonic function coefficients. In practical applications, other implementation forms can also be selected for this parameter organization method, which is not limited in this application.

[0100] This parametric approach determines the projection center based on location, the coverage area based on the covariance matrix, the overlay weight based on opacity, and the color output based on spherical harmonic coefficients during rendering, thereby generating sensor data that matches the perception viewpoint. Since Gaussian points can simultaneously describe geometric distribution and appearance changes, the scene can be resynthesized according to the current perception viewpoint after the vehicle's pose is updated, maintaining input data consistent with the vehicle's state.

[0101] By adopting this Gaussian point parameter structure, the 3D Gaussian model can jointly express the spatial morphology and appearance related to the perception perspective of the road scene. The correspondence between the rendering result and the actual sensor perception perspective is more stable, which in turn ensures that the sensor data update and control command feedback are consistent during the simulation test, and improves the scene coherence and data availability in the closed-loop test.

[0102] In one possible embodiment, the loss function As shown in the formula below:

[0103]

[0104] in, , , These represent the weight parameters of the three loss terms. To predict the image, For real images, To calculate the absolute value of the differences between all pixels, It is a structural similarity index. This is a regularization term for opacity.

[0105] In the specific implementation, the loss function is used as the parameter update target when training the 3D Gaussian model. The training samples consist of images collected from real road environments and their corresponding poses. In each iteration, the model generates a predicted image based on the current Gaussian point position, covariance matrix, opacity, and spherical harmonic coefficients. The predicted image is then compared with the real image. The total loss is formed by calculating the absolute value of all pixel differences, the structural similarity index, and the opacity regularization term. The model parameters are then updated through backpropagation. The opacity regularization term can be implemented by L1 penalty or threshold constraint on the opacity of the Gaussian point. The structural similarity index can be calculated based on windowed local brightness, contrast, and structural information. In practical applications, other implementations of this regularization term can also be chosen, and this application does not limit this.

[0106] This loss function incorporates pixel-level consistency, structural consistency, and parameter constraints into the optimization objective during training, enabling the 3D Gaussian model to more accurately approximate real images when reconstructing road scenes and suppressing abnormal expansion or collapse of the opacity parameter. The resulting model can be used to generate simulated sensor data corresponding to the vehicle's current position, thus providing a continuous and consistent input for closed-loop simulation testing.

[0107] By adopting this loss function, the model fits image details and overall structure more evenly during training, the difference between the reconstruction results and the real scene is reduced, the stability of the opacity parameter is enhanced, and thus the consistency and usability of the 3D Gaussian model in multi-perception view simulation generation are improved.

[0108] Figure 3 A flowchart illustrating the simulation testing method for the intelligent driving algorithm provided in this application embodiment. Figure 2 .like Figure 3 As shown, the method includes:

[0109] S301. Acquire sensor data of the vehicle. The sensor data includes at least one of real data and simulation data. The real data is collected by multiple sensors when the vehicle is in a real road environment. The simulation data is obtained by simulating the perception perspective of multiple sensors based on a three-dimensional Gaussian model of the road segment where the vehicle is located.

[0110] S302. Input the sensor data into the intelligent driving algorithm to be tested to obtain control commands;

[0111] S303. Determine the vehicle's position and orientation information based on control commands;

[0112] S304. Update sensor data based on pose information and a 3D Gaussian model;

[0113] In its implementation, after receiving control commands, the simulation platform inputs these commands into the vehicle motion update module. Combining the current vehicle speed, tire rotation angle, and attitude state, it calculates the vehicle's pose change for the next moment and accumulates the new pose information. Subsequently, the simulation platform uses this pose information as an extrinsic parameter input for cameras, LiDAR, or other sensing sensors, performing perception viewpoint association processing with the 3D Gaussian model. This allows the model to re-render images, depth maps, point clouds, or other forms of sensor data under the new viewing direction and spatial position. The 3D Gaussian model can be pre-trained based on real road data or incrementally corrected during testing based on scene update results, ensuring that the updated sensor data remains consistent with the vehicle's current state.

[0114] This update method enables the algorithm under test to immediately receive a perception input matching the current vehicle pose after outputting control commands, and the changes in the perception perspective of the sensor data remain synchronized with the vehicle's movement. Thus, the correspondence between environmental input and vehicle state in closed-loop testing is maintained, and the reconstructed results of the sensor data can continuously change with the control response, thereby ensuring that subsequent algorithm inference, trajectory evaluation, and scene interaction analysis are all based on a consistent state.

[0115] S305. If the preset test termination condition is not triggered, return to execute S302;

[0116] S306. Generate and output a test report for the intelligent driving algorithm to be tested.

[0117] Based on the aforementioned embodiments, further, updating sensor data based on pose information includes: inputting pose information into a three-dimensional Gaussian model to generate corresponding sensor image simulation data; and updating sensor data based on the sensor image simulation data.

[0118] In its implementation, the system receives vehicle pose information derived from control commands, maps this pose information to rendering perception viewpoint parameters of a 3D Gaussian model, and drives the model to complete scene reconstruction and image generation under the corresponding perception viewpoint. The 3D Gaussian model can express the scene appearance based on the spatial position of Gaussian points, covariance matrix, opacity, and spherical harmonic function coefficients, thereby outputting sensor image simulation data consistent with the scene seen by the vehicle under the current pose. Subsequently, the system writes this sensor image simulation data into the sensor data cache or data queue to replace the currently mismatched acquired data, or to fuse and update it with other sensor data, so that the sensor data maintains consistency with the perception viewpoint corresponding to the vehicle's current position.

[0119] In this embodiment, sensor image simulation data is generated based on a 3D Gaussian model driven by pose information, and the sensor data is updated accordingly. This allows the algorithm under test to continuously receive perception inputs that match the vehicle's state during closed-loop operation. This method enables sensor data to change synchronously with changes in vehicle posture and integrates real-world data and simulation-generated data into the same data link, thereby maintaining perception continuity and state consistency during the testing process.

[0120] In one possible implementation, the test termination conditions include at least one of the following: reaching a preset test duration, a vehicle collision event, and the vehicle deviating from the road.

[0121] In practical applications, the preset test duration is used to limit the duration of a single test task, while collision events and deviations from the road are used to reflect abnormal states of the intelligent driving algorithm under test in terms of safety and path control. These three can be set individually or combined within the same test task. In this embodiment, the termination condition trigger signal is directly used as the loop exit signal, causing the test system to stop returning to the input of the intelligent driving algorithm under test after the termination condition is met, thereby generating the corresponding test report.

[0122] By incorporating time constraints, collision constraints, and road deviation constraints into the same termination criterion, simulation tests can be terminated promptly when the predetermined coverage time is reached, a safety failure occurs, or trajectory instability occurs, while maintaining consistency between test records and termination reasons. This facilitates statistical analysis of the stability and safety of the intelligent driving algorithm under test.

[0123] Based on the aforementioned embodiments, the method further includes: extracting the dynamic trajectory of dynamic objects in real data; constructing a three-dimensional dynamic model based on the dynamic trajectory; and superimposing the three-dimensional dynamic model onto a three-dimensional Gaussian model.

[0124] In practical implementation, real-world data can include multiple frames of images, point clouds, pose information, and timestamps. First, dynamic objects in consecutive frames are detected, tracked, and associated, extracting their center point coordinate sequences and orientation change sequences. Then, displacement trajectories and dynamic trajectories are calculated by combining the sampling time intervals. Based on this dynamic trajectory, the dynamic object can be modeled as a 3D target model with a rigid body shape, or its appearance information can be mapped to a mesh model, point cloud model, or skeleton-driven model with time attributes, enabling the model to update its position and pose according to the dynamic trajectory parameters during rendering. After the 3D dynamic model is constructed, it is spatially registered with a 3D Gaussian model, allowing the dynamic object to be superimposed on the static road, building, and environmental background according to its corresponding coordinate system. Sensor data containing dynamic interaction relationships is simultaneously output during the same scene rendering process.

[0125] In practical applications, the 3D Gaussian model provides a static representation of the road scene, while the 3D dynamic model supplements the motion information of targets such as vehicles and pedestrians. The two models, when superimposed, form a scene representation that includes occlusion relationships, relative position changes, and dynamic interactions. When the intelligent driving algorithm under test outputs control commands, the system can re-render the superimposed scene based on the updated vehicle pose and current timestamp, thereby generating sensor data input consistent with the vehicle's current state.

[0126] By adopting the above method, the motion changes of dynamic targets can be synchronously incorporated into the scene expression, and the spatial relationship between the static background and the dynamic targets can be kept consistent. This ensures that the input data in the simulation test and the vehicle control results remain consistent, and improves the scene consistency and dynamic interactive expression capability during closed-loop testing.

[0127] Based on the aforementioned embodiments, it further includes: transferring the surface texture features of dynamic objects in real data to a three-dimensional dynamic model.

[0128] In practical implementation, the system first detects and segments dynamic objects based on real data, and extracts their surface appearance regions by combining instance tracking results. Then, it calculates texture mapping parameters for the surface appearance regions to obtain texture coordinates, color maps, and material parameters related to the target's perception viewpoint. Subsequently, the texture coordinates and material parameters are written into the mesh surface or skeleton-driven surface of the 3D dynamic model, ensuring that the 3D dynamic model maintains an appearance consistent with the real data under different perception viewpoints. When the 3D dynamic model updates its position with the vehicle's dynamic trajectory, its corresponding surface texture features synchronously participate in the rendering output, thereby forming a dynamic object image with realistic appearance details. In practical applications, this texture transfer process can be implemented using multi-perception view texture fusion, occlusion region completion, and temporal consistency constraints, etc., and this application does not limit it in this regard.

[0129] By transferring surface texture features from real-world data to a 3D dynamic model, the 3D dynamic model can maintain a visual appearance consistent with dynamic targets in real-world scenes during subsequent simulation rendering, ensuring stable detail representation of dynamic targets despite changes in position and posture. Consequently, the sensor data generated in simulation tests is more consistent with the visual characteristics of dynamic objects in real roads, improving the realism and temporal continuity of sensor inputs in dynamic scenes.

[0130] In one possible implementation, the test report includes at least one of trajectory tracking error, safety metrics, comfort metrics, and perceived consistency.

[0131] In practical implementation, the test report is automatically generated by the test module after each closed-loop simulation. The test module summarizes and processes the control commands, pose information, sensor perception results, and scene update results collected during vehicle operation, and calculates various indicators according to a preset statistical window. When reflecting vehicle path control performance, the test report includes the average, peak, or cumulative error of trajectory tracking error; when reflecting safety performance, it includes information such as the number of collisions, the duration of boundary crossings, or the minimum distance; when reflecting ride comfort, it includes the root mean square value of acceleration or the peak value of Jerk; when reflecting the consistency between the perceived input and the current state, it includes the perception consistency score or consistency deviation value. The above indicators can be output individually or in combination to form a complete test result for the same algorithm version. In practical applications, the test report can also be presented in the form of tables, curves, or statistical summaries, which is not limited in this application.

[0132] In terms of working principle, the test report uniformly collects and summarizes key operational quantities during the closed-loop test process, transforming the algorithm's control results, actual vehicle response, and the degree of matching after sensor input updates into quantifiable indicators. This makes different algorithm versions comparable under the same test scenario. Since the test report includes at least one of the following: trajectory tracking error, safety indicators, comfort indicators, and perception consistency, it can reflect the algorithm's performance from multiple dimensions, including control accuracy, safety performance, ride smoothness, and perception matching.

[0133] With this approach, test results are no longer limited to a single collision or whether the endpoint is reached. Instead, they can be comprehensively evaluated by combining trajectory, comfort, and perception consistency. This improves the completeness and analytical relevance of closed-loop simulation test reports and provides clearer quantitative basis for the iterative optimization of intelligent driving algorithms.

[0134] In one possible implementation, before inputting the pose information into the 3D Gaussian model, the method further includes: acquiring supplementary data, which is data actually collected by multiple sensors of the vehicle during the test; and updating the 3D Gaussian model based on the supplementary data.

[0135] In practical implementation, after the vehicle executes the intelligent driving algorithm under test and generates new pose changes, the sensors simultaneously collect new image frames, point cloud frames, depth information, and positioning information. The controller timestamps and processes this data to form supplementary data corresponding to the current scene position. Subsequently, the supplementary data is input into the incremental update module of the 3D Gaussian model to correct the parameters of the original Gaussian points. When new occlusions, local structural changes, or missing details are detected, new Gaussian points are added or the weight distribution of existing Gaussian points is adjusted, thereby maintaining the consistency of the model with the real test scene. In practical applications, this update module can be executed by a graphics processor or a processor with parallel computing capabilities; this application does not limit this.

[0136] After the 3D Gaussian model is updated, it receives subsequent pose information for rendering calculations and outputs sensor image simulation data that matches the current vehicle state. This data continues to drive the intelligent driving algorithm under test. This method allows newly acquired real-world environmental information to be promptly fed back into the scene model during testing, maintaining the synchronization between subsequent simulation inputs and the real road environment. It also ensures that the vehicle's perception results after lane changes, steering, or approaching dynamic targets remain consistent with its current pose, thereby improving the continuity of sensor inputs and the completeness of scene representation in closed-loop testing.

[0137] In one possible implementation, the sensor includes at least one of a camera, a lidar, and a millimeter-wave radar, and the sensor data includes at least one of image data, point cloud data, and radar data.

[0138] In practical implementation, the vehicle's cameras, LiDAR, and millimeter-wave radar are connected to the data acquisition unit via in-vehicle Ethernet, CAN (Controller Area Network) bus, or USB (Universal Serial Bus) interface. The data acquisition unit buffers, synchronizes, and converts the outputs of each sensor to form a unified sensor data stream. Image data may include RGB (Red, Green, Blue) images, grayscale images, or cropped perception images; point cloud data may include a set of 3D coordinate points with intensity information; and radar data may include a target list, range-velocity spectrum, or point trace information. Depending on the test scenario, sensor data can be collected from real roads or generated by rendering a 3D Gaussian model and is synchronously refreshed after updating the vehicle's pose, ensuring that the intelligent driving algorithm under test continuously receives input corresponding to the current vehicle state.

[0139] In terms of working principle, the sensor inputs from multiple sensors collectively characterize the vehicle's current environment. Image data provides appearance and semantic information, point cloud data provides spatial structure information, and radar data provides distance and speed information for dynamic targets. The combination of these three can support closed-loop testing of the perception, decision-making, and control links. Because sensor data can be updated in real time with vehicle control commands, and the algorithm output remains consistent with subsequent inputs, the correspondence between scene perception and vehicle motion can be maintained during continuous testing.

[0140] By adopting this implementation method, the simulation test link can access multi-source data from cameras, LiDAR, and millimeter-wave radar, and establish a unified input relationship between image data, point cloud data, and radar data. This enables the intelligent driving algorithm under test to complete closed-loop operation based on multimodal perception information. This setup improves the completeness and state correlation of test inputs, allowing the test report results regarding trajectory tracking, safety, and perception consistency to more realistically reflect the algorithm's performance in complex road scenarios.

[0141] In one possible implementation, the simulation data also includes anomalous data generated when at least one of the multiple sensors is in a faulty state or when preset noise is introduced.

[0142] In practical implementation, when generating sensor image simulation data, point cloud simulation data, or radar simulation data, the simulation platform can inject anomaly markers into designated sensor channels. Based on these markers, the data in the corresponding channels can be processed through occlusion, drift, truncation, sampling sparsification, or amplitude perturbation to generate anomalous data. For camera channels, blurring, stripes, exposure shifts, or partial blackouts can be superimposed on the image; for lidar channels, the number of point clouds can be reduced, outliers can be inserted, or the distance distribution can be changed; for millimeter-wave radar channels, random biases can be applied to the target velocity, azimuth angle, or intensity value. The anomalous data still maintains its correspondence with the vehicle's current pose and scene representation, so as to reflect the true input characteristics after sensor anomalies in closed-loop testing. In practical applications, the model and implementation method of the anomaly injection module can be selected according to the simulation platform architecture, and this application does not limit this.

[0143] During operation, the system first generates normal simulation data based on the current scenario. Then, it intervenes in at least one sensor channel based on a preset fault template or noise template. Subsequently, it outputs abnormal data and sends it to the intelligent driving algorithm under test. After receiving the abnormal data, the intelligent driving algorithm under test can output control commands accordingly. The system then combines the control commands to update the vehicle pose and reconstruct subsequent sensor inputs, so that abnormal conditions can continuously affect the algorithm testing link in a continuous closed loop.

[0144] By adopting this method, the simulation data can cover abnormal situations such as sensor failure, signal degradation, and noise pollution, so that the test input can more completely reflect the perception degradation characteristics under complex working conditions, thereby effectively verifying the stability of the algorithm in recognition, decision-making and control under abnormal sensor conditions.

[0145] One possible implementation also includes: generating and outputting a graphical interface based on the test report, the graphical interface including the vehicle's simulated driving screen, driving trajectory, and quantitative curves of control commands.

[0146] In its implementation, after generating a test report, the system extracts the corresponding scene frame number, vehicle pose sequence, control command sequence, and evaluation index data from the report and inputs them into the visualization rendering module. The rendering module uses a graphics engine to perform frame-level reconstruction of the simulation scene, then combines this with the vehicle and road models to generate a simulated driving scene. Simultaneously, it draws the driving trajectory in a two-dimensional or three-dimensional coordinate system based on the pose sequence, and samples, normalizes, and maps the control commands according to a unified time reference to form a quantized curve. This graphical interface can be output through a local display terminal or a remote interactive interface, allowing users to simultaneously view test results, vehicle motion processes, and control response relationships. In this application, the rendering module, display terminal, and interactive interface can all be implemented using existing general-purpose hardware; other models can also be selected in practical applications, and this application does not impose any limitations on this.

[0147] By converting the test report results into simulation images, trajectories, and quantized curves, the scene performance, vehicle motion, and control behavior during the test process can be presented on the same interface, facilitating the analysis of the continuity, consistency, and control effectiveness of the algorithm output. Since the graphical interface is directly generated from the test report, its displayed content is consistent with the test results, providing an intuitive basis for subsequent result verification, version comparison, and problem localization.

[0148] This application provides a simulation testing method for intelligent driving algorithms, used for closed-loop feedback simulation in complex traffic scenarios during the development and verification of intelligent driving systems. The vehicle makes algorithmic decisions in a dynamic traffic environment, including intersections, lane changes, overtaking, and pedestrian crossings. After the algorithm makes a decision, it generates perception input data in real time that matches the new pose to verify the algorithm's closed-loop performance in real-world scenarios. Furthermore, by generating high-fidelity images and point cloud data, the object detection and semantic segmentation in the perception module can identify traffic participants, including vehicles, pedestrians, and traffic lights. Simultaneously, it supports multi-algorithm comparative testing, evaluating the safety and comfort differences between aggressive lane changes and conservative obstacle avoidance in the same scenario.

[0149] In one example, an autonomous vehicle performs a left lane change at an intersection, with dynamic traffic participants present, including pedestrians crossing and oncoming vehicles proceeding straight. The raw data collected includes multi-view camera images, LiDAR point clouds, and vehicle pose information. When acquiring vehicle sensor data, the sensors include at least one of cameras, LiDAR, and millimeter-wave radar, and the sensor data includes at least one of image data, point cloud data, and radar data. In a real road environment, driving data at the intersection is simultaneously acquired using multiple cameras (forward-looking, surround-looking, LiDAR, GNSS / IMU, etc.). During acquisition, the time synchronization error is kept less than 10ms, the pose accuracy is better than 0.1m, and camera intrinsic and extrinsic parameters are calibrated. The scene information of the road segment where the vehicle is located is determined from the real data output from multiple sensors, and the dynamic trajectories of dynamic objects are extracted. Keyframes are also extracted, one frame every 0.1 seconds, forming an image-pose pair sequence {(It, Tt)}, where Tt is the camera pose corresponding to the t-th frame.

[0150] When constructing the initial model of the road segment where the vehicle is located based on real data, a real-time renderable neural scene model is learned from image-pose pairs using 3DGS technology. Sparse point clouds are reconstructed using SfM and used as initial Gaussian distribution parameters for 3DGS to initialize the Gaussian point cloud. After multiple Gaussian points and the image plane are determined, the multiple Gaussian points are projected onto the image plane and rendered to form the initial model. Each Gaussian point includes at least one parameter from the following: position parameter, covariance matrix, opacity, and spherical harmonic function coefficients. The initial model is then trained according to a preset loss function, and the Adam optimizer iteratively updates the Gaussian parameters until the model converges, resulting in a 3D Gaussian model of the road segment where the vehicle is located. The loss function includes a term for the absolute pixel value difference between the predicted image and the real image, a structural similarity index term, and an opacity regularization term, with corresponding weight parameters for each of the three loss terms.

[0151] After obtaining the 3D Gaussian model, a 3D dynamic model is constructed based on the dynamic trajectories of dynamic objects in the real data. The surface texture features of the dynamic objects in the real data are then transferred to the 3D dynamic model. Finally, the 3D dynamic model is superimposed on the 3D Gaussian model to form a simulation scene that includes a static road environment and dynamic traffic participants. The new trajectory output by the algorithm under test requires vehicles to change lanes in advance to avoid oncoming vehicles. After changing lanes, the perception module needs to update lane lines, pedestrian positions, and traffic signal status in real time. Therefore, the original trajectory Torig is loaded as the initial input in the simulation environment, and the algorithm under test is run. The algorithm under test can be a decision control algorithm based on an end-to-end model. The algorithm outputs a new trajectory Tnew, which contains the pose sequence of the vehicle changing lanes in advance, such as the vehicle changing from the left lane to the middle lane at t=5s.

[0152] Subsequently, simulations were performed on the perception perspectives of multiple sensors based on a 3D Gaussian model to determine the simulation data. Based on the pose at each moment in the new trajectory Tnew, the RGB image of the corresponding perspective was rendered using a 3DGS model. During rendering, pixel colors were synthesized through alpha blending, and dynamic object modeling results were overlaid. These dynamic object modeling results included lightweight 3D models of pedestrians and oncoming vehicles. The position and motion state of the dynamic objects were simulated using a physics engine to ensure that pedestrian obstacle avoidance paths and occlusion relationships after vehicle lane changes were consistent with scene interactions. The simulation data could be data generated under normal conditions or abnormal data generated when at least one of the multiple sensors was malfunctioning or when preset noise was introduced. At least one of the real data and simulation data was used as the current sensor data input to the intelligent driving algorithm under test. The algorithm then output control commands, including the vehicle's steering, acceleration, and braking control quantities at the current moment.

[0153] The dynamically generated RGB image is fed into the perception module of the algorithm under test. This perception module includes a target detection network, and may also include a semantic segmentation module, to identify targets such as lane lines, pedestrians, oncoming vehicles, other vehicles, and traffic lights. It outputs perception results such as lane line position and pedestrian distance, and feeds the perception module's output back to the decision control module to generate new control commands, including adjusting steering angle and acceleration, and driving the vehicle dynamics model to update its pose. Based on the control commands output by the intelligent driving algorithm, the vehicle's pose information is determined. This pose information includes vehicle position and attitude changes. Supplementary data is acquired before inputting the pose information into the 3D Gaussian model. This supplementary data consists of real data collected by multiple sensors on the vehicle during testing. The 3D Gaussian model is then updated based on this supplementary data. After the model update is complete, the pose information is input into the 3D Gaussian model to generate corresponding sensor image simulation data. The current sensor data is updated based on this sensor image simulation data. When the sensor type includes LiDAR or millimeter-wave radar, point cloud data or radar data under the corresponding pose is generated synchronously and written into the updated sensor data, continuously forming a cyclical link of behavior, new perspectives, new perceptions, and new behaviors.

[0154] The updated sensor data is then input into the intelligent driving algorithm under test, continuously executing a loop of control command generation, pose information determination, model update, and sensor data update. During the loop test, preset test termination conditions are continuously monitored. These conditions include at least one of the following: reaching a preset test duration, a vehicle collision event, or vehicle deviation from the road. Collision events include collisions with pedestrians or oncoming vehicles. If a test termination condition is not triggered, iterative execution continues. After a test termination condition is triggered, at least one of the following is statistically analyzed during the test: trajectory tracking error, safety index, comfort index, and perception consistency. Trajectory tracking error includes deviation from the target trajectory after lane change; safety index includes collision rate; comfort index includes lateral acceleration and jerk. Simulation results from different algorithms are compared, including aggressive lane-changing algorithms and conservative obstacle avoidance algorithms.

[0155] After the statistics are completed, a test report for the intelligent driving algorithm under test is generated and output. Based on the test report, a graphical interface is generated, which includes the simulated driving scene of the vehicle, the driving trajectory, and the quantitative curve of the control command. For the same intersection left lane change scenario, aggressive lane change algorithm and conservative obstacle avoidance algorithm are input in the same scenario. The simulation results of the two in terms of safety, efficiency, and comfort are displayed side by side. Combining multi-view camera images, LiDAR point clouds, vehicle pose information, dynamically generated RGB images, point cloud data or radar data, as well as lane lines, pedestrian positions, traffic signal status, and changes in the occlusion relationship of oncoming vehicles, the closed-loop simulation process of the vehicle changing from the left lane to the middle lane at t=5s and the time before and after is presented in the graphical interface.

[0156] Figure 4 This is a schematic diagram of the structure of the simulation testing device for the intelligent driving algorithm provided in the embodiments of this application, as shown below. Figure 4 As shown, the simulation testing device 40 for intelligent driving algorithms provided in this embodiment includes:

[0157] The acquisition module is used for S1 to acquire sensor data of the vehicle. The sensor data includes at least one of real data and simulation data. The real data is collected by multiple sensors when the vehicle is in a real road environment. The simulation data is obtained by simulating the perception perspective of multiple sensors based on a three-dimensional Gaussian model of the road segment where the vehicle is located.

[0158] The test module is used by S2 to input sensor data into the intelligent driving algorithm under test to obtain control commands;

[0159] The test module is also used in S3 to update sensor data based on control commands;

[0160] The test module is also used in S4 to return to execution of S2 if the preset test termination condition is not triggered.

[0161] The output module is used by S5 to generate and output test reports for the intelligent driving algorithm under test.

[0162] In one possible embodiment, acquiring vehicle sensor data includes:

[0163] Obtain real data;

[0164] Based on real data, a three-dimensional Gaussian model of the road segment where the vehicle is located is constructed;

[0165] Based on the three-dimensional Gaussian model, the simulation data is determined.

[0166] In one possible implementation, a three-dimensional Gaussian model of the road segment where the vehicle is located is constructed based on real data, including:

[0167] Build an initial model based on real data;

[0168] Based on the preset loss function, the initial model is trained until convergence, resulting in a three-dimensional Gaussian model of the road segment where the vehicle is located.

[0169] In one possible implementation, an initial model is constructed based on real data, including:

[0170] Based on real data, multiple Gaussian points and image planes were determined;

[0171] Multiple Gaussian points are projected onto the image plane and rendered to obtain the initial model.

[0172] In one possible embodiment, the Gaussian point includes at least one parameter among location, covariance matrix, opacity, and spherical harmonic coefficients.

[0173] In one possible embodiment, the loss function As shown in the formula below:

[0174]

[0175] in, , , These represent the weight parameters of the three loss terms. To predict the image, For real images, To calculate the absolute value of the differences between all pixels, It is a structural similarity index. This is a regularization term for opacity.

[0176] In one possible embodiment, updating sensor data based on control commands includes:

[0177] Based on control commands, determine the vehicle's position and orientation information;

[0178] The sensor data is updated based on pose information and a 3D Gaussian model.

[0179] In one possible embodiment, updating sensor data based on pose information includes:

[0180] The pose information is input into the three-dimensional Gaussian model to generate the corresponding sensor image simulation data;

[0181] Update sensor data based on sensor image simulation data.

[0182] In one possible embodiment, the test termination conditions include at least one of the following: reaching a preset test duration, a vehicle collision event, and the vehicle deviating from the road.

[0183] In one possible embodiment, it also includes:

[0184] Extract the dynamic trajectory of dynamic objects from real data;

[0185] Construct a three-dimensional dynamic model based on dynamic trajectory;

[0186] The three-dimensional dynamic model is superimposed onto the three-dimensional Gaussian model.

[0187] In one possible embodiment, it also includes:

[0188] Transfer the surface texture features of dynamic objects in real data to a 3D dynamic model.

[0189] In one possible embodiment, the test report includes at least one of trajectory tracking error, safety metrics, comfort metrics, and perceived consistency.

[0190] In one possible embodiment, before inputting the pose information into the 3D Gaussian model, the method further includes:

[0191] Obtain supplementary data, which consists of actual data collected by multiple sensors of the vehicle during the test.

[0192] The 3D Gaussian model is updated based on the supplementary data.

[0193] In one possible embodiment, the sensor includes at least one of a camera, a lidar, and a millimeter-wave radar, and the sensor data includes at least one of image data, point cloud data, and radar data.

[0194] In one possible embodiment, the simulation data also includes anomalous data generated when at least one of the multiple sensors is in a faulty state or when preset noise is introduced.

[0195] In one possible embodiment, it also includes:

[0196] Based on the test report, a graphical interface is generated and output, which includes the vehicle's simulated driving screen, driving trajectory, and quantitative curves of control commands.

[0197] The intelligent driving algorithm simulation testing device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0198] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 50 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the device 50 further includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus.

[0199] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.

[0200] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0201] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0202] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0203] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0204] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0205] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0206] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0207] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0208] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0209] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0210] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0211] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0212] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0213] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A simulation testing method for an intelligent driving algorithm, characterized in that, include: S1. Acquire sensor data of the vehicle, wherein the sensor data includes at least one of real data and simulation data, wherein the real data is collected by multiple sensors when the vehicle is in a real road environment, and the simulation data is obtained by simulating the perception perspective of the multiple sensors based on a three-dimensional Gaussian model of the road segment where the vehicle is located. S2. Input the sensor data into the intelligent driving algorithm to be tested to obtain control commands; S3. Update the sensor data based on the control command; S4. If the preset test termination condition is not triggered, return to execute S2; S5. Generate and output a test report for the intelligent driving algorithm to be tested.

2. The method according to claim 1, characterized in that, The acquisition of vehicle sensor data includes: Obtain real data; Based on the real data, a three-dimensional Gaussian model of the road segment where the vehicle is located is constructed; Based on the aforementioned three-dimensional Gaussian model, simulation data is determined.

3. The method according to claim 2, characterized in that, The construction of a three-dimensional Gaussian model of the road segment where the vehicle is located based on the real data includes: Based on the real data, an initial model is constructed; Based on a preset loss function, the initial model is trained until convergence to obtain a three-dimensional Gaussian model of the road segment where the vehicle is located.

4. The method according to claim 3, characterized in that, The process of constructing an initial model based on the real data includes: Based on the real data, multiple Gaussian points and image planes are determined; The multiple Gaussian points are projected onto the image plane and rendered to obtain the initial model.

5. The method according to claim 4, characterized in that, The Gaussian point includes at least one of the following parameters: position, covariance matrix, opacity, and spherical harmonic coefficients.

6. The method according to claim 3, characterized in that, The loss function As shown in the formula below: in, , , These represent the weight parameters of the three loss terms. To predict the image, For real images, To calculate the absolute value of the differences between all pixels, It is a structural similarity index. This is a regularization term for opacity.

7. The method according to claim 1, characterized in that, The step of updating the sensor data based on the control command includes: Based on the control commands, the vehicle's position and pose information is determined; The sensor data is updated based on the pose information and the three-dimensional Gaussian model.

8. The method according to claim 7, characterized in that, The step of updating the sensor data based on the pose information includes: The pose information is input into the three-dimensional Gaussian model to generate corresponding sensor image simulation data; The sensor data is updated based on the simulated sensor image data.

9. The method according to claim 1, characterized in that, The test termination conditions include at least one of the following: reaching a preset test duration, the vehicle experiencing a collision event, and the vehicle deviating from the road.

10. The method according to claim 2, characterized in that, Also includes: Extract the dynamic trajectory of dynamic objects from the real data; Based on the dynamic trajectory, a three-dimensional dynamic model is constructed; The three-dimensional dynamic model is superimposed onto the three-dimensional Gaussian model.

11. The method according to claim 10, characterized in that, Also includes: The surface texture features of dynamic objects in the real data are transferred to the three-dimensional dynamic model.

12. The method according to claim 1, characterized in that, The test report includes at least one of the following: trajectory tracking error, safety indicators, comfort indicators, and perceptual consistency.

13. The method according to claim 8, characterized in that, Before inputting the pose information into the three-dimensional Gaussian model, the method further includes: Obtain supplementary data, which is the actual data collected by multiple sensors of the vehicle during the test; The three-dimensional Gaussian model is updated based on the supplementary data.

14. The method according to claim 1, characterized in that, The sensor includes at least one of a camera, a lidar, and a millimeter-wave radar, and the sensor data includes at least one of image data, point cloud data, and radar data.

15. The method according to claim 1, characterized in that, The simulation data also includes abnormal data generated when at least one of the plurality of sensors is in a faulty state or when preset noise is introduced.

16. The method according to claim 1, characterized in that, Also includes: Based on the test report, a graphical interface is generated and output, which includes the simulated driving screen of the vehicle, the driving trajectory, and the quantization curve of the control commands.

17. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-16.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-16.