Neural-radiance-field-based autonomous driving scene simulation method, device and simulation platform

Abstract

The application relates to a neural radiation field-based automatic driving scene simulation method and device and a simulation platform, wherein the method comprises the following steps: simulating an automatic driving scene based on a neural radiation field to obtain a simulation scene; performing data processing on the simulation scene to obtain object metadata and corresponding position information; and based on the assistance of a three-dimensional space view of the simulation scene and a first bird's-eye view of a dynamic object track, performing scene editing on a dynamic object in the simulation scene according to the object metadata and the corresponding position information. Through the application, the problem that scene editing functions are limited and the process is complex in the related art is solved, and the scene editing on the dynamic object in the simulation scene is accurately controllable based on the assistance of the three-dimensional space view of the simulation scene and the first bird's-eye view of the dynamic object track, so that the deficiency of scene editing is made up.

Description

Technical Field

[0001] This application relates to the field of computer vision scene simulation technology, and in particular to autonomous driving scene simulation methods, devices and simulation platforms based on neural radiation fields. Background Technology

[0002] As autonomous driving technology continues to develop, the demand for rigorous testing of safe autonomous vehicles is also increasing. For autonomous vehicles to meet the requirements for driving in the real world, they need to undergo rigorous testing in various aspects. In testing autonomous vehicles, key boundary scenarios are crucial for safe driving. However, the real world exhibits a long-tail effect; boundary scenarios in driving are often quite extreme, making real-world testing very expensive and dangerous. Therefore, the application of simulation in the field of autonomous driving has been proposed.

[0003] Current autonomous driving scene simulation schemes based on neural radiation fields involve modeling moving objects and static backgrounds separately, and learning the movement and appearance changes of dynamic objects to reconstruct the autonomous driving scene. However, this scheme has limited scene editing capabilities and a complex process. Currently, a clear, accurate, and controllable scene editing scheme is lacking. Summary of the Invention

[0004] This embodiment provides a method, device, and simulation platform for simulating autonomous driving scenarios based on neural radiation fields, in order to solve the problems of limited scene editing functions and complex processes in related technologies.

[0005] Firstly, this embodiment provides a method for simulating autonomous driving scenarios based on neural radiation fields, including:

[0006] The simulation scenario is obtained by simulating the autonomous driving scenario based on the neural radiation field;

[0007] The simulation scene is processed to obtain object metadata and corresponding location information;

[0008] With the aid of a three-dimensional spatial view of the simulation scene and a first bird's-eye view of the dynamic object trajectory, scene editing is performed on the dynamic objects in the simulation scene based on the object metadata and the corresponding position information.

[0009] In some embodiments, aided by a three-dimensional spatial view of the simulation scene and a first bird's-eye view of the dynamic object's trajectory, scene editing is performed on the dynamic objects in the simulation scene based on the object's metadata and the corresponding position information, including:

[0010] Based on the lidar point cloud, object metadata, and location information in the simulation scene, a first bird's-eye view of the dynamic object trajectory is constructed in the camera coordinate system.

[0011] On the first bird's-eye view of the trajectory of the dynamic object, the position of the dynamic object in the three-dimensional space view is drawn based on the object metadata.

[0012] In the camera coordinate system, while keeping the object metadata unchanged, the position information is edited to perform scene editing on the dynamic objects in the simulation scene;

[0013] The edited object metadata and location information are projected onto the image coordinate system to complete rendering, resulting in a new simulation scene.

[0014] In some embodiments, based on the lidar point cloud, object metadata, and location information in the simulation scene, a first bird's-eye view of the dynamic object trajectory is constructed in the camera coordinate system, including:

[0015] The lidar point cloud in the simulation scene is drawn in the second bird's-eye view of the lidar coordinate system, and the second bird's-eye view under the lidar coordinate system is transformed to the camera coordinate system.

[0016] The object metadata and the location information are projected onto the transformed second bird's-eye view to obtain a first bird's-eye view of the dynamic object trajectory.

[0017] In some embodiments, scene editing is performed on dynamic objects in the simulation scene, including:

[0018] The simulation scene includes the ability to delete objects, add objects, and / or modify the trajectories of dynamic objects.

[0019] In some embodiments, autonomous driving scenarios are simulated based on neural radiation fields to obtain simulated scenarios, including:

[0020] Based on the data collected from autonomous driving scenarios, scene images, camera parameters, object label information, and LiDAR point clouds are obtained; the scene images correspond to the camera parameters.

[0021] Based on the scene image, the LiDAR point cloud, and the object label information, a three-dimensional Gaussian representation of the autonomous driving scene is determined; the three-dimensional Gaussian representation includes a static background model and multiple dynamic object models.

[0022] The simulation scene is obtained by optimizing the 3D Gaussian based on the scene image, the camera parameters, and the preset loss function.

[0023] In some embodiments, determining a three-dimensional Gaussian representation of the autonomous driving scene based on the scene image, the LiDAR point cloud, and the object label information includes:

[0024] Based on the scene image, the lidar point cloud, and the object label information, the simulation scene is decomposed into a static background and multiple dynamic objects;

[0025] Determine the Gaussian representation of the static background model corresponding to the static background;

[0026] Determine the Gaussian representation of the dynamic object model corresponding to each of the dynamic objects;

[0027] The three-dimensional Gaussian representation of the autonomous driving scene is determined based on the Gaussian representation of the static background model and the Gaussian representation of each of the dynamic object models.

[0028] In some embodiments, the 3D Gaussian model is optimized based on the scene image, the camera parameters, and a preset loss function to obtain the simulation scene, including:

[0029] The Gaussian points are projected from three-dimensional space onto a two-dimensional plane using the transformation matrix in the camera parameters to obtain the rendered image;

[0030] The gradient update parameters are determined based on the rendered image and the scene image;

[0031] Based on the gradient update parameters, the three-dimensional Gaussian is iteratively optimized until the loss function is satisfied, thus obtaining the simulation scene.

[0032] In some embodiments, the method further includes:

[0033] During the iterative optimization of the three-dimensional Gaussian, adaptive density control is used to prune and remove the three-dimensional Gaussian points in the autonomous driving scenario.

[0034] In some embodiments, the method further includes:

[0035] Based on a no-reference evaluation metric, the image quality of the simulated scene in the scene editing is evaluated, and the evaluation results are obtained.

[0036] Secondly, this embodiment provides an autonomous driving scenario simulation device based on neural radiation fields, including: a simulation unit and an editing unit;

[0037] The simulation unit is used to simulate autonomous driving scenarios based on neural radiation fields to obtain simulation scenarios;

[0038] The editing unit is used to process data in the simulation scene to obtain object metadata and corresponding location information;

[0039] With the aid of a three-dimensional spatial view of the simulation scene and a first bird's-eye view of the dynamic object trajectory, scene editing is performed on the dynamic objects in the simulation scene based on the object metadata and the corresponding position information.

[0040] Thirdly, this embodiment provides a simulation platform, including: a user layer, a service layer, and a data layer; the service layer is connected to the user layer and the data layer respectively;

[0041] The user layer is used to assign corresponding user permissions to different users;

[0042] The data layer is used to store data from the simulation platform;

[0043] The service layer is used to execute the autonomous driving scenario simulation method based on neural radiation fields as described in the first aspect, under requests for different user permissions.

[0044] In some embodiments, the simulation platform further includes a presentation layer and an interface layer;

[0045] The interface layer is used to receive data requests from the presentation layer, call the relevant services of the service layer, and return the request result corresponding to the data request to the presentation layer.

[0046] The presentation layer is used to display the result of the request.

[0047] Compared with related technologies, the autonomous driving scene simulation method, device, and simulation platform based on neural radiation fields provided in this embodiment simulates autonomous driving scenes based on neural radiation fields to obtain simulated scenes; performs data processing on the simulated scenes to obtain object metadata and corresponding location information; and, with the assistance of a three-dimensional spatial view of the simulated scene and a first bird's-eye view of the dynamic object trajectories, edits the dynamic objects in the simulated scene according to the object metadata and corresponding location information. This solves the problems of limited scene editing functions and complex processes in related technologies. The assistance of a three-dimensional spatial view of the simulated scene and a first bird's-eye view of the dynamic object trajectories enables accurate and controllable scene editing of dynamic objects in the simulated scene, making up for the deficiencies in scene editing.

[0048] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0049] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0050] Figure 1 This is a hardware structure block diagram of a terminal device for an autonomous driving scene simulation method based on neural radiation fields provided in an embodiment of this application;

[0051] Figure 2 This is a flowchart of an embodiment of an autonomous driving scenario simulation method based on neural radiation fields provided in this application;

[0052] Figure 3 This is a flowchart of step S210;

[0053] Figure 4 This is a schematic diagram of the scene editing process provided in one embodiment of this application;

[0054] Figure 5 This is a schematic diagram of a three-dimensional spatial view provided in an embodiment of this application;

[0055] Figure 6 This is a schematic diagram of a first bird's-eye view of the trajectory of a dynamic object provided in an embodiment of this application;

[0056] Figure 7 This is a structural block diagram of an autonomous driving scene simulation device based on neural radiation field provided in an embodiment of this application;

[0057] Figure 8 This is a structural block diagram of a simulation platform provided in one embodiment of this application.

[0058] In the diagram: 102, processor; 104, memory; 106, transmission device; 108, input / output device; 210, simulation unit; 220, editing unit. Detailed Implementation

[0059] To better understand the purpose, technical solution, and advantages of this application, the application is described and explained below in conjunction with the accompanying drawings and embodiments.

[0060] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning as understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these,” used in this application, do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to such processes, methods, products, or devices. The terms “connected,” “linked,” and “coupled,” used in this application, are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. The term “multiple” used in this application refers to two or more. The "and / or" operator describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: A alone, A and B simultaneously, and B alone. Typically, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," and "third," etc., used in this application are merely for distinguishing similar objects and do not represent a specific ordering of the objects.

[0061] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the autonomous driving scene simulation method based on neural radiation fields in this embodiment. (See diagram for example.) Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0062] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the autonomous driving scenario simulation method based on neural radiation fields in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0063] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0064] This embodiment provides a method for simulating autonomous driving scenarios based on neural radiation fields. Figure 2 This is a flowchart of the autonomous driving scenario simulation method based on neural radiation fields in this embodiment, as shown below. Figure 2 As shown, the process includes the following steps:

[0065] Step S210: Simulate the autonomous driving scenario based on the neural radiation field to obtain the simulation scenario;

[0066] Step S220: Perform data processing on the simulation scene to obtain object metadata and corresponding location information;

[0067] Step S230: Based on the assistance of the three-dimensional spatial view of the simulation scene and the first bird's-eye view of the dynamic object trajectory, scene editing is performed on the dynamic objects in the simulation scene according to the object metadata and corresponding position information.

[0068] It should be noted that Neural Radiance Fields (NeRF) refers to using a multilayer perceptron neural network to implicitly learn a static 3D scene, enabling the synthesis (rendering) of arbitrary new perspectives in complex scenes. In the field of autonomous driving, based on NeRF, dynamic objects and static backgrounds in the autonomous driving scene are modeled separately in the simulation scene. For dynamic objects, the changes in their movement and appearance are learned to obtain the simulation scene; thus, the simulation scene contains corresponding static background models and dynamic object models. One NeRF can correspond to one model. It should be noted that during the simulation process, this can include the implementation of step S220 regarding data processing; or it can directly call a third-party platform to obtain the simulation scene and the data collected from the autonomous driving scene.

[0069] This involves using image processing algorithms or neural network models to process the simulation scene data, extracting object metadata and corresponding location information from the object label information. The object metadata can be maintained by the array `obj_metadata`, which stores the unique IDs, dimensions, and heights of all objects in the current simulation scene. The location information is maintained by the array `obj_info`, a three-dimensional array that stores the object's position coordinates (coordinates along the x, y, and z axes in the camera coordinate system) and yaw angle at each timestamp.

[0070] Since the 3D spatial view of the simulation scene is in the camera coordinate system, the first bird's-eye view of the dynamic object trajectory can be associated with the lidar coordinate system and the camera coordinate system, and the object metadata and corresponding position information are in the camera coordinate system; therefore, the object metadata and corresponding position information can be projected onto the first bird's-eye view in the lidar coordinate system for editing, and then rendered into the 3D spatial view, thereby realizing scene editing of dynamic objects in the simulation scene. This achieves accurate and controllable scene editing of dynamic objects in the simulation scene, making up for the deficiencies in scene editing.

[0071] The steps described above are explained in detail below:

[0072] In some of these embodiments, such as Figure 3 As shown, step S210, which involves simulating the autonomous driving scenario based on the neural radiation field to obtain the simulation scenario, includes the following steps:

[0073] Step S211: Based on the acquisition of autonomous driving scenarios, scene images, camera parameters, object label information, and LiDAR point clouds are obtained; the scene images correspond to the camera parameters.

[0074] Step S212: Determine the 3D Gaussian representation of the autonomous driving scene based on the scene image, LiDAR point cloud, and object label information; the 3D Gaussian representation includes a static background model and multiple dynamic object models.

[0075] Step S213: Optimize the 3D Gaussian based on the scene image, camera parameters, and a preset loss function to obtain the simulation scene.

[0076] This embodiment can be considered as a scene simulation process, which can be divided into data acquisition, model building, and optimization.

[0077] For data acquisition: image sensors can be used to capture images of the autonomous driving scene, obtaining scene images, camera parameters, and object label information. The scene images and camera parameters correspond; if the image sensor is a camera, then the camera parameters include intrinsic and extrinsic parameters. Intrinsic parameters include the camera's optical center, focal length, pixel scaling, and translation, while extrinsic parameters are the camera's rotation matrix and translation vector, i.e., the transformation matrix, which determines the camera's pose. The scene image can be an RGB image. LiDAR is used to capture images of the autonomous driving scene, obtaining a LiDAR point cloud; this LiDAR point cloud can be an SFM point cloud.

[0078] For model building: the simulation scene is decomposed into a static background and multiple dynamic objects. For the static background, a Gaussian representation of the corresponding static background model is used; for each dynamic object, a Gaussian representation of the corresponding dynamic object model is used. The set of these Gaussian representations determines the 3D Gaussian representation of the autonomous driving scene. Anisotropic 3D Gaussian Splatting serves as a high-quality representation of neural radiation fields, with the entire scene space represented as Gaussian points. The geometry in the scene is modeled as 3D Gaussian functions. In 3D Gaussian rendering, the 3D Gaussians in the scene can be viewed as colored ellipsoids. Projecting these ellipsoids along a specific angle onto the projection plane corresponding to the pose is called splatting. Projecting an ellipsoid onto the plane results in an ellipse. The opacity is obtained by calculating the distance between the pixel to be solved and the center of the ellipse (the closer the distance, the less transparent). The color of each ellipsoid is independent of distance. Colors are synthesized through alpha mixing. In other embodiments, other methods can also be used to represent the autonomous driving scene; there are no limitations on this.

[0079] Regarding optimization: relevant optimization methods can be used to optimize the 3D Gaussian based on the scene image, camera parameters, and a preset loss function to obtain the simulated scene, without any restrictions. Examples include: using a block-based strategy for 3D Gaussian training and rendering, and gradient iterative optimization.

[0080] This embodiment can accurately capture the view-related appearance in the scene, improving the simulation effect of autonomous driving scenarios.

[0081] In some embodiments, step S212, which involves determining the 3D Gaussian representation of the autonomous driving scene based on the scene image, LiDAR point cloud, and object label information, includes the following steps:

[0082] Based on scene images, LiDAR point clouds, and object label information, the simulation scene is decomposed into a static background and multiple dynamic objects.

[0083] Determine the Gaussian representation of the static background model corresponding to the static background;

[0084] Determine the Gaussian representation of the dynamic object model corresponding to each dynamic object;

[0085] Based on the Gaussian representation of the static background model and the Gaussian representation of each dynamic object model, the 3D Gaussian representation of the autonomous driving scenario is determined.

[0086] Specifically, based on scene images, LiDAR point clouds, and object label information, the simulation scene is decomposed into a static background and multiple dynamic objects.

[0087] For a static background model with a static background, the three-dimensional covariance matrix is ​​used to represent it as follows:

[0088] ∑=RSS T R T ;

[0089] In the formula, R represents the rotation matrix; S represents the scaling matrix; S T and R T ∑ represents the transpose of the corresponding matrix; ∑ is a positive semi-definite real symmetric matrix, which is a three-dimensional covariance matrix representing the shape of Gaussian points in space. It can always be orthogonally diagonalized. Since it is positive semi-definite, the diagonal matrix representing eigenvalues ​​can be split.

[0090] Eigenvalue decomposition is performed on the 3D covariance matrix to obtain the rotation matrix, scaling matrix, and corresponding transpose. To maintain positive semi-definiteness, a quaternion q is used to represent the rotation matrix, and the diagonal elements of the scaling matrix are extracted to obtain the scaling coefficients s. During gradient updates, the rotation quaternion q and scaling coefficients s are used to simplify the computation. After optimization, the representation reverts to the 3D covariance matrix. Therefore, the Gaussian representation of the static background model is:

[0091] G S ={μ,q,s,α,c};

[0092] In the formula, μ is the position coordinate (x, y, z); α represents the opacity; and c represents the color.

[0093] Dynamic objects are extracted using laser radar point clouds (SFM point clouds) and object label information. The SFM point clouds are used as the initial input to the dynamic object model, adding a time dimension to the static background model. Opacity and position are represented as time-dependent functions to express the effect of object motion at different time stamps. The Gaussian representation of the dynamic object model is then:

[0094] G d (t)={μ(t),q,s,α(t),c}

[0095] In the formula: μ(t) represents the position coordinates that change over time; α(t) represents the opacity that changes over time.

[0096] Two parts of data are extracted from the object label information: the object's metadata and its location information. The transformation of the object's coordinate system requires the transformation matrix (extrinsic parameter) from the camera parameters. The camera's projection matrix, distortion correction matrix, matrix for transforming LiDAR coordinates to the camera coordinate system, and matrix for transforming IMU coordinates to LiDAR coordinates are extracted from the camera's intrinsic parameters. Therefore, the autonomous driving scene can ultimately be represented as a combination of a static background and multiple dynamic object models, with its 3D Gaussian representation as follows:

[0097] G c =∑ <O,G d ,M,P,T>+G s ;

[0098] In the formula, O represents object metadata such as length, width, height, and unique ID; Gd represents the dynamic object model; M represents the camera parameters including the projection transformation matrix between different coordinate systems; P represents the attitude information including the object's position coordinates and yaw angle (belonging to position information); T represents the timestamp (belonging to position information); and Gs represents the static background model.

[0099] This embodiment enables a three-dimensional Gaussian representation of an autonomous driving scenario and simplifies the structure of the three-dimensional Gaussian representation, facilitating subsequent model optimization.

[0100] In some embodiments, step S213, which optimizes the 3D Gaussian based on the scene image, camera parameters, and a preset loss function to obtain the simulation scene, includes the following steps:

[0101] The Gaussian points are projected from three-dimensional space onto a two-dimensional plane using the transformation matrix in the camera parameters to obtain the rendered image;

[0102] Determine the gradient update parameters based on the rendered image and the scene image;

[0103] Based on gradient-updated parameters, the 3D Gaussian is iteratively optimized until the loss function is satisfied, thus obtaining the simulation scene.

[0104] Specifically, after obtaining the 3D Gaussian representation, the Gaussian points are projected from 3D space to a 2D plane using the transformation matrix in the camera parameters. A block-based fast differentiable rendering algorithm is then used to obtain the rendered image. The loss function is calculated based on the rendered image and the scene image, and then backpropagation is performed to calculate the gradient and update the parameters.

[0105] During the iterative optimization process, stochastic gradient descent is used until the loss function is satisfied, resulting in the simulated scenario. Specifically, the sigmoid activation function is used to constrain the opacity within the range [0,1), obtaining a smooth gradient. The scale of the covariance is constrained using an exponential activation function. Therefore, the final loss function can be expressed as:

[0106] lι=(1-λ)l1+λl ssim ;

[0107] In the formula, l1 represents the loss of image color; l ssim SSIM loss represents the image structure similarity; λ represents the control hyperparameter, which is usually set to 0.2.

[0108] This embodiment improves the efficiency of iterative optimization while enhancing the accuracy of scene simulation.

[0109] In some embodiments, the autonomous driving scenario simulation method based on neural radiation fields further includes the following steps:

[0110] During the iterative optimization of the 3D Gaussian, adaptive density control is used to prune and remove 3D Gaussian points in the autonomous driving scenario.

[0111] Specifically, during the gradient update process of iterative optimization, adaptive density control is used to prune and remove 3D Gaussian points in the scene. Gaussian points are added to non-repeating regions in the reconstructed simulation scene, while Gaussian points are deleted from over-reconstructed regions, ensuring a relatively stable number of Gaussian points in the simulation scene. Since autonomous driving deals with large-scale urban scenes, the actual distance between most points and the center of the boundless scene is significant. To represent the scene with fewer points while maintaining simulation accuracy, in this embodiment, adaptive density control uses a scale factor to control the size of the 3D Gaussian points, using larger points for distant regions and smaller points for nearby regions.

[0112] This embodiment enables parallel processing of large scenes, improving processing efficiency. Ultimately, this method can provide high-quality visual effects while rendering in real time, adapting to available computing resources and making it possible to capture and render large-scale scenes.

[0113] In some embodiments, step S230, aided by a three-dimensional spatial view of the simulation scene and a first bird's-eye view of the dynamic object trajectory, performs scene editing on the dynamic objects in the simulation scene based on object metadata and corresponding position information, including the following steps:

[0114] Step S231: Based on the lidar point cloud, object metadata, and location information in the simulation scene, construct a first bird's-eye view of the dynamic object trajectory in the camera coordinate system.

[0115] Step S232: On the first bird's-eye view of the dynamic object's trajectory, draw the object position of the dynamic object in the three-dimensional space view based on the object's metadata.

[0116] Step S233: In the camera coordinate system, keep the object metadata unchanged, edit the position information, and perform scene editing on the dynamic objects in the simulation scene;

[0117] Step S234: Project the edited object metadata and position information onto the image coordinate system to complete the rendering and obtain a new simulation scene.

[0118] Specifically, such as Figure 4 As shown, after data processing to obtain object metadata and corresponding location information, scene editing can be considered to include multi-view assistance and location editing. Multi-view assistance refers to the assistance of a 3D spatial view and a first bird's-eye view. The 3D spatial view is constructed in the camera coordinate system and carries the simulation scene. Since the LiDAR point cloud is in the LiDAR coordinate system, while the object metadata and location information are in the camera coordinate system, and because they are located in different coordinate systems, it is necessary to transform the LiDAR point cloud to the camera coordinate system to construct a first bird's-eye view of the dynamic object trajectory. In other embodiments, constructing the first bird's-eye view of the dynamic object trajectory can also be achieved in other ways, and there is no limitation on this.

[0119] Based on the first bird's-eye view of the dynamic object's trajectory, the object's position in the 3D space view is plotted using the object's metadata in the camera coordinate system, providing a more intuitive understanding of the scene's 3D spatial situation. By combining the 2D first bird's-eye view and the 3D space view, a comprehensive understanding of the scene's condition can be achieved, visually demonstrating the changes in object state before and after scene editing. In this embodiment, the 3D space view is plotted using the coordinates of the eight vertices of the object modeled as a cuboid, with each cuboid labeled with its object ID and the frame number it belongs to. Furthermore, to align with normal 3D space coordinates, the y and z axis data are swapped in the camera coordinate system, resulting in a 3D coordinate system unfolded in an xyz format. The 3D space view of the scene plotted using matplotlib is shown below. Figure 5 As shown.

[0120] During scene editing, the object metadata remains unchanged in the camera coordinate system, while the position information is edited (by modifying the data in the `obj_info` array to generate a new scene). Since the position information is edited directly in the original camera coordinate system (by modifying the `obj_info` array), the transformation matrix provided by the camera parameters is then used to project the edited object metadata and position information onto the image coordinate system to complete the subsequent rendering, resulting in a new simulation scene; this allows for scene editing of dynamic objects in the simulation scene.

[0121] In this embodiment, since the position information of the camera coordinate system has the meaning of real objects and is real-world information, with the data unit being meters, precise scene editing is achieved. Editing directly in the camera coordinate system allows for a direct comparison of the changes in object position before and after editing. Editing in the camera coordinate system makes the object position more accurate, is convenient to operate, and allows for direct editing of the required trajectory.

[0122] In some embodiments, step S231, which involves constructing a first bird's-eye view of the dynamic object trajectory in the camera coordinate system based on the lidar point cloud, object metadata, and location information in the simulation scene, includes the following steps:

[0123] The lidar point cloud in the simulation scene is drawn in the second bird's-eye view of the lidar coordinate system, and the second bird's-eye view in the lidar coordinate system is transformed to the camera coordinate system.

[0124] The object's metadata and location information are projected onto the transformed second bird's-eye view to obtain the first bird's-eye view of the dynamic object's trajectory.

[0125] Specifically, editing the trajectory of dynamic objects requires using a first bird's-eye view to clearly and intuitively describe the movement trajectory information of objects in the scene. The first bird's-eye view corresponds to the view of the dynamic object trajectory in the camera coordinate system, including the object's metadata and position information in the camera coordinate system. The second bird's-eye view corresponds to the view of the LiDAR point cloud in the LiDAR coordinate system. Therefore, to construct this first bird's-eye view, we need the LiDAR point cloud (located in the LiDAR coordinate system), object metadata, and position information (located in the camera coordinate system). Thus, we need to transform the LiDAR point cloud into the camera coordinate system using a transformation matrix (found in the camera parameters). The process of drawing the LiDAR point cloud in the second bird's-eye view in the LiDAR coordinate system is as follows: 1. Read the LiDAR point cloud. The storage format of the LiDAR point cloud is n×4; where 4 represents the array dimension of the point cloud storage, i.e., there are n point clouds, each with four parameters including three-dimensional coordinates and reflection intensity. 2. Read the corresponding x and y coordinates according to the set second bird's-eye view range and convert them to the camera coordinate system; 3. Map the height of the LiDAR point cloud to the pixel value according to the corresponding z-axis coordinate. Simultaneously, to visually demonstrate the positional information in the camera coordinate system, the corresponding information of the x=0 and z=40 coordinate axes in the camera coordinate system is marked in the second bird's-eye view in the LiDAR coordinate system, thus converting the second bird's-eye view in the LiDAR coordinate system to the camera coordinate system.

[0126] After rendering the LiDAR point cloud, the dynamic object is modeled as a cuboid. Based on the object's metadata and position information, the coordinates of its eight vertices are calculated. The maximum and minimum values ​​of these eight coordinates are selected and used to form the final two-dimensional coordinate border, which is then plotted on the second bird's-eye view plane to obtain the first bird's-eye view of the dynamic object's trajectory. Since there are many objects in the scene, different colors are used to represent the position information of each object in each frame, forming the final trajectory. The first bird's-eye view of the object's trajectory, rendered using matplotlib, is shown below. Figure 6 As shown.

[0127] This embodiment unifies the processing to the camera coordinate system, enabling the generation of a first bird's-eye view of the dynamic object's trajectory, followed by subsequent data processing.

[0128] In other embodiments, in order to clearly observe the movement of a specific object, a selection interface is provided to specify the object to be drawn and the timestamp, which can effectively grasp the movement trajectory of objects in the scene and facilitate scene editing.

[0129] In some of these embodiments, scene editing of dynamic objects in a simulation scene includes the following steps:

[0130] Perform object deletion, object addition, and / or object trajectory modification on dynamic objects in the simulation scene.

[0131] Specifically, scene editing includes, but is not limited to, deleting objects, adding objects, and modifying object trajectories. These scene editing operations can be performed individually or simultaneously.

[0132] The following explains how to edit each scene:

[0133] For object deletion: Remove the information maintained in the obj_info array. The obj_info array maintains the object's position coordinates and yaw angle at each timestamp in the current scene. By removing the position information of the object to be deleted at the specified timestamp, the object can be skipped during rendering, thus achieving object deletion.

[0134] For adding new objects: Expand the information in the array obj_info under the specified timestamp, add an object ID and the corresponding object position coordinates and yaw angle, and add the corresponding object metadata to the array obj_metadata to realize the addition of objects in the autonomous driving scenario.

[0135] For modifying object trajectories: Modify the object's position coordinates and yaw angle (rotation angle) in the `obj_info` array. In the camera coordinate system, centered on the autonomous vehicle, the x-axis represents left and right (positive right); the y-axis represents up and down (positive down); the z-axis represents forward and backward (positive forward); the rotation angle is the yaw angle around the y-axis, ranging from -180 degrees to 180 degrees, with 0 degrees corresponding to the positive x-axis and -90 degrees to the positive z-axis. This is converted to radians when stored in this array. Changing the position and yaw angle under a specified object ID and timestamp achieves trajectory modification. Since objects in the scene are modeled individually, the edited object models can be directly rendered to obtain a new scene.

[0136] This embodiment enables editing in various scenarios, such as object deletion, object addition, and object trajectory modification, making it convenient to use.

[0137] In some embodiments, the autonomous driving scenario simulation method based on neural radiation fields further includes the following steps:

[0138] Based on a no-reference evaluation metric, the image quality of a scene editing simulation scene is evaluated, and the evaluation results are obtained.

[0139] Specifically, image quality evaluation metrics can be divided into two main categories: referenced and non-referenced. Referenced metrics refer to those where an original image can be used for quality comparison with the new image; for example, PSNR and SSIM are referenced metrics. Non-referenced metrics, on the other hand, do not require an original image for comparison; only one image needs to be evaluated. For scene editing, because the larger the scene modification, the greater the difference from the original image, referenced metrics only compare the differences between the two images and cannot evaluate the rendering effect of the edited scene. It can be argued that related technologies not only have limited scene editing functions and complex processes, but also lack objective evaluation metrics for scene editing effects. Therefore, the use of non-referenced metrics for scene editing is proposed. In this embodiment, Natural Image Quality (NIQE) and the No-Reference Spatial Quality Evaluator (BRISQUE) are used as non-referenced image quality metrics for scene editing.

[0140] NIQE requires no prior information about the distorted image or any training on human opinion scores. It evaluates image quality by analyzing local features and global statistical properties, considering factors such as sharpness, contrast, and noise. BRISQUE also does not require an original reference image. It uses an SVM model to model the relationship between image quality and features, evaluating image quality by analyzing local structural information and gradient statistics. Given the characteristics of these two evaluation metrics, they can be directly used to evaluate the quality of edited scene rendering and verify the editing effect.

[0141] To verify the effectiveness of scene editing, six images, both reconstructed and edited, were evaluated using assessment metrics. SD reconstruction was the method used in this application, while Mars was used as a comparison method. Additionally, to more intuitively compare the results of the assessment metrics, a completely distorted image was selected as a control group.

[0142] Table 1

[0143]

[0144] Table 1 presents the evaluation results. A comparison of the PSRN, SSIM, and LPIPS metrics between the blurred image and the position-edited image in Table 1 shows that the blurred image even achieved a higher evaluation metric. This is because reference-based image quality evaluation metrics such as PSRN compare the image to the original image; the greater the difference, the lower the score. Therefore, in scene editing, the greater the editing modifications to the scene, the lower the reference-based evaluation metrics will be. This proves that traditional reference-based evaluation methods cannot effectively assess the quality of edited scene images. PSRN (Peak Signal-to-Noise Ratio) is a metric for evaluating image quality. SSIM (Structural Similarity Index) is a metric used to quantify the structural similarity between two images. LPIPS (Learned Perceptual Image Patch Similarity) is primarily used to measure the difference between two images.

[0145] The last two columns of unreferenced evaluation metrics in Table 1 show that the lower the image quality, the higher the NIQE and BRISQUE scores. Higher scores indicate lower image quality. The scores for blurred and distorted images are significantly higher than those for the original and reconstructed images. This demonstrates the rationality of using these unreferenced evaluation metrics, providing an objective evaluation of scene editing effects and facilitating accurate quality assessment. Furthermore, the unreferenced evaluation metric values ​​of the edited images are not significantly different from those of the original images, and sometimes even better than those of the reconstructed images, indicating the effectiveness of the scene editing method proposed in this application. The edited scenes maintain high quality.

[0146] This embodiment also provides an autonomous driving scenario simulation device based on neural radiation fields. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below refer to combinations of software and / or hardware that implement a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0147] Figure 7 This is a structural block diagram of the autonomous driving scenario simulation device based on neural radiation fields in this embodiment, as shown below. Figure 7 As shown, the device includes: a simulation unit 210 and an editing unit 220;

[0148] Simulation unit 210 is used to simulate autonomous driving scenarios based on neural radiation fields to obtain simulation scenarios;

[0149] Editing unit 220 is used to process data in the simulation scene to obtain object metadata and corresponding location information;

[0150] With the aid of a 3D spatial view of the simulation scene and a first bird's-eye view of the dynamic object trajectory, scene editing is performed on the dynamic objects in the simulation scene based on the object metadata and corresponding position information.

[0151] The aforementioned device solves the problems of limited scene editing functions and complex processes in related technologies. With the assistance of a three-dimensional spatial view of the simulation scene and a first bird's-eye view of the dynamic object trajectory, it enables accurate and controllable scene editing of dynamic objects in the simulation scene, thus making up for the deficiencies in scene editing.

[0152] In some embodiments, the editing unit 220 is also configured to construct a first bird's-eye view of the dynamic object trajectory in the camera coordinate system based on the lidar point cloud, object metadata, and location information in the simulation scene.

[0153] On the first bird's-eye view of the dynamic object's trajectory, the object's position in the three-dimensional space view is drawn based on the object's metadata.

[0154] In the camera coordinate system, while keeping the object metadata unchanged, the position information is edited to perform scene editing on dynamic objects in the simulation scene;

[0155] The edited object metadata and location information are projected onto the image coordinate system to complete the rendering, resulting in a new simulation scene.

[0156] In some embodiments, the editing unit 220 is also used to draw the lidar point cloud in the simulation scene in a second bird's-eye view in the lidar coordinate system, and to transform the second bird's-eye view in the lidar coordinate system to the camera coordinate system.

[0157] The object's metadata and location information are projected onto the transformed second bird's-eye view to obtain the first bird's-eye view of the dynamic object's trajectory.

[0158] In some embodiments, the editing unit 220 is also used to delete objects, add objects, and / or modify the trajectories of dynamic objects in the simulation scene.

[0159] In some embodiments, the simulation unit 210 is also used to obtain scene images, camera parameters, object label information, and LiDAR point clouds based on the acquisition of autonomous driving scenarios; the scene images correspond to the camera parameters.

[0160] Based on scene images, LiDAR point clouds, and object label information, a 3D Gaussian representation of the autonomous driving scene is determined; the 3D Gaussian representation includes a static background model and multiple dynamic object models.

[0161] Based on the scene image, camera parameters, and a preset loss function, the 3D Gaussian is optimized to obtain the simulation scene.

[0162] In some embodiments, the simulation unit 210 is also configured to decompose the simulation scene into a static background and multiple dynamic objects based on the scene image, the lidar point cloud and the object label information.

[0163] Determine the Gaussian representation of the static background model corresponding to the static background;

[0164] Determine the Gaussian representation of the dynamic object model corresponding to each dynamic object;

[0165] Based on the Gaussian representation of the static background model and the Gaussian representation of each dynamic object model, the 3D Gaussian representation of the autonomous driving scenario is determined.

[0166] In some embodiments, simulation unit 210 is also used to project Gaussian points from three-dimensional space to a two-dimensional plane using a transformation matrix in camera parameters to obtain a rendered image;

[0167] Determine the gradient update parameters based on the rendered image and the scene image;

[0168] Based on gradient-updated parameters, the 3D Gaussian is iteratively optimized until the loss function is satisfied, thus obtaining the simulation scene.

[0169] In some embodiments, the simulation unit 210 is also used to prune and remove three-dimensional Gaussian points in the autonomous driving scenario by using adaptive density control during the iterative optimization of the three-dimensional Gaussian.

[0170] In some of these embodiments, the autonomous driving scenario simulation device based on neural radiation fields also includes an evaluation unit;

[0171] The evaluation unit is used to evaluate the image quality of a scene editing simulation scene based on a no-reference evaluation metric, and obtain the evaluation result.

[0172] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0173] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0174] This embodiment also provides a simulation platform, such as Figure 8 As shown, the simulation platform includes a user layer, a service layer, and a data layer; the service layer is connected to both the user layer and the data layer.

[0175] The user layer is used to assign corresponding user permissions to different users.

[0176] The data layer is used to store data from the simulation platform;

[0177] The service layer is used to execute the steps in any of the above method embodiments under requests from users with different permissions.

[0178] Specifically, the user layer is used to assign different user permissions to different users, thus differentiating user permissions. The data layer stores the simulation platform's data, including but not limited to scene images, camera parameters, object label information, LiDAR point clouds, simulation scenes, evaluation results, scene models, and databases. The scene model is the trained model. The database uses MySQL for the platform's data information and also uses Redis for caching. The service layer, in addition to executing method implementations, provides diverse services. For example, Spring Boot provides backend data services, and Spring Security provides security and authentication, ensuring user security and strict system permission division. Druid and Oshi technologies monitor the platform's service status and collect platform operating parameters to help administrators understand the platform's condition. The autonomous driving scene simulation editing function provides scene simulation and editing capabilities.

[0179] In some embodiments, the simulation platform further includes a presentation layer and an interface layer;

[0180] The interface layer is used to receive data requests from the presentation layer, call the relevant services in the service layer, and return the request results corresponding to the data requests to the presentation layer.

[0181] The presentation layer is used to display the results of the request.

[0182] Specifically, the presentation layer displays the request results and provides visualization capabilities. The scene simulation editing interface primarily showcases the effects of autonomous driving scene editing, providing visualizations of scene simulation, scene editing, multi-view assistance, and simulation editing evaluation metrics. The web platform management interface is mainly for administrators, enabling user management, system monitoring, and simulation data management via the web.

[0183] The interface layer receives data requests from the presentation layer and calls relevant services in the service layer, returning the request results to the presentation layer. It provides the presentation layer with relevant information such as scene simulation and editing. Meanwhile, the services are deployed using containerization technology and run in Docker containers to achieve relative isolation between services, while also enabling communication and interaction between services through interface calls.

[0184] The simulation platform uses a B / S-based architecture and a SpringBoot+Vue front-end and back-end separation framework to build an autonomous driving simulation visualization management platform.

[0185] The backend uses Spring Boot, which significantly improves build and deployment speed through annotations and configuration files. It embeds web containers such as Tomcat, Jetty, and Undertow, eliminating the need for WAR deployment; it can run in a server environment using JAR files. For security control, the Spring Security framework is used, implementing user login authentication, resource permission authorization, and other management functions. It also provides security protection, including defense against common web attacks such as cross-site scripting and session attacks. The data layer uses MyBatis to operate on MySQL, maintaining the three-tier MVC architecture. The view layer handles request reception and response, the service layer implements system business logic, and the persistence layer (DAO) directly performs database operations. This three-tier design reduces coupling between layers.

[0186] The front-end is built using Vue's scaffolding tool, package management is handled by npm, and element-ui is used as the front-end UI component library. Routing is handled by vue-router, a front-end routing tool provided by Vue, enabling page routing control, partial page refreshes, and on-demand loading, thus building a single-page application and achieving front-end / back-end separation.

[0187] The simulation platform employs a one-click automated deployment approach. Using Jenkins and Git, Jenkins directly pulls and builds the repository code. On the backend, Jenkins integrates a Maven environment, allowing code from the repository to be packaged into JAR files and sent to the server via shell command configuration. On the frontend, Jenkins integrates a Node.js environment, packaging the code into a dist package and placing it directly on Nginx for reverse proxying. The platform server uses Docker containers for deployment, with automated deployment achieved through shell script commands configured within Jenkins.

[0188] This embodiment demonstrates the construction of a simulation platform based on front-end and back-end separation Web development technology, enabling unified management of simulation data and providing an intuitive display of simulation editing effects.

[0189] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0190] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0191] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0192] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0193] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A method for simulating autonomous driving scenarios based on neural radiation fields, characterized in that, include: The simulation scenario is obtained by simulating the autonomous driving scenario based on the neural radiation field; The camera is used to acquire images of autonomous driving scenarios; The simulation scene is processed to obtain object metadata and corresponding location information; With the aid of a three-dimensional spatial view of the simulation scene and a first bird's-eye view of the dynamic object trajectory, scene editing is performed on the dynamic objects in the simulation scene based on the object metadata and the corresponding position information. This includes: Based on the lidar point cloud, object metadata, and location information in the simulation scene, a first bird's-eye view of the dynamic object trajectory is constructed in the camera coordinate system. On the first bird's-eye view of the trajectory of the dynamic object, the position of the dynamic object in the three-dimensional space view is drawn based on the object metadata. In the camera coordinate system, while keeping the object metadata unchanged, the position information is edited to perform scene editing on the dynamic objects in the simulation scene; The edited object metadata and location information are projected onto the image coordinate system to complete rendering, resulting in a new simulation scene.

2. The autonomous driving scenario simulation method based on neural radiation fields according to claim 1, characterized in that, Based on the lidar point cloud, object metadata, and location information in the simulation scene, a first bird's-eye view of the dynamic object trajectory is constructed in the camera coordinate system, including: The lidar point cloud in the simulation scene is drawn in the second bird's-eye view of the lidar coordinate system, and the second bird's-eye view under the lidar coordinate system is transformed to the camera coordinate system. The object metadata and the location information are projected onto the transformed second bird's-eye view to obtain a first bird's-eye view of the dynamic object trajectory.

3. The autonomous driving scenario simulation method based on neural radiation fields according to claim 1, characterized in that, Scene editing for dynamic objects in the simulation scene includes: The simulation scene includes the ability to delete objects, add objects, and / or modify the trajectories of dynamic objects.

4. The autonomous driving scenario simulation method based on neural radiation fields according to claim 1, characterized in that, The simulation of autonomous driving scenarios is based on neural radiation fields, resulting in simulation scenarios including: Based on the data collected from autonomous driving scenarios, scene images, camera parameters, object label information, and LiDAR point clouds are obtained; the scene images correspond to the camera parameters. Based on the scene image, the LiDAR point cloud, and the object label information, a three-dimensional Gaussian representation of the autonomous driving scene is determined; the three-dimensional Gaussian representation includes a static background model and multiple dynamic object models. The simulation scene is obtained by optimizing the 3D Gaussian based on the scene image, the camera parameters, and the preset loss function.

5. The autonomous driving scenario simulation method based on neural radiation fields according to claim 4, characterized in that, Based on the scene image, the LiDAR point cloud, and the object label information, a three-dimensional Gaussian representation of the autonomous driving scene is determined, including: Based on the scene image, the lidar point cloud, and the object label information, the simulation scene is decomposed into a static background and multiple dynamic objects; Determine the Gaussian representation of the static background model corresponding to the static background; Determine the Gaussian representation of the dynamic object model corresponding to each of the dynamic objects; The three-dimensional Gaussian representation of the autonomous driving scene is determined based on the Gaussian representation of the static background model and the Gaussian representation of each of the dynamic object models.

6. The autonomous driving scenario simulation method based on neural radiation fields according to claim 4, characterized in that, Based on the scene image, the camera parameters, and a preset loss function, the 3D Gaussian optimization is performed to obtain the simulation scene, including: The Gaussian points are projected from three-dimensional space onto a two-dimensional plane using the transformation matrix in the camera parameters to obtain the rendered image; The gradient update parameters are determined based on the rendered image and the scene image; Based on the gradient update parameters, the three-dimensional Gaussian is iteratively optimized until the loss function is satisfied, thus obtaining the simulation scene.

7. The autonomous driving scenario simulation method based on neural radiation fields according to claim 6, characterized in that, The method further includes: During the iterative optimization of the three-dimensional Gaussian, adaptive density control is used to prune and remove the three-dimensional Gaussian points in the autonomous driving scenario.

8. The autonomous driving scenario simulation method based on neural radiation fields according to claim 1, characterized in that, The method further includes: Based on a no-reference evaluation metric, the image quality of the simulated scene in the scene editing is evaluated, and the evaluation results are obtained.

9. An autonomous driving scenario simulation device based on neural radiation fields, characterized in that, include: Simulation unit and editing unit; The simulation unit is used to simulate autonomous driving scenarios based on neural radiation fields to obtain simulation scenarios; The camera is used to acquire images of autonomous driving scenarios; The editing unit is used to process data in the simulation scene to obtain object metadata and corresponding location information; With the aid of a three-dimensional spatial view of the simulation scene and a first bird's-eye view of the dynamic object trajectory, scene editing is performed on the dynamic objects in the simulation scene based on the object metadata and the corresponding position information. This includes: Based on the lidar point cloud, object metadata, and location information in the simulation scene, a first bird's-eye view of the dynamic object trajectory is constructed in the camera coordinate system. On the first bird's-eye view of the trajectory of the dynamic object, the position of the dynamic object in the three-dimensional space view is drawn based on the object metadata. In the camera coordinate system, while keeping the object metadata unchanged, the position information is edited to perform scene editing on the dynamic objects in the simulation scene; The edited object metadata and location information are projected onto the image coordinate system to complete rendering, resulting in a new simulation scene.

10. A simulation platform, characterized in that, include: User layer, service layer, and data layer; The service layer is connected to both the user layer and the data layer. The user layer is used to assign corresponding user permissions to different users; The data layer is used to store data from the simulation platform; The service layer is used to execute the autonomous driving scenario simulation method based on neural radiation fields according to any one of claims 1 to 8, under requests from users with different permissions.

11. The simulation platform according to claim 10, characterized in that, The simulation platform also includes: a presentation layer and an interface layer; The interface layer is used to receive data requests from the presentation layer, call the relevant services of the service layer, and return the request result corresponding to the data request to the presentation layer. The presentation layer is used to display the result of the request.

Images