Automatic driving scene simulation method and device based on neural radiation field and simulation platform

Abstract

The invention relates to an automatic driving scene simulation method and device based on a neural radiation field and a simulation platform, and the method comprises the steps: carrying out the simulation of an automatic driving scene based on the neural radiation field, and obtaining a simulation scene; performing data processing on the simulation scene to obtain object metadata and corresponding position information; and performing scene editing on the dynamic object in the simulation scene according to the object metadata and the corresponding position information based on the assistance of the three-dimensional space view of the simulation scene and the first aerial view of the dynamic object track. Through the scene editing method and device, the problems of limited scene editing function and complex process in the prior art are solved, accurate and controllable scene editing of the dynamic object in the simulation scene is realized based on the assistance of the three-dimensional space view of the simulation scene and the first aerial view of the dynamic object track, and the defects in the scene editing aspect are overcome.

Description

Technical Field

[0001] The present application relates to the technical field of computer vision scene simulation, and in particular to an autonomous driving scene simulation method, device and simulation platform based on neural radiation fields. Background Art

[0002] As autonomous driving technology continues to develop, the demand for rigorous testing of safe autonomous vehicles is also increasing. For autonomous vehicles to be able to meet the requirements of driving in the real natural world, they need to be rigorously tested in all aspects. In the testing of autonomous vehicles, key boundary scenarios are crucial for safe driving. However, there is a long tail effect in the real world, and boundary scenarios in driving are usually more extreme, which makes testing in the real world very expensive and dangerous. Therefore, the idea of ​​applying simulation in the field of autonomous driving has been proposed.

[0003] The current autonomous driving scene simulation solution based on neural radiation fields is to model the moving objects and static background in the autonomous driving scene separately, learn the movement and appearance changes of the dynamic objects, and then reconstruct the autonomous driving scene. However, the scene editing function of the autonomous driving scene reconstructed by this solution is limited and the process is complicated. At present, there is a lack of a scene editing solution with clear process, accuracy and controllability. Summary of the invention

[0004] In this embodiment, a method, device and simulation platform for autonomous driving scene simulation based on neural radiation field are provided to solve the problems of limited scene editing functions and complicated processes in related technologies.

[0005] In the first aspect, in this embodiment, a method for simulating an autonomous driving scene based on a neural radiation field is provided, including:

[0006] Simulate the autonomous driving scenario based on the neural radiation field to obtain a simulation scenario;

[0007] Performing data processing on the simulation scene to obtain object metadata and corresponding position information;

[0008] Based on the three-dimensional space view of the simulation scene and the assistance of the first bird's-eye view of the trajectory of the dynamic object, scene editing is performed on the dynamic object in the simulation scene according to the object metadata and the corresponding position information.

[0009] In some embodiments, based on the three-dimensional spatial view of the simulation scene and the assistance of the first bird's-eye view of the dynamic object trajectory, scene editing is performed on the dynamic object in the simulation scene according to the object metadata and the corresponding position information, including:

[0010] Constructing a first bird's-eye view of the trajectory of the dynamic object in a camera coordinate system according to the laser radar point cloud, the object metadata, and the position information in the simulation scene;

[0011] On the first bird's-eye view of the dynamic object trajectory, drawing the object position of the dynamic object in the three-dimensional space view based on the object metadata;

[0012] In the camera coordinate system, the metadata of the object is kept unchanged, the position information is edited, and scene editing is performed on the dynamic object in the simulation scene;

[0013] The edited object metadata and the position information are projected into the image coordinate system to complete rendering, thereby obtaining the new simulation scene.

[0014] In some embodiments, constructing a first bird's-eye view of the dynamic object trajectory in a camera coordinate system according to the laser radar point cloud, the object metadata, and the position information in the simulation scene includes:

[0015] Drawing the laser radar point cloud in the simulation scene in a second bird's-eye view of the laser radar coordinate system, and converting the second bird's-eye view in the laser radar coordinate system to the camera coordinate system;

[0016] The object metadata and the position information are projected into the converted second bird's-eye view to obtain a first bird's-eye view of the dynamic object trajectory.

[0017] In some embodiments, scene editing of dynamic objects in the simulation scene includes:

[0018] Deleting, adding, and / or modifying the trajectory of dynamic objects in the simulation scene.

[0019] In some of the embodiments, the autonomous driving scene is simulated based on the neural radiation field to obtain a simulated scene, including:

[0020] Based on the collection of the autonomous driving scene, a scene image, camera parameters, object label information and a lidar point cloud are obtained; the scene image corresponds to the camera parameters;

[0021] Determine a three-dimensional Gaussian representation of the autonomous driving scene according to the scene image, the lidar point cloud, and the object label information; the three-dimensional Gaussian includes a static background model and a plurality of dynamic object models;

[0022] The three-dimensional Gaussian is optimized according to the scene image, the camera parameters and a preset loss function to obtain the simulation scene.

[0023] In some of the embodiments, determining a three-dimensional Gaussian representation of the autonomous driving scene according to the scene image, the lidar point cloud, and the object label information includes:

[0024] Decomposing the simulation scene into a static background and a plurality of dynamic objects according to the scene image, the laser radar point cloud and the object label information;

[0025] determining a Gaussian representation of a static background model corresponding to the static background;

[0026] determining a Gaussian representation of a dynamic object model corresponding to each of the dynamic objects;

[0027] A three-dimensional Gaussian representation of the autonomous driving scene is determined based on the Gaussian representation of the static background model and the Gaussian representation of each of the dynamic object models.

[0028] In some embodiments, optimizing the three-dimensional Gaussian according to the scene image, the camera parameters and a preset loss function to obtain the simulation scene includes:

[0029] The Gaussian points are projected from the three-dimensional space to the two-dimensional plane through the transformation matrix in the camera parameters to obtain a rendered image;

[0030] Determining a gradient update parameter according to the rendered image and the scene image;

[0031] Based on the gradient update parameters, the three-dimensional Gaussian is iteratively optimized until the loss function is satisfied to obtain the simulation scene.

[0032] In some embodiments, the method further comprises:

[0033] In the process of iteratively optimizing the three-dimensional Gaussian, adaptive density control is used to clip and eliminate the three-dimensional Gaussian points in the autonomous driving scene.

[0034] In some embodiments, the method further comprises:

[0035] Based on the non-reference evaluation index, the image quality of the simulation scene in the scene editing is evaluated to obtain an evaluation result.

[0036] In a second aspect, in this embodiment, an autonomous driving scene simulation device based on a neural radiation field is provided, comprising: a simulation unit and an editing unit;

[0037] The simulation unit is used to simulate the autonomous driving scene based on the neural radiation field to obtain a simulation scene;

[0038] The editing unit is used to perform data processing on the simulation scene to obtain object metadata and corresponding position information;

[0039] Based on the three-dimensional space view of the simulation scene and the assistance of the first bird's-eye view of the trajectory of the dynamic object, scene editing is performed on the dynamic object in the simulation scene according to the object metadata and the corresponding position information.

[0040] In a third aspect, a simulation platform is provided in this embodiment, including: a user layer, a service layer and a data layer; the service layer is connected to the user layer and the data layer respectively;

[0041] The user layer is used to grant corresponding user rights to different users;

[0042] The data layer is used to store the data of the simulation platform;

[0043] The service layer is used to execute the autonomous driving scenario simulation method based on neural radiation field described in the first aspect under the request of different user permissions.

[0044] In some of the embodiments, the simulation platform further includes: a presentation layer and an interface layer;

[0045] The interface layer is used to receive the data request of the presentation layer, call the relevant services of the service layer, and return the request result corresponding to the data request to the presentation layer;

[0046] The presentation layer is used to display the request result.

[0047] Compared with the related art, the autonomous driving scene simulation method, device and simulation platform based on neural radiation field provided in this embodiment obtain a simulation scene by simulating the autonomous driving scene based on the neural radiation field; perform data processing on the simulation scene to obtain object metadata and corresponding position information; based on the three-dimensional space view of the simulation scene and the first bird's-eye view of the dynamic object trajectory, the dynamic objects in the simulation scene are edited according to the object metadata and the corresponding position information, which solves the problems of limited scene editing function and complicated process in the related art, and realizes accurate and controllable scene editing of dynamic objects in the simulation scene with the assistance of the three-dimensional space view of the simulation scene and the first bird's-eye view of the dynamic object trajectory, making up for the deficiencies in scene editing.

[0048] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0050] Figure 1 It is a hardware structure block diagram of a terminal device of an autonomous driving scene simulation method based on a neural radiation field provided in an embodiment of the present application;

[0051] Figure 2 is a flow chart of an autonomous driving scene simulation method based on neural radiation field provided in one embodiment of the present application;

[0052] Figure 3 is a flowchart of step S210;

[0053] Figure 4 is a schematic diagram of a scene editing process provided by an embodiment of the present application;

[0054] Figure 5 is a schematic diagram of a three-dimensional space view provided by an embodiment of the present application;

[0055] Figure 6 is a schematic diagram of a first bird's-eye view of a dynamic object trajectory provided by an embodiment of the present application;

[0056] Figure 7 It is a structural block diagram of an autonomous driving scene simulation device based on neural radiation field provided in one embodiment of the present application;

[0057] Figure 8 It is a structural block diagram of a simulation platform provided in one embodiment of the present application.

[0058] In the figure: 102, processor; 104, memory; 106, transmission device; 108, input and output device; 210, simulation unit; 220, editing unit. DETAILED DESCRIPTION

[0059] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0060] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0061] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 is a hardware structure block diagram of a terminal of the method for simulating an autonomous driving scenario based on a neural radiation field in this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0062] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for simulating an autonomous driving scenario based on a neural radiation field in the present embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0063] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0064] In this embodiment, a method for simulating an autonomous driving scenario based on a neural radiation field is provided. Figure 2 is a flow chart of the method for simulating an autonomous driving scenario based on a neural radiation field of this embodiment. Figure 2 As shown, the process includes the following steps:

[0065] Step S210, simulating the autonomous driving scene based on the neural radiation field to obtain a simulation scene;

[0066] Step S220, performing data processing on the simulation scene to obtain object metadata and corresponding position information;

[0067] Step S230 , based on the three-dimensional space view of the simulation scene and the assistance of the first bird's-eye view of the dynamic object trajectory, scene editing is performed on the dynamic object in the simulation scene according to the object metadata and the corresponding position information.

[0068] It should be noted that Neural Radiance Fields (NeRF) refers to using a multi-layer perceptron neural network to implicitly learn a static 3D scene to achieve arbitrary new perspective synthesis (rendering) of complex scenes. In the field of autonomous driving, based on neural radiation fields, dynamic objects and static backgrounds in autonomous driving scenes are modeled separately in simulation scenes, and the movement and appearance changes of dynamic objects are learned to obtain simulation scenes; then there are corresponding static background models and dynamic object models in the simulation scene. Among them, one neural radiation field can correspond to one model. It should be noted that during the simulation process, it may include the implementation of data processing in step S220; it may also be a direct call to a third-party platform to obtain the simulation scene and the data collected from the autonomous driving scene.

[0069] Among them, the image processing algorithm or neural network model can be used to process the simulation scene data to advance the object metadata and corresponding position information from the object tag information. Among them, the object metadata can be maintained by the array obj_metadata. The array obj_metadata stores the unique ID, length, width and height metadata information of all objects in the current simulation scene. The position information is maintained by the array obj_info, which is a three-dimensional array that stores the object position coordinates (the coordinates on the x-axis, y-axis and z-axis of the three-dimensional space in the camera coordinate system) and the yaw angle in each timestamp.

[0070] Since the three-dimensional space view of the simulation scene is in the camera coordinate system, the first bird's-eye view of the dynamic object trajectory can be associated with the lidar coordinate system and the camera coordinate system, and the object metadata and the corresponding position information are in the camera coordinate system; therefore, the object metadata and the corresponding position information can be projected to the first bird's-eye view in the lidar coordinate system to complete the editing, and then rendered to the three-dimensional space view, so as to realize scene editing of the dynamic objects in the simulation scene, thereby realizing accurate and controllable scene editing of the dynamic objects in the simulation scene, making up for the deficiencies in scene editing.

[0071] The following is a detailed description of each of the above steps:

[0072] In some of the embodiments, Figure 3 As shown, the step S210 of simulating the autonomous driving scene based on the neural radiation field to obtain the simulation scene includes the following steps:

[0073] Step S211, based on the collection of the autonomous driving scene, a scene image, camera parameters, object label information and a lidar point cloud are obtained; the scene image corresponds to the camera parameters;

[0074] Step S212, determining a three-dimensional Gaussian representation of the autonomous driving scene according to the scene image, the lidar point cloud, and the object label information; the three-dimensional Gaussian includes a static background model and multiple dynamic object models;

[0075] Step S213, optimizing the three-dimensional Gaussian according to the scene image, camera parameters and a preset loss function to obtain a simulation scene.

[0076] This embodiment can be considered as a scene simulation process, which can be divided into data collection, model building and optimization processes.

[0077] Among them, for data collection: image sensors can be used to collect autonomous driving scenes to obtain scene images, camera parameters and object label information; scene images correspond to camera parameters. If the image sensor is a camera, the camera parameters include the camera's intrinsic and extrinsic parameters; the intrinsic parameters are the camera's optical center, focal length, pixel scaling and translation parameters, and the extrinsic parameters are the camera's rotation matrix and translation vector, that is, the transformation matrix, which determines the camera's position and posture. The scene image can be an RGB image. A laser radar is used to collect autonomous driving scenes to obtain a laser radar point cloud; the laser radar point cloud can be an SFM point cloud.

[0078] Among them, for model establishment: decompose the simulation scene into a static background and multiple dynamic objects. For the static background, Gaussian is used to represent the corresponding static background model; for each dynamic object, Gaussian is used to represent the corresponding dynamic object model, then the three-dimensional Gaussian representation of the autonomous driving scene can be determined by combining these Gaussians. Among them, anisotropic three-dimensional Gaussian (3D Gaussian Splatting) is used as a high-quality neural radiation field expression method, and the entire scene space is represented as Gaussian points. The geometric shapes in the scene are modeled as three-dimensional Gaussian functions. In the rendering of three-dimensional Gaussian, the three-dimensional Gaussian in the scene can be regarded as a colored ellipsoid. The ellipsoid is projected onto the projection plane where the corresponding posture is located at a specific angle. This step is called splatting. An ellipsoid is projected onto a plane to obtain an ellipse, and then the opacity can be obtained by calculating the distance between the pixel to be solved and the center of the ellipse (the closer it is, the more opaque it is). The color of each ellipsoid is independent of the distance. The color is synthesized by α mixing. In other embodiments, other methods can also be used to represent the autonomous driving scene, which is not limited.

[0079] Among them, for optimization: relevant optimization methods can be used to optimize the three-dimensional Gaussian according to the scene image, camera parameters and preset loss function to obtain a simulation scene, and there is no restriction on this. For example: using three-dimensional Gaussian training and rendering based on block strategy, gradient iterative optimization, etc.

[0080] Through this embodiment, the view-related appearance in the scene can be correctly captured, thereby improving the simulation effect of the autonomous driving scene.

[0081] In some embodiments, determining the three-dimensional Gaussian representation of the autonomous driving scene according to the scene image, the lidar point cloud, and the object label information in step S212 includes the following steps:

[0082] Decompose the simulation scene into a static background and multiple dynamic objects based on the scene image, LiDAR point cloud, and object label information;

[0083] determining a Gaussian representation of a static background model corresponding to the static background;

[0084] determining a Gaussian representation of a dynamic object model corresponding to each dynamic object;

[0085] A three-dimensional Gaussian representation of the autonomous driving scene is determined based on the Gaussian representation of the static background model and the Gaussian representation of each dynamic object model.

[0086] Specifically, the simulation scene is decomposed into a static background and multiple dynamic objects according to the scene image, the lidar point cloud and the object label information.

[0087] For the static background model with a static background, the three-dimensional covariance matrix is ​​expressed as:

[0088] ∑=RSS T R T ;

[0089] In the formula, R represents the rotation matrix; S represents the scaling matrix; S T and R T represents the transpose of the corresponding matrix; ∑ is a real symmetric matrix with semi-positive definiteness, and is a three-dimensional covariance matrix representing the shape of Gaussian points in space, which can be orthogonally diagonalized. Since it is semi-positive definite, the diagonal matrix representing the eigenvalues ​​can be split.

[0090] Perform eigenvalue decomposition on the three-dimensional covariance matrix to obtain the rotation matrix, scaling matrix, and corresponding transpose; in order to maintain semi-positive definiteness, choose to use quaternion q to represent the rotation matrix, extract the diagonal elements of the scaling matrix to obtain the scaling factor s, and use the rotation quaternion q and scaling factor s to simplify the calculation when updating the gradient. After the optimization is completed, it is converted back to the representation of the three-dimensional covariance matrix. Then the Gaussian representation of the static background model is:

[0091] G S ={μ,q,s,α,c};

[0092] Where μ is the position coordinate (x, y, z); α represents opacity; and c represents color.

[0093] Dynamic objects are extracted through the lidar point cloud (SFM point cloud) and object label information, and the SFM point cloud is used as the initialization input of the dynamic object model. The time dimension is added on the basis of the static background model, and the opacity and position are expressed as time-related functions to express the effect of object movement at different timestamps. Then the Gaussian representation of the dynamic object model is:

[0094] G d (t)={μ(t),q,s,α(t),c}.

[0096] Where: μ(t) represents the position coordinate that changes with time; α(t) represents the opacity that changes with time.

[0097] Two parts of data are extracted from the object label information, namely the object metadata and position information of the object. The transformation of the object position coordinate system requires the transformation matrix (external parameter) in the camera parameters. The camera projection matrix, distortion correction matrix, matrix from lidar coordinates to camera coordinate system, and matrix from IMU coordinate system to lidar coordinates are extracted from the intrinsic parameters of the camera parameters. Then the autonomous driving scene can be finally represented as a combination of a static background and multiple dynamic object models, and its three-dimensional Gaussian representation is:

[0098] G c =∑ <O,G d ,M,P,T>+G s ;

[0099] Where O represents object metadata such as object length, width, height, and unique ID; Gd represents the dynamic object model; M represents the camera parameters including the projection transformation matrix between different coordinate systems; P represents the posture information including the object position coordinates and yaw angle (belonging to the position information); T represents the timestamp (belonging to the position information); Gs represents the static background model.

[0100] Through this embodiment, the three-dimensional Gaussian representation of the autonomous driving scene is realized, and the structure of the three-dimensional Gaussian is simplified to facilitate subsequent model optimization.

[0101] In some embodiments, the step S213 optimizes the three-dimensional Gaussian according to the scene image, the camera parameters and the preset loss function to obtain the simulation scene, including the following steps:

[0102] The Gaussian points are projected from the three-dimensional space to the two-dimensional plane through the transformation matrix in the camera parameters to obtain the rendered image;

[0103] Determine gradient update parameters according to the rendered image and the scene image;

[0104] Based on the gradient update parameters, the three-dimensional Gaussian is iteratively optimized until the loss function is satisfied and the simulation scene is obtained.

[0105] Specifically, after obtaining the representation of the three-dimensional Gaussian, the Gaussian points are projected from the three-dimensional space to the two-dimensional plane through the transformation matrix in the camera parameters, and the rendered image is obtained using the block-based fast differentiable rendering algorithm. The loss function is calculated based on the rendered image and the scene image, and then back propagation is performed to calculate the gradient update parameters.

[0106] In the iterative optimization process, the stochastic gradient descent method is used until the loss function is satisfied and the simulation scene is obtained. Specifically, the sigmoid activation function is used to constrain the opacity so that it is in the range of [0,1) and obtain a smooth gradient. For the scale of the covariance, the exponential activation function is used to constrain it. Then the final loss function can be expressed as:

[0107] lι=(1-λ)l1+λl ssim ;

[0108] In the formula, l1 represents the loss of image color; l ssim represents the SSIM loss of image structure similarity; λ represents the control hyperparameter, which is generally set to 0.2.

[0109] Through this embodiment, the iterative optimization efficiency is improved while the accuracy of scene simulation is improved.

[0110] In some of the embodiments, the method for simulating an autonomous driving scenario based on a neural radiation field further includes the following steps:

[0111] During the iterative optimization of the 3D Gaussian, adaptive density control is used to clip and eliminate the 3D Gaussian points in the autonomous driving scene.

[0112] Specifically, in the iterative optimization gradient update process, adaptive density control is used to crop and remove the three-dimensional Gaussian points in the scene, add Gaussian points to the non-repeated areas of the simulation scene reconstruction, and delete Gaussian points from the over-reconstructed areas to ensure that the number of Gaussian points in the simulation scene is relatively stable. Since autonomous driving is faced with large scenes of urban scale, the positions of most points in urban scenes are actually far away from the center of the borderless scene. In order to represent the scene with fewer points while maintaining the accuracy of the simulation; in this embodiment, adaptive density control controls the size of the three-dimensional Gaussian points through a scale factor, using larger points for distant areas and smaller points for nearby areas.

[0113] Through this embodiment, large-scale scenes can be processed in parallel, and processing efficiency can be improved. Ultimately, the method can provide high-quality visual effects while rendering in real time, and adapt to available computing resources, making it possible to capture and render large-scale scenes.

[0114] In some embodiments, the step S230 of performing scene editing on the dynamic objects in the simulation scene according to the object metadata and the corresponding position information with the assistance of the three-dimensional spatial view of the simulation scene and the first bird's-eye view of the dynamic object trajectory includes the following steps:

[0115] Step S231, constructing a first bird's-eye view of the dynamic object trajectory in a camera coordinate system according to the laser radar point cloud, object metadata, and position information in the simulation scene;

[0116] Step S232, on the first bird's-eye view of the dynamic object trajectory, drawing the object position of the dynamic object in the three-dimensional space view based on the object metadata;

[0117] Step S233, in the camera coordinate system, keep the object metadata unchanged, edit the position information, and perform scene editing on the dynamic object in the simulation scene;

[0118] Step S234 , projecting the edited object metadata and position information into the image coordinate system to complete rendering and obtain a new simulation scene.

[0119] Specifically, Figure 4 As shown, on the basis of obtaining the object metadata and the corresponding position information after data processing, scene editing can be considered to also include multi-view assistance and position editing. Among them, multi-view assistance refers to the assistance of the three-dimensional space view and the first bird's-eye view. The three-dimensional space view is constructed in the camera coordinate system and carries the simulation scene. Since the lidar point cloud is in the lidar coordinate system, and the object metadata and position information are in the camera coordinate system, since they are located in different coordinate systems, the lidar point cloud needs to be converted to the camera coordinate system to construct a first bird's-eye view of the dynamic object trajectory. In other embodiments, constructing the first bird's-eye view of the dynamic object trajectory can also be achieved in other ways, and there is no limitation to this.

[0120] Then, on the basis of obtaining the first bird's-eye view for constructing the trajectory of the dynamic object, the object metadata in the camera coordinate system is used to draw the object position of the dynamic object in the three-dimensional space view, so as to more intuitively understand the three-dimensional space situation of the scene. The combined use of the two-dimensional first bird's-eye view and the three-dimensional space view can fully understand the situation of the scene and intuitively display the state changes of objects before and after scene editing. In this embodiment, the three-dimensional space view can be drawn using the eight vertex coordinates after the object is modeled as a cuboid, and the object ID and the number of frames to which each cuboid belongs are marked. In addition, in order to be consistent with the normal three-dimensional space coordinates, the data information of the y and z axes are exchanged in the camera coordinate system, so that the three-dimensional coordinate system is expanded in the form of xyz. The three-dimensional space view of the scene drawn using matplotlib is as follows: Figure 5 shown.

[0121] When editing a scene, in the camera coordinate system, keep the object metadata unchanged and edit the position information (modify the data in the array obj_info to generate a new scene). Since the position information is edited directly in the original camera coordinate system (modify the array obj_info), the transformation matrix provided by the camera parameters is used to project the edited object metadata and position information to the image coordinate system to complete the subsequent rendering and obtain a new simulation scene; so as to realize scene editing of dynamic objects in the simulation scene.

[0122] Through this embodiment, since the position information of the camera coordinate system has the meaning of real objects, it is world information in the real world, and the data unit is meter, precise scene editing of the position can be achieved; editing directly in the camera coordinate system can intuitively compare the changes in the position of the object before and after editing; editing in the camera coordinate system makes the object position more accurate and easy to operate, and the required trajectory can be directly edited.

[0123] In some embodiments, the step S231 of constructing a first bird's-eye view of the trajectory of the dynamic object in the camera coordinate system according to the laser radar point cloud, object metadata, and position information in the simulation scene includes the following steps:

[0124] Draw the laser radar point cloud in the simulation scene in the second bird's-eye view of the laser radar coordinate system, and transform the second bird's-eye view in the laser radar coordinate system into the camera coordinate system;

[0125] The object metadata and position information are projected into the transformed second bird's-eye view to obtain the first bird's-eye view of the dynamic object trajectory.

[0126] Specifically, for the editing of dynamic object trajectories, the first bird's-eye view needs to be used to intuitively and clearly describe the movement trajectory information of objects in the scene. The first bird's-eye view is a bird's-eye view corresponding to the dynamic object trajectory in the camera coordinate system, and the first bird's-eye view includes the object metadata and position information in the camera coordinate system. The second bird's-eye view corresponds to the bird's-eye view of the lidar point cloud in the lidar coordinate system. In order to construct the first bird's-eye view, the lidar point cloud (located in the lidar coordinate system), object metadata and position information (located in the camera coordinate system) are needed, so the lidar point cloud needs to be converted to the camera coordinate system, using the transformation matrix from the lidar coordinate system to the camera coordinate system (in the camera parameters). The process of drawing the lidar point cloud in the second bird's-eye view of the lidar coordinate system is as follows: 1. Read the lidar point cloud, and the storage format of the lidar point cloud is n×4; where 4 represents that the array dimension of the point cloud storage is n×4, that is, there are n point clouds, and each point cloud has four parameters including three-dimensional coordinates and reflection intensity. 2. Read the corresponding x-coordinate and y-coordinate according to the set second bird's-eye view range and convert them to the camera coordinate system; 3. Map the height of the lidar point cloud to the pixel value according to the corresponding z-axis coordinate. At the same time, in order to intuitively see the position information in the camera coordinate system, the corresponding information of the two coordinate axes x=0 and z=40 in the camera coordinate system is marked in the second bird's-eye view of the lidar coordinate system, and the second bird's-eye view in the lidar coordinate system is converted to the camera coordinate system.

[0127] After the lidar point cloud is drawn, the dynamic object is modeled as a cuboid. The coordinates of the eight vertices are calculated based on the object metadata and position information. The maximum and minimum values ​​of the eight coordinates are selected and combined into a final two-dimensional coordinate frame and drawn on the second bird's-eye view plane to obtain the first bird's-eye view of the dynamic object's trajectory. Since there are many objects in the scene, different objects are drawn with different colors in each frame to form the final trajectory route. The first bird's-eye view of the object trajectory drawn using matplotlib is shown in the figure below. Figure 6 shown.

[0128] Through this embodiment, the processing is unified in the camera coordinate system to achieve the generation of the first bird's-eye view of the trajectory of the dynamic object and the subsequent data processing.

[0129] In other embodiments, in order to clearly observe the movement of a specific object, a selection interface is provided to specify the object to be drawn and a timestamp, which can well grasp the movement trajectory of the object in the scene and provide convenience for scene editing.

[0130] In some embodiments, scene editing of dynamic objects in a simulation scene includes the following steps:

[0131] Delete, add, and / or modify the trajectory of dynamic objects in the simulation scene.

[0132] Specifically, scene editing includes but is not limited to object deletion, object addition, object trajectory modification, etc. These scene edits can be performed separately or simultaneously.

[0133] The following describes how to edit each scene:

[0134] For object deletion: remove the information maintained in the array obj_info. The array obj_info maintains the object position coordinates and yaw angles at each timestamp in the current scene. Remove the position information of the object to be deleted at the specified timestamp, so that the object can be skipped during rendering and the object can be deleted.

[0135] For newly added objects: expand the information in the array obj_info at the specified timestamp, add an object ID and the corresponding object position coordinates and yaw angle, add the object metadata corresponding to the newly added object in the array obj_metadata, and realize the addition of objects in the autonomous driving scene.

[0136] For object trajectory modification: modify the object position coordinates and yaw angle (rotation angle) in the array obj_info. In the camera coordinate system, with the self-driving car as the center, the x-axis represents left and right, with the right as the positive direction; the y-axis represents up and down, with downward as the positive direction; the z-axis represents front and back, with forward as the positive direction; the rotation angle is the yaw angle around the y-axis, ranging from -180 degrees to 180 degrees, with the positive direction of the x-axis corresponding to 0 degrees and the positive direction of the z-axis corresponding to -90 degrees, which are converted to radians when stored in the array. Change the position and yaw angle of the specified object ID and timestamp to achieve trajectory modification. Since the objects in the scene are modeled separately, the edited object models can be directly rendered to obtain a new scene.

[0137] Through this embodiment, various scene editing such as object deletion, object addition, object trajectory modification, etc. can be realized, which is convenient to use.

[0138] In some of the embodiments, the method for simulating an autonomous driving scenario based on a neural radiation field further includes the following steps:

[0139] Based on the non-reference evaluation index, the image quality of the scene editing simulation scene is evaluated and the evaluation results are obtained.

[0140] Specifically, the evaluation indicators of image quality evaluation can be divided into two categories, namely with reference and without reference. With reference means that there is an original image that can be compared with the new image for quality comparison. For example, PSNR and SSIM are both evaluation indicators with reference. Without reference means that there is no need for the original image for comparison, and there is only one image that needs to be evaluated. For scene editing, because the greater the scene modification, the greater the gap with the original image, the evaluation indicators with reference will only compare the difference between the two images, and cannot evaluate the rendering effect of the edited scene; it can be considered that the related technology not only has limited scene editing functions and complex processes; but also lacks objective evaluation indicators for scene editing effects. Therefore, it is proposed to use evaluation indicators without reference for scene editing. In this embodiment, two image qualities without reference, natural image quality (NIQE) and reference-free image space quality evaluator (BRISQUE), are used as evaluation indicators for scene editing.

[0141] Among them, NIQE does not require prior information of distorted images nor does it need to train any human opinion scores. It evaluates the quality of images by analyzing local features and global statistical properties of images, taking into account factors such as image sharpness, contrast, and noise. Among them, BRISQUE does not require original reference images either. It uses a SVM model to model the relationship between image quality and features, and evaluates image quality by analyzing local structural information and gradient statistics of images. Given the characteristics of these two evaluation indicators, they can be directly used to evaluate the rendering quality of edited scenes and verify the editing effect.

[0142] In order to verify the effect of scene editing, six scene reconstruction and edited images were selected for evaluation using evaluation indicators. SD reconstruction is the method of this application, and Mars is the comparison method. At the same time, in order to more intuitively compare the results of the evaluation indicators, an additional completely distorted image was selected as the comparison group.

[0143] Table 1

[0144]

[0145] Table 1 shows the evaluation results. From the comparison of PSRN, SSIM, and LPIPS of the blurred image and the position-edited image in Table 1, it can be seen that the blurred image even achieved a higher evaluation index. The reason for this is that the reference image quality evaluation indexes such as PSRN are all compared with the original image. The greater the difference, the lower the score of the evaluation index. Therefore, the greater the editing modification of the scene in scene editing, the lower the reference evaluation index will be. This proves that the use of traditional reference evaluation cannot effectively evaluate the quality of the edited scene image. Among them, PSRN is the peak signal-to-noise ratio, which is a metric for evaluating image quality. Among them, SSIM is the structural similarity index, which is an indicator used to quantify the structural similarity between two images. Among them, LPIPS is the learning perception image block similarity, which is mainly used to measure the difference between two images.

[0146] The last two columns of non-reference evaluation index data in Table 1 show that the worse the image quality, the higher the NIQE and BRISQUE index scores. Here, the higher the score, the worse the image quality. The two evaluation indexes of blurred images and distorted images are significantly higher than those of original images and reconstructed images. This illustrates the rationality of using such non-reference evaluation indicators, that is, objective evaluation indicators for scene editing effects, which facilitate and accurately complete the quality assessment of scene editing effects. It can be found that the values ​​of the non-reference evaluation indicators of the edited images are not much different from those of the original images, and sometimes even better than the reconstructed images, which illustrates the effectiveness of the scene editing method of the present application. As a result, the edited scenes can still maintain high quality.

[0147] In this embodiment, an autonomous driving scene simulation device based on neural radiation field is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. The terms "module", "unit", "subunit", etc. used below may implement a combination of software and / or hardware for predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation in hardware, or a combination of software and hardware, is also possible and conceivable.

[0148] Figure 7 is a structural block diagram of the automatic driving scene simulation device based on the neural radiation field of this embodiment, such as Figure 7 As shown, the device includes: a simulation unit 210 and an editing unit 220;

[0149] A simulation unit 210, configured to simulate an autonomous driving scene based on a neural radiation field to obtain a simulation scene;

[0150] The editing unit 220 is used to process the simulation scene data to obtain object metadata and corresponding position information;

[0151] Based on the assistance of the three-dimensional space view of the simulation scene and the first bird's-eye view of the trajectory of the dynamic object, scene editing is performed on the dynamic object in the simulation scene according to the object metadata and the corresponding position information.

[0152] Through the above-mentioned device, the problems of limited scene editing functions and complicated processes in related technologies are solved. With the assistance of the three-dimensional spatial view of the simulation scene and the first bird's-eye view of the trajectory of the dynamic object, accurate and controllable scene editing of the dynamic objects in the simulation scene is realized, thus making up for the deficiencies in scene editing.

[0153] In some of the embodiments, the editing unit 220 is further used to construct a first bird's-eye view of the trajectory of the dynamic object in the camera coordinate system according to the laser radar point cloud, the object metadata and the position information in the simulation scene;

[0154] On the first bird's-eye view of the dynamic object trajectory, drawing the object position of the dynamic object in the three-dimensional space view based on the object metadata;

[0155] In the camera coordinate system, keep the object metadata unchanged, edit the position information, and perform scene editing on dynamic objects in the simulation scene;

[0156] The edited object metadata and position information are projected into the image coordinate system to complete the rendering and obtain a new simulation scene.

[0157] In some of the embodiments, the editing unit 220 is further used to draw the laser radar point cloud in the simulation scene in the second bird's-eye view of the laser radar coordinate system, and convert the second bird's-eye view in the laser radar coordinate system to the camera coordinate system;

[0158] The object metadata and position information are projected into the transformed second bird's-eye view to obtain the first bird's-eye view of the dynamic object trajectory.

[0159] In some of the embodiments, the editing unit 220 is further configured to delete, add and / or modify the trajectory of dynamic objects in the simulation scene.

[0160] In some of the embodiments, the simulation unit 210 is further used to obtain scene images, camera parameters, object label information and lidar point clouds based on the collection of the autonomous driving scene; the scene images correspond to the camera parameters;

[0161] Determine a three-dimensional Gaussian representation of the autonomous driving scene based on the scene image, the LiDAR point cloud, and the object label information; the three-dimensional Gaussian includes a static background model and multiple dynamic object models;

[0162] According to the scene image, camera parameters and preset loss function, the three-dimensional Gaussian is optimized to obtain the simulation scene.

[0163] In some of the embodiments, the simulation unit 210 is further used to decompose the simulation scene into a static background and a plurality of dynamic objects according to the scene image, the laser radar point cloud and the object label information;

[0164] determining a Gaussian representation of a static background model corresponding to the static background;

[0165] determining a Gaussian representation of a dynamic object model corresponding to each dynamic object;

[0166] A three-dimensional Gaussian representation of the autonomous driving scene is determined based on the Gaussian representation of the static background model and the Gaussian representation of each dynamic object model.

[0167] In some of the embodiments, the simulation unit 210 is further used to project the Gaussian points from the three-dimensional space to the two-dimensional plane through the transformation matrix in the camera parameters to obtain a rendered image;

[0168] Determine gradient update parameters according to the rendered image and the scene image;

[0169] Based on the gradient update parameters, the three-dimensional Gaussian is iteratively optimized until the loss function is satisfied and the simulation scene is obtained.

[0170] In some of the embodiments, the simulation unit 210 is further used to utilize adaptive density control to clip and remove three-dimensional Gaussian points in the autonomous driving scene during the iterative optimization of the three-dimensional Gaussian.

[0171] In some of the embodiments, the autonomous driving scenario simulation device based on neural radiation field further includes an evaluation unit;

[0172] The evaluation unit is used to evaluate the image quality of the scene editing simulation scene based on the non-reference evaluation index to obtain the evaluation result.

[0173] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0174] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0175] In this embodiment, a simulation platform is also provided. Figure 8 As shown, the simulation platform includes: a user layer, a service layer and a data layer; the service layer is connected to the user layer and the data layer respectively;

[0176] User layer, used to grant corresponding user permissions to different users;

[0177] Data layer, used to store simulation platform data;

[0178] The service layer is used to execute the steps in any of the above method embodiments under the request of different user permissions.

[0179] Specifically, the user layer is used to grant corresponding user permissions to different users; distinguish user permissions. The data layer is used to store the data of the simulation platform; the data of the simulation platform includes but is not limited to scene images, camera parameters, object label information, lidar point cloud, simulation scene, evaluation results, scene model, database, etc. The scene model is a related model that has been trained. The database uses a Mysql database for the platform's data information, and uses Redis for caching. In addition to the execution method implementation example, the service layer also provides a variety of services. For example: using SpringBoot to provide backend data services, SpringSecurity provides security and authority authentication to ensure user security and strict division of system permissions. Druid and Oshi technologies realize the monitoring of platform service status, and collect platform operating parameters to facilitate administrators to understand the platform status. The autonomous driving scene simulation editing function provides scene simulation and scene editing.

[0180] In some of the embodiments, the simulation platform further includes: a presentation layer and an interface layer;

[0181] The interface layer is used to receive data requests from the presentation layer, call related services of the service layer, and return the request results corresponding to the data requests to the presentation layer;

[0182] The presentation layer is used to display request results.

[0183] Specifically, the presentation layer is used to display the request results and provides visualization functions. The scene simulation editing end mainly displays the effect of autonomous driving scene editing, and provides visualization of scene simulation, scene editing, multi-view assistance, and simulation editing evaluation indicators. The Web platform management end is mainly for administrators, who manage users, monitor the system, and manage simulation data through the Web end.

[0184] The interface layer is used to receive data requests from the presentation layer, call related services of the service layer, and return the request results corresponding to the data requests to the presentation layer; it provides the presentation layer with relevant information such as scene simulation editing. At the same time, the service is deployed using containerization technology and runs in a docker container to achieve relative isolation of the service, and the communication and interaction between services are realized through interface calls.

[0185] For the simulation platform, it uses a B / S-based architecture and SpringBoot+Vue's front-end and back-end separation framework to build an autonomous driving simulation visualization management platform.

[0186] The backend uses Spring Boot, which greatly improves the speed of building and deployment through annotations and configuration files. It has built-in web containers such as Tomcat, Jetty, and Undertow. It does not need to be deployed in the form of a war package, and can be run in a server environment using a jar package. In terms of security control, the Spring Security security framework is used to implement management functions such as user login authentication and resource permission authorization, while providing security protection, including defense against common web attacks such as cross-site scripting attacks and session attacks. Mybatis is used to operate mysql in the data layer, maintaining the three-tier architecture of MVC. The view layer View completes request reception and response, the service layer Service implements the system business logic, and the persistence layer Dao directly performs database-level operations. The three-tier design reduces the coupling between layers.

[0187] The front-end is built using Vue's scaffolding tool, using npm for package management, and element-ui as the front-end UI component library. The routing jump uses the front-end routing tool vue-router provided by Vue to implement page routing control, partial refresh and on-demand loading, build single-page applications, and achieve front-end and back-end separation.

[0188] The simulation platform adopts a one-click automated deployment method. Using the jenkins+git method, jenkins directly pulls the repository code and builds it. On the backend, jenkins integrates the maven environment, so that the code in the code repository can be directly packaged into a jar package and sent to the server through the configuration of shell commands. On the frontend, jenkins integrates the nodejs environment, packages the code into a dist package and directly puts it on nginx for reverse proxy. The platform server is deployed using a docker container, and the shell script command is configured in jenkins to build automated deployment.

[0189] Through this embodiment, the simulation platform is built based on the Web development technology with front-end and back-end separation, the simulation data is managed in a unified manner, and an intuitive display of simulation editing effects is provided.

[0190] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0191] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0192] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0193] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0194] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A method for simulating an autonomous driving scenario based on a neural radiation field, characterized in that: include: Simulate the autonomous driving scenario based on the neural radiation field to obtain a simulation scenario; Performing data processing on the simulation scene to obtain object metadata and corresponding position information; Based on the three-dimensional space view of the simulation scene and the assistance of the first bird's-eye view of the trajectory of the dynamic object, scene editing is performed on the dynamic object in the simulation scene according to the object metadata and the corresponding position information.

2. The method for simulating an autonomous driving scene based on a neural radiation field according to claim 1, characterized in that: Based on the three-dimensional space view of the simulation scene and the assistance of the first bird's-eye view of the trajectory of the dynamic object, scene editing is performed on the dynamic object in the simulation scene according to the object metadata and the corresponding position information, including: Constructing a first bird's-eye view of the trajectory of the dynamic object in a camera coordinate system according to the laser radar point cloud, the object metadata, and the position information in the simulation scene; On the first bird's-eye view of the dynamic object trajectory, drawing the object position of the dynamic object in the three-dimensional space view based on the object metadata; In the camera coordinate system, the metadata of the object is kept unchanged, the position information is edited, and scene editing is performed on the dynamic object in the simulation scene; The edited object metadata and the position information are projected into the image coordinate system to complete rendering, thereby obtaining the new simulation scene.

3. The method for simulating an autonomous driving scene based on a neural radiation field according to claim 2, characterized in that: According to the laser radar point cloud, the object metadata and the position information in the simulation scene, a first bird's-eye view of the dynamic object trajectory is constructed in a camera coordinate system, including: Drawing the laser radar point cloud in the simulation scene in a second bird's-eye view of the laser radar coordinate system, and converting the second bird's-eye view in the laser radar coordinate system to the camera coordinate system; The object metadata and the position information are projected into the converted second bird's-eye view to obtain a first bird's-eye view of the dynamic object trajectory.

4. The method for simulating an autonomous driving scene based on a neural radiation field according to claim 2, characterized in that: Performing scene editing on the dynamic objects in the simulation scene includes: Deleting, adding, and / or modifying the trajectory of dynamic objects in the simulation scene.

5. The method for simulating an autonomous driving scene based on a neural radiation field according to claim 1, characterized in that: Based on the neural radiation field, the autonomous driving scenario is simulated to obtain the simulation scenario, including: Based on the collection of the autonomous driving scene, a scene image, camera parameters, object label information and a lidar point cloud are obtained; the scene image corresponds to the camera parameters; Determine a three-dimensional Gaussian representation of the autonomous driving scene according to the scene image, the lidar point cloud, and the object label information; the three-dimensional Gaussian includes a static background model and a plurality of dynamic object models; The three-dimensional Gaussian is optimized according to the scene image, the camera parameters and a preset loss function to obtain the simulation scene.

6. The method for simulating an autonomous driving scene based on a neural radiation field according to claim 5, characterized in that: Determining a three-dimensional Gaussian representation of the autonomous driving scene according to the scene image, the laser radar point cloud, and the object label information includes: Decomposing the simulation scene into a static background and a plurality of dynamic objects according to the scene image, the laser radar point cloud and the object label information; determining a Gaussian representation of a static background model corresponding to the static background; determining a Gaussian representation of a dynamic object model corresponding to each of the dynamic objects; A three-dimensional Gaussian representation of the autonomous driving scene is determined based on the Gaussian representation of the static background model and the Gaussian representation of each of the dynamic object models.

7. The method for simulating an autonomous driving scene based on a neural radiation field according to claim 5, characterized in that: Optimizing the three-dimensional Gaussian according to the scene image, the camera parameters and a preset loss function to obtain the simulation scene includes: The Gaussian points are projected from the three-dimensional space to the two-dimensional plane through the transformation matrix in the camera parameters to obtain a rendered image; Determining a gradient update parameter according to the rendered image and the scene image; Based on the gradient update parameters, the three-dimensional Gaussian is iteratively optimized until the loss function is satisfied to obtain the simulation scene.

8. The method for simulating an autonomous driving scenario based on a neural radiation field according to claim 7, characterized in that: The method further comprises: In the process of iteratively optimizing the three-dimensional Gaussian, adaptive density control is used to clip and eliminate the three-dimensional Gaussian points in the autonomous driving scene.

9. The method for simulating an autonomous driving scene based on a neural radiation field according to claim 1, characterized in that: The method further comprises: Based on the non-reference evaluation index, the image quality of the simulation scene in the scene editing is evaluated to obtain an evaluation result.

10. An autonomous driving scene simulation device based on neural radiation field, characterized in that: include: Simulation unit and editing unit; The simulation unit is used to simulate the autonomous driving scene based on the neural radiation field to obtain a simulation scene; The editing unit is used to perform data processing on the simulation scene to obtain object metadata and corresponding position information; Based on the three-dimensional space view of the simulation scene and the assistance of the first bird's-eye view of the trajectory of the dynamic object, scene editing is performed on the dynamic object in the simulation scene according to the object metadata and the corresponding position information.

11. A simulation platform, characterized in that: include: User layer, service layer, and data layer; The service layer is connected to the user layer and the data layer respectively; The user layer is used to grant corresponding user rights to different users; The data layer is used to store the data of the simulation platform; The service layer is used to execute the autonomous driving scenario simulation method based on neural radiation field as described in any one of claims 1 to 9 under the request of different user permissions.

12. The simulation platform according to claim 11, characterized in that: The simulation platform also includes: a presentation layer and an interface layer; The interface layer is used to receive the data request of the presentation layer, call the relevant services of the service layer, and return the request result corresponding to the data request to the presentation layer; The presentation layer is used to display the request result.

Images