Method and device for reconstructing and editing automatic driving environment based on 3dgs

Abstract

The invention discloses a method and a device for reconstructing and editing an automatic driving environment based on 3dgs. The method comprises the following steps: acquiring sensor data and perception data of the automatic driving environment; the sensor data and the sensing data are preprocessed; performing reconstruction training of the automatic driving environment based on the preprocessed sensor data and perception data, and performing optimization based on a training result; and based on the trained reconstruction scene, editing objects in the scene to obtain a final automatic driving simulation environment. Based on the sensor data and the sensing data of the automatic driving environment, a real scene can be quickly reconstructed, a highly vivid picture can be rendered, and the problem that an existing simulation method of the automatic driving environment is low in modeling precision is solved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method and device for reconstructing and editing an autonomous driving environment based on 3DGS. Background Art

[0002] Autonomous driving verification and simulation environments are usually carried out in manually or semi-automatically built simulation simulators, similar to the SUMO simulator or the Carla simulator. The advantage of this type of simulation engine is that the content inside can be edited at will because most of it is handmade, but the disadvantage is that it takes a lot of manpower and time to build the most detailed model possible, but there will still be a big difference in realism and the scale of alignment with the real world.

[0003] Therefore, the prior art needs to be improved. Summary of the invention

[0004] The technical problem to be solved by the present invention is that, in view of the defects of the prior art, the present invention provides a method and device for reconstructing and editing an autonomous driving environment based on 3DGS, so as to solve the problem of low modeling accuracy of the traditional simulation method of the autonomous driving environment.

[0005] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0006] In a first aspect, the present invention provides a method for reconstructing and editing an autonomous driving environment based on 3DGS, comprising:

[0007] Obtain sensor data and perception data of the autonomous driving environment;

[0008] Preprocessing the sensor data and the perception data;

[0009] Reconstruct the autonomous driving environment based on preprocessed sensor data and perception data, and optimize based on the training results;

[0010] Based on the reconstructed scene after training, the objects in the scene are edited to obtain the final autonomous driving simulation environment.

[0011] In one implementation, the step of acquiring sensor data and perception data of the autonomous driving environment includes:

[0012] Obtain the posture of the vehicle-mounted camera, the captured image, and the laser radar point cloud to obtain the sensor data;

[0013] Obtaining a perceptually predicted object bounding box to obtain the perceptual data.

[0014] In one implementation, the preprocessing of the sensor data and the perception data includes:

[0015] The laser radar point clouds are merged, and the object bounding boxes are screened to retain the bounding boxes of dynamic objects.

[0016] In one implementation, the merging of the laser radar point clouds and the screening of the perceived predicted object bounding boxes to retain the bounding boxes of dynamic objects include:

[0017] Convert all the LiDAR point clouds of all frames to the world coordinate system and merge the converted LiDAR point clouds;

[0018] Downsampling the scene point cloud in the world coordinate system, calculating the position difference of the object bounding box between two adjacent frames, filtering the static objects according to the calculated position difference, and retaining the bounding box of the dynamic object;

[0019] The bounding box of the static object is used as the static scene of the autonomous driving environment.

[0020] In one implementation, the reconstructing training of the autonomous driving environment based on the preprocessed sensor data and perception data, and optimizing based on the training results, includes:

[0021] According to the bounding box information of each frame, the object point cloud contained in the corresponding frame is obtained, and it is combined with the point cloud of the static scene. The 3DGS attribute information of each point cloud is projected onto the scene image using the camera's internal and external parameters.

[0022] The losses of the rendered scene image and the real scene image are calculated, and back propagation and optimization are performed based on the losses.

[0023] In one implementation, the back propagation and optimization according to the loss include:

[0024] The bounding boxes of dynamic objects in the training scene are optimized by rotation and translation.

[0025] In one implementation, the editing of objects in the scene based on the reconstructed scene after training to obtain the final autonomous driving simulation environment includes:

[0026] The objects in the scene are edited by deleting the bounding boxes of dynamic and static objects and / or adding the bounding boxes of dynamic and static objects to obtain the final autonomous driving simulation environment.

[0027] In a second aspect, the present invention provides a device for reconstructing and editing an autonomous driving environment based on 3DGS, comprising:

[0028] A data acquisition module, used to acquire sensor data and perception data of the autonomous driving environment;

[0029] A data preprocessing module, used for preprocessing the sensor data and the perception data;

[0030] The reconstruction training module is used to perform reconstruction training of the autonomous driving environment based on the preprocessed sensor data and perception data, and optimize based on the training results;

[0031] The scene editing module is used to edit the objects in the scene based on the reconstructed scene after training to obtain the final autonomous driving simulation environment.

[0032] In a third aspect, the present invention provides a terminal, comprising: a processor and a memory, wherein the memory stores a reconstruction and editing program for an autonomous driving environment based on 3DGS, and when the reconstruction and editing program for an autonomous driving environment based on 3DGS is executed by the processor, it is used to implement the operation of the reconstruction and editing method for an autonomous driving environment based on 3DGS as described in the first aspect.

[0033] In a fourth aspect, the present invention further provides a medium, which is a computer-readable storage medium, and which stores a reconstruction and editing program for an autonomous driving environment based on 3DGS. When the reconstruction and editing program for an autonomous driving environment based on 3DGS is executed by a processor, it is used to implement the operation of the reconstruction and editing method for an autonomous driving environment based on 3DGS as described in the first aspect.

[0034] The present invention adopts the above technical solution to achieve the following effects:

[0035] The present invention obtains sensor data and perception data of the autonomous driving environment, and preprocesses the sensor data and perception data, and can perform reconstruction training of the autonomous driving environment based on the preprocessed sensor data and perception data, and optimize based on the training results, thereby editing the objects in the scene based on the reconstructed scene after training to obtain the final autonomous driving simulation environment. Based on the sensor data and perception data of the autonomous driving environment, the present invention can quickly reconstruct a real scene and render a highly realistic picture, solving the problem of low modeling accuracy of the existing simulation method of the autonomous driving environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0037] Figure 1It is a flowchart of a method for reconstructing and editing an autonomous driving environment based on 3DGS in one implementation of the present invention.

[0038] Figure 2 It is a schematic diagram of a point cloud scanned by a laser radar and a bounding box of a dynamic object in one implementation of the present invention.

[0039] Figure 3 It is a schematic diagram of deleting objects in a scene in one implementation of the present invention.

[0040] Figure 4 It is a schematic diagram of adding an object in a scene in one implementation of the present invention.

[0041] Figure 5 It is a flowchart of reconstruction training of an autonomous driving environment based on 3DGS in one implementation of the present invention.

[0042] Figure 6 It is a functional principle diagram of a terminal in one implementation of the present invention.

[0043] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0045] Exemplary Methods

[0046] Autonomous driving verification and simulation environments are usually carried out in manually or semi-automatically built simulation simulators, similar to the SUMO simulator or the Carla simulator. The advantage of this type of simulation engine is that the content inside can be edited at will because most of it is handmade, but the disadvantage is that it takes a lot of manpower and time to build the most detailed model possible, but there will still be a big difference in realism and the scale of alignment with the real world.

[0047] In view of the above technical problems, an embodiment of the present invention provides a method for reconstructing and editing an autonomous driving environment based on 3DGS. The method can perform reconstruction training of the autonomous driving environment based on the preprocessed sensor data and perception data by acquiring the sensor data and perception data of the autonomous driving environment, and preprocessing the sensor data and perception data, and optimize based on the training results, thereby editing the objects in the scene based on the reconstructed scene after training to obtain the final autonomous driving simulation environment. Based on the sensor data and perception data of the autonomous driving environment, the present invention can quickly reconstruct a real scene and render a highly realistic picture, solving the problem of low modeling accuracy of the existing simulation method of the autonomous driving environment.

[0048] like Figure 1 As shown, an embodiment of the present invention provides a method for reconstructing and editing an autonomous driving environment based on 3DGS, comprising the following steps:

[0049] Step S100, obtaining sensor data and perception data of the autonomous driving environment.

[0050] In this embodiment, in order to achieve a highly realistic simulation effect of an autonomous driving environment, a method for reconstructing and editing a three-dimensional scene expressed based on 3dgs (i.e., 3D Gaussian scene reconstruction and rendering technology) is proposed. The proposed method has obvious advantages over previous methods: the method provided in this embodiment automatically implements both the reconstruction and rendering parts, and only requires the preparation of necessary data (i.e., pre-collected sensor data and perception data) to reconstruct a real scene and render a highly realistic picture in minutes; at the same time, the dynamic objects and static backgrounds in the scene are modeled in combination, so the dynamic objects can be edited to a certain extent after reconstruction; in addition, the method in this embodiment is based on 3dgs, and some new objects can also be inserted into the scene for verification of scenes such as AEB (automatic emergency braking).

[0051] Specifically, in an implementation of this embodiment, step S100 includes the following steps:

[0052] Step S101, acquiring the posture of the vehicle-mounted camera, the captured image and the laser radar point cloud to obtain the sensor data;

[0053] Step S102, obtaining a perceptually predicted object bounding box to obtain the perceptual data.

[0054] In this embodiment, the reconstruction and editing method of the autonomous driving environment based on 3DGS is to express the scene based on 3DGS software. Therefore, when constructing the simulation scene, the parameters that need to be input include: the posture of the on-board camera, the pictures taken by the camera, the point cloud scanned by the laser radar (lidar), and the bounding box (bbox) of the dynamic / static object; the data of the posture of the on-board camera, the pictures taken by the camera, and the point cloud scanned by the laser radar can be collected by the vehicle through the corresponding sensors in the actual autonomous driving scene, and stored in the corresponding database to obtain the sensor data; the sensor data is used for the reconstruction and rendering of the autonomous driving environment simulation; and for the bounding box of the dynamic / static object, the vehicle performs perception prediction through the perception model in the actual autonomous driving scene to obtain the perception data; the perception data can also be used for the reconstruction and rendering of the autonomous driving environment simulation.

[0055] As an example, the vehicle camera posture in this embodiment can be obtained from the preset camera2world matrix, which contains the positions and postures of various vehicle cameras when shooting images; the picture taken by the camera is the picture corresponding to the vehicle camera posture, that is, the picture is taken under a corresponding posture in the camera2world matrix; the point cloud scanned by the laser radar (lidar) and the bounding box (bbox) of the dynamic / static object are obtained at the same time and in the same scene as the picture taken by the camera. Figure 2 As shown, Figure 2 The figure below shows the point cloud scanned by LiDAR and the bounding boxes of dynamic / static objects. Figure 2 The marked boxes are the bounding boxes of the perceived dynamic / static objects.

[0056] like Figure 1 As shown, in one implementation of the embodiment of the present invention, the method for reconstructing and editing the autonomous driving environment based on 3DGS further includes the following steps:

[0057] Step S200: pre-processing the sensor data and the perception data.

[0058] In this embodiment, for the input raw data (i.e., the posture of the vehicle-mounted camera, the pictures taken by the camera, the point cloud scanned by the lidar, and the bounding boxes of dynamic / static objects), it is also necessary to merge the point cloud scanned by the lidar, and at the same time, filter the bounding boxes of dynamic / static objects to obtain data for reconstruction.

[0059] Specifically, in an implementation of this embodiment, step S200 includes the following steps:

[0060] Step S201, merging the laser radar point clouds, and filtering the object bounding boxes to retain the bounding boxes of dynamic objects.

[0061] Specifically, in an implementation of this embodiment, step S201 includes the following steps:

[0062] Step S201a, converting all the laser radar point clouds of all frames into the world coordinate system, and merging the converted laser radar point clouds;

[0063] Step S201b, downsampling the scene point cloud in the world coordinate system, calculating the position difference of the object bounding box between two adjacent frames, filtering the static objects according to the calculated position difference, and retaining the bounding box of the dynamic object;

[0064] Step S201c, taking the bounding box of the static object as the static scene of the autonomous driving environment.

[0065] In this embodiment, according to the original input data, it is first necessary to convert all the point clouds obtained by the lidar scanning of all frames into the world coordinate system, specifically: convert them to the world coordinate system through the position matrix of the lidar (i.e., the lidar2world matrix); after converting to the world coordinate system, merge the point clouds; in order to speed up the process, it is also necessary to downsample the point cloud of the scene in the world coordinate system to reduce a part of the point cloud.

[0066] After processing the point cloud obtained by the laser radar scanning, in this embodiment, it is also necessary to filter out static objects based on the bounding box information of the dynamic / static objects in each frame; the specific filtering method is: by calculating the position difference of the bounding box of the dynamic / static object between two adjacent frames, finally calculating the total difference sum (that is, the sum of the position differences between all two adjacent frames), and judging whether the calculated total difference sum is less than a certain value (for example, less than 5m), if it is less than a certain value, it is determined to be the bounding box of the static object, and it needs to be excluded at this time; after filtering out the static objects, only the bounding box of the dynamic object is retained. At this time, the bounding box of the static object is set in the form of a scene, that is, the bounding box of the static object is used as the reconstructed static scene.

[0067] like Figure 1 As shown, in one implementation of the embodiment of the present invention, the method for reconstructing and editing the autonomous driving environment based on 3DGS further includes the following steps:

[0068] Step S300: Reconstruction training of the autonomous driving environment is performed based on the preprocessed sensor data and perception data, and optimization is performed based on the training results.

[0069] In this embodiment, the merged lidar point cloud, the bounding boxes of the retained dynamic objects, and the bounding boxes of the static objects that serve as the scene are used to perform reconstruction training of the 3DGS-based autonomous driving environment to optimize the efficiency and quality of 3DGS-based reconstruction and rendering.

[0070] Specifically, in an implementation of this embodiment, step S300 includes the following steps:

[0071] Step S301, obtaining the object point cloud contained in the corresponding frame according to the bounding box information of each frame, and combining it with the point cloud of the static scene, and projecting the 3DGS attribute information of each point cloud onto the scene image using the camera internal and external parameters;

[0072] Step S302, calculating the loss of the rendered scene image and the real scene image, and performing back propagation and optimization according to the loss.

[0073] In this embodiment, in the 3DGS training process, it is necessary to obtain the object point cloud contained in the corresponding frame according to the bounding box information of the dynamic object in each frame, and combine it with the point cloud of the static scene, and rasterize (splat) the 3DGS attribute information of each point cloud onto the picture at the corresponding moment using the camera's intrinsic parameters and extrinsic parameters; wherein the 3DGS attributes of each point cloud include: scale (representing the size of the Gaussian sphere), rotation (representing the rotation of the Gaussian sphere), opacity (representing opacity), SH feature (representing spherical harmonic coefficients), position (representing the center point of the three-dimensional Gaussian), etc.; the rasterization (splat) is a rendering technology, which can be understood as projection onto a picture.

[0074] After rasterizing (splatting) onto the image, the optimization is performed by calculating a loss between the rendered image and the real image, and backpropagating the calculated loss. The specific loss calculation method is: calculating the L1 loss (mean absolute error loss) of the color of the rendered image (for example, the color value of each pixel, that is, RGB) and the RGB value of the real image, and the ssim loss (a loss function used to measure the similarity of two images).

[0075] Specifically, in an implementation of this embodiment, step S302 includes the following steps:

[0076] Step S302a, optimizing the bounding box of the dynamic object in the training scene by rotation and translation.

[0077] In this embodiment, since the bounding box of the dynamic object predicted by the perception model has a certain offset and error, it is necessary to optimize the rotation and displacement of the bounding box of the dynamic object during the optimization process in this embodiment; specifically, the rotation and displacement of the bbox are optimized by adding the attributes of the bounding box of the dynamic object (i.e., rotation and displacement) to the optimizable parameters, and then, during back propagation, the gradient will be passed to this parameter for optimization, and the obtained delta value (hedge value) will be added to the initial rotation and offset to obtain the optimized version of the rotation and offset.

[0078] The method in this embodiment can automatically achieve rapid modeling, and even for dynamic scenes, modeling can be completed in a short time (i.e., one hour), which greatly improves the modeling efficiency compared to the time spent by the method based on manual modeling (for example, the rendering method based on NeRF takes several hours). In addition, the view rendered by the method in this embodiment can highly restore the realism of the real world, and can be so delicate as to restore the words on the license plate.

[0079] like Figure 1 As shown, in one implementation of the embodiment of the present invention, the method for reconstructing and editing the autonomous driving environment based on 3DGS further includes the following steps:

[0080] Step S400, based on the reconstructed scene after training, edit the objects in the scene to obtain the final autonomous driving simulation environment.

[0081] In this embodiment, in addition to automatically realizing the reconstruction and rendering of the autonomous driving environment, scenes or dynamic objects that need to be adjusted can also be edited according to the operation to obtain the desired autonomous driving simulation environment.

[0082] Specifically, in an implementation of this embodiment, step S400 includes the following steps:

[0083] Step S401, editing the objects in the scene by deleting the bounding boxes of dynamic and static objects and / or adding the bounding boxes of dynamic and static objects to obtain the final autonomous driving simulation environment.

[0084] In this embodiment, the timeline of the autonomous driving simulation environment can be adjusted at will according to the time adjustment operation to locate the time to the moment that needs to be edited, so as to edit the dynamic objects in the scene, or directly drag and rotate dynamic vehicles or people to a certain extent. Compared with the rendering method of NeRF, real-time editing can be achieved in this embodiment.

[0085] In addition to the above drag and rotate editing methods, this embodiment can also easily delete and insert new objects to generalize the scene of autonomous driving simulation. Compared with NeRF, which is based on implicit methods, it is difficult to make editing changes and has great limitations. This method is based on explicit 3DGS and can change the content of the scene in a "visible and tangible" way.

[0086] like Figure 3 As shown, Figure 3 This is a schematic diagram of deleting objects in the scene. Figure 3 (a) is the simulation scene before deletion, in which there are two conspicuous black cars; Figure 3 (b) is the simulation scene after deletion, in which there is only one conspicuous black car; Figure 4 As shown, Figure 4 This is a schematic diagram of adding objects to the scene. Figure 4 The ice cream cone pointed by the middle arrow is where the new object is inserted.

[0087] like Figure 5 As shown, as an example, in a practical application scenario, the reconstruction training of the autonomous driving environment based on 3DGS includes the following steps:

[0088] S11, input: camera pose, image, lidar point cloud, perception predicted bounding box;

[0089] S12, preprocessing: point cloud merging, bounding box processing;

[0090] S13, training scene setup: 3dgs, rotation and translation of bounding box;

[0091] S14, randomly select a frame and obtain the bounding box of the current frame;

[0092] S15, 3dgs for combined scenes and dynamic objects;

[0093] S16, rendering;

[0094] S17, calculate loss and back propagation.

[0095] The method proposed in this embodiment has obvious advantages over the existing manual rendering method: the reconstruction and rendering parts in this embodiment are automatic, and only the necessary data needs to be prepared to reconstruct a real scene and render highly realistic pictures in minutes; at the same time, the dynamic objects and static backgrounds in the scene are modeled in combination, so the dynamic objects can be edited to a certain extent after reconstruction; in addition, the method in this embodiment is based on 3dgs, and some new objects can also be inserted into the scene for verification of scenes such as AEB (automatic emergency braking).

[0096] This embodiment achieves the following technical effects through the above technical solution:

[0097] This embodiment obtains sensor data and perception data of the autonomous driving environment, and pre-processes the sensor data and perception data, and can perform reconstruction training of the autonomous driving environment based on the pre-processed sensor data and perception data, and optimize based on the training results, thereby editing the objects in the scene based on the reconstructed scene after training to obtain the final autonomous driving simulation environment. This embodiment can quickly reconstruct a real scene and render a highly realistic picture based on the sensor data and perception data of the autonomous driving environment, solving the problem of low modeling accuracy of existing simulation methods for autonomous driving environments.

[0098] Exemplary Devices

[0099] Based on the above embodiments, the present invention further provides a 3DGS-based autonomous driving environment reconstruction and editing device, comprising:

[0100] A data acquisition module, used to acquire sensor data and perception data of the autonomous driving environment;

[0101] A data preprocessing module, used for preprocessing the sensor data and the perception data;

[0102] The reconstruction training module is used to perform reconstruction training of the autonomous driving environment based on the preprocessed sensor data and perception data, and optimize based on the training results;

[0103] The scene editing module is used to edit the objects in the scene based on the reconstructed scene after training to obtain the final autonomous driving simulation environment.

[0104] This embodiment achieves the following technical effects through the above technical solution:

[0105] This embodiment obtains sensor data and perception data of the autonomous driving environment, and pre-processes the sensor data and perception data, and can perform reconstruction training of the autonomous driving environment based on the pre-processed sensor data and perception data, and optimize based on the training results, thereby editing the objects in the scene based on the reconstructed scene after training to obtain the final autonomous driving simulation environment. This embodiment can quickly reconstruct a real scene and render a highly realistic picture based on the sensor data and perception data of the autonomous driving environment, solving the problem of low modeling accuracy of existing simulation methods for autonomous driving environments.

[0106] Based on the above embodiment, the present invention further provides a terminal, whose principle block diagram can be as follows: Figure 6 shown.

[0107] The terminal includes: a processor, a memory, an interface, a display screen and a communication module connected via a system bus; wherein the processor of the terminal is used to provide computing and control capabilities; the memory of the terminal includes a storage medium and an internal memory; the storage medium stores an operating system and a computer program; the internal memory provides an environment for the operation of the operating system and the computer program in the storage medium; the interface is used to connect to external devices; the display screen is used to display corresponding information; and the communication module is used to communicate with a cloud server or other devices.

[0108] When the computer program is executed by a processor, it is used to implement the operation of the reconstruction and editing method of the autonomous driving environment based on 3DGS.

[0109] It can be understood by those skilled in the art that Figure 6 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal to which the scheme of the present invention is applied. The specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0110] In one embodiment, a terminal is provided, which includes: a processor and a memory, the memory storing a reconstruction and editing program for an autonomous driving environment based on 3DGS, and the reconstruction and editing program for an autonomous driving environment based on 3DGS, when executed by the processor, is used to implement the operations of the above-mentioned reconstruction and editing method for an autonomous driving environment based on 3DGS.

[0111] In one embodiment, a storage medium is provided, wherein the storage medium stores a reconstruction and editing program for an autonomous driving environment based on 3DGS, and when the reconstruction and editing program for an autonomous driving environment based on 3DGS is executed by a processor, it is used to implement the operations of the above-mentioned reconstruction and editing method for an autonomous driving environment based on 3DGS.

[0112] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and volatile memory.

[0113] In summary, the present invention provides a method and device for reconstructing and editing an autonomous driving environment based on 3DGS, the method comprising: obtaining sensor data and perception data of the autonomous driving environment; preprocessing the sensor data and the perception data; performing reconstruction training of the autonomous driving environment based on the preprocessed sensor data and perception data, and optimizing based on the training results; and editing objects in the scene based on the reconstructed scene after training to obtain the final autonomous driving simulation environment. Based on the sensor data and perception data of the autonomous driving environment, the present invention can quickly reconstruct a real scene and render a highly realistic picture, solving the problem of low modeling accuracy of existing simulation methods for autonomous driving environments.

[0114] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for reconstructing and editing an autonomous driving environment based on 3DGS, characterized in that: include: Obtain sensor data and perception data of the autonomous driving environment; Preprocessing the sensor data and the perception data; Reconstruct the autonomous driving environment based on preprocessed sensor data and perception data, and optimize based on the training results; Based on the reconstructed scene after training, the objects in the scene are edited to obtain the final autonomous driving simulation environment.

2. The method for reconstructing and editing an autonomous driving environment based on 3DGS according to claim 1, characterized in that: The obtaining of sensor data and perception data of the autonomous driving environment includes: Obtain the posture of the vehicle-mounted camera, the captured image, and the laser radar point cloud to obtain the sensor data; Obtaining a perceptually predicted object bounding box to obtain the perceptual data.

3. The method for reconstructing and editing an autonomous driving environment based on 3DGS according to claim 2, characterized in that: The preprocessing of the sensor data and the perception data includes: The laser radar point clouds are merged, and the object bounding boxes are screened to retain the bounding boxes of dynamic objects.

4. The method for reconstructing and editing an autonomous driving environment based on 3DGS according to claim 3, characterized in that: The merging of the laser radar point clouds and screening of the object bounding boxes predicted by the perception to retain the bounding boxes of the dynamic objects include: Convert all the LiDAR point clouds of all frames to the world coordinate system and merge the converted LiDAR point clouds; Downsampling the scene point cloud in the world coordinate system, calculating the position difference of the object bounding box between two adjacent frames, filtering the static objects according to the calculated position difference, and retaining the bounding box of the dynamic object; The bounding box of the static object is used as the static scene of the autonomous driving environment.

5. The method for reconstructing and editing an autonomous driving environment based on 3DGS according to claim 1, characterized in that: The reconstruction training of the autonomous driving environment based on the preprocessed sensor data and perception data, and optimization based on the training results, include: According to the bounding box information of each frame, the object point cloud contained in the corresponding frame is obtained, and it is combined with the point cloud of the static scene. The 3DGS attribute information of each point cloud is projected onto the scene image using the camera's internal and external parameters. The losses of the rendered scene image and the real scene image are calculated, and back propagation and optimization are performed based on the losses.

6. The method for reconstructing and editing an autonomous driving environment based on 3DGS according to claim 5, characterized in that: The back propagation and optimization according to the loss include: The bounding boxes of dynamic objects in the training scene are optimized by rotation and translation.

7. The method for reconstructing and editing an autonomous driving environment based on 3DGS according to claim 1, characterized in that: The object in the scene is edited based on the reconstructed scene after training to obtain the final autonomous driving simulation environment, including: The objects in the scene are edited by deleting the bounding boxes of dynamic and static objects and / or adding the bounding boxes of dynamic and static objects to obtain the final autonomous driving simulation environment.

8. A 3DGS-based autonomous driving environment reconstruction and editing device, characterized in that: include: A data acquisition module, used to acquire sensor data and perception data of the autonomous driving environment; A data preprocessing module, used for preprocessing the sensor data and the perception data; The reconstruction training module is used to perform reconstruction training of the autonomous driving environment based on the preprocessed sensor data and perception data, and optimize based on the training results; The scene editing module is used to edit the objects in the scene based on the reconstructed scene after training to obtain the final autonomous driving simulation environment.

9. A terminal, characterized in that: include: A processor and a memory, wherein the memory stores a reconstruction and editing program for an autonomous driving environment based on 3DGS, and when the reconstruction and editing program for an autonomous driving environment based on 3DGS is executed by the processor, it is used to implement the operation of the reconstruction and editing method for an autonomous driving environment based on 3DGS as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a 3DGS-based autonomous driving environment reconstruction and editing program, which, when executed by a processor, is used to implement the operation of the 3DGS-based autonomous driving environment reconstruction and editing method as described in any one of claims 1 to 7.

Images