Dynamic 3DGS-SLAM method and system based on Gaussian pyramid and adaptive densification

Abstract

The invention relates to the technical field of computer vision and robot positioning, and discloses a dynamic 3DGS-SLAM method and system based on Gaussian pyramid and adaptive densification. The method comprises the following steps: generating a semantic mask through real-time dynamic target detection so as to distinguish a static background from a dynamic object; constructing a 3D Gaussian map model of the scene; rendering images in parallel on a full-resolution view and a down-sampling resolution view by adopting a motion perception Gaussian pyramid rendering method, and fusing rendering results by utilizing a dynamic mask so as to eliminate motion edge artifacts; a self-adaptive densification strategy based on a monotonic attenuation function is adopted, the initialization radius of the Gaussian ellipsoid is dynamically adjusted based on the training progress, and high-density initialization and progressive redundancy pruning are executed to optimize the geometric fidelity; and finally, performing coarse-to-fine camera pose optimization, and outputting a precise pose and a high-fidelity static map. According to the method, the problems of tracking drift and rendering artifacts under dynamic interference are effectively solved, and the positioning precision and the rendering quality are improved.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, autonomous robot navigation and augmented reality, and particularly to a method and system for simultaneous localization and mapping (SLAM), specifically a high-precision, high-performance SLAM scheme based on multi-resolution rendering and adaptive geometry densification suitable for dynamic environments. Background Technology

[0002] Visual SLAM technology is the core technology that enables robots and AR / VR devices to achieve self-localization and scene understanding in unknown environments. Traditional SLAM systems are usually optimized based on sparse feature points or dense photometric errors and assume that the environment is static. However, there are a large number of moving objects in the real world (such as pedestrians and vehicles). These dynamic targets can break the geometric-photometric consistency assumption of camera observations, leading to serious drift in pose estimation and even system failure.

[0003] In recent years, novel scene representation methods such as Neural Radiation Field (NeRF) and 3D Gaussian Splatting (3DGS) have been introduced into SLAM, significantly improving the visual quality of reconstructed maps. 3DGS, in particular, has become a research hotspot in high-fidelity SLAM due to its explicit representation and efficient rasterization rendering characteristics. However, most existing 3DGS-based SLAM methods are still based on the assumption of a static environment. Although recent studies have attempted to exclude dynamic regions by introducing semantic segmentation masks (e.g., DG-SLAM), significant limitations remain in rendering quality for complex dynamic scenes: blurring, artifacts, and flickering are common at the edges of dynamic objects, severely affecting visual consistency; simultaneously, traditional fixed initialization strategies struggle to adapt to non-uniform geometric complexity, leading to loss of detail in high-curvature regions and redundant representation in flat regions, thus hindering further improvements in overall rendering fidelity.

[0004] Therefore, there is an urgent need in this field for a SLAM solution that can achieve both high-precision pose estimation and high-fidelity map rendering in complex dynamic environments. Summary of the Invention

[0005] This invention aims to solve the technical problems of obvious motion edge rendering artifacts and the difficulty in balancing accuracy and efficiency in high dynamic environments in existing dynamic 3DGS-SLAM methods.

[0006] To address the aforementioned technical problems, this invention proposes a dynamic SLAM method and system based on multi-resolution rendering and adaptive densification.

[0007] On one hand, the present invention provides a dynamic SLAM method, comprising:

[0008] Accurate dynamic region semantic masks are generated by combining pre-trained real-time object detection models (such as OneFormer) with geometric consistency checks (such as reprojection depth checks).

[0009] A 3D Gaussian ellipsoid is used as the explicit scene representation model.

[0010] This invention introduces motion-aware Gaussian pyramid rendering into a dynamic SLAM framework. This method renders scenes in parallel at different resolutions (such as full resolution and 1 / 2 resolution) and uses dynamic masks to intelligently fuse the rendering results at different scales, effectively suppressing motion edge artifacts across scales.

[0011] An adaptive densification strategy based on a monotonically decaying function is proposed. This strategy designs a dynamic radius adjustment algorithm based on a monotonically decaying function, which enables the initial radius of the Gaussian ellipsoid to adaptively change with the training progress (a large radius in the early stage covers flat areas, and a small radius in the later stage captures details). It also couples high-density initialization with subsequent progressive redundancy pruning to improve geometric fidelity.

[0012] A coarse-to-fine camera pose optimization process was adopted. First, a robust dense visual odometry was used to provide a coarse pose estimate, and then fine alignment was performed based on the optimized Gaussian map rendering results, ensuring the accuracy and stability of the pose estimation.

[0013] On the other hand, the present invention provides a system for implementing the above method, including a dynamic mask generation module, a scene map modeling module, a multi-resolution rendering module, an adaptive densification module, and a pose optimization module.

[0014] Compared with the prior art, the present invention has the following significant advantages:

[0015] 1. High rendering quality: By employing Gaussian pyramids and adaptive densification, artifacts and flickering at the edges of dynamic objects are effectively eliminated, further improving the quality of rendering details. Experiments show that on highly dynamic datasets such as TUM, its LPIPS (Perceptual Similarity Indices) are significantly lower than baseline methods.

[0016] 2. High positioning accuracy: By combining Gaussian pyramids with adaptively denser Gaussian maps, reliable dynamic masks, and coarse-to-fine pose optimization, it can maintain extremely high pose tracking accuracy even in extreme dynamic environments, with an absolute trajectory error (ATE) superior to existing mainstream solutions.

[0017] Figure caption

[0018] The figures incorporated herein by reference and forming part of this specification illustrate embodiments consistent with this application and, together with the specification, serve to explain the principles of this application, and do not constitute an undue limitation of this application.

[0019] Figure 1 The overall flowchart of the dynamic SLAM method provided in the embodiments of the present invention is shown.

[0020] Figure 2 This is a schematic diagram illustrating the principle of motion-aware Gaussian pyramid rendering in an embodiment of the present invention.

[0021] Figure 3 This is a schematic diagram illustrating the principle of the adaptive densification strategy in an embodiment of the present invention.

[0022] Figure 4 This is a comparison chart of the rendering results evaluated in an embodiment of the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand the technical solutions of this application, the technical solutions of this application will be clearly and completely described below in conjunction with the embodiments of this application. It should be noted that the terms "first," "second," "higher level," and "lower level," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects, not to describe a specific order or sequence. Therefore, these terms can be interchanged where appropriate, and the embodiments described in this application can be implemented in a different order than that shown or described. The embodiments described below do not represent all embodiments consistent with this application; they are merely examples consistent with some aspects of this application as detailed in the appended claims.

[0024] The application scenarios described in this application (such as dynamic indoor environments, AR / VR navigation, etc.) are for the purpose of more clearly illustrating the technical solutions and do not constitute a limitation on the technical solutions. With the emergence of new application scenarios (such as autonomous driving and robot inspection), the technical solutions provided in this application are also applicable to similar technical problems (such as improving SLAM accuracy under dynamic disturbances).

[0025] 3D Gaussian Splatting (3DGS) is an explicit scene representation method that uses a large number of anisotropic 3D Gaussian ellipsoids to model the geometry and appearance of a scene. Each Gaussian ellipsoid is defined by the following parameters: center position Covariance matrix (It can be decomposed into a rotation matrix R and a scaling matrix S, i.e.) Opacity and spherical harmonics (SH) coefficients (Used to express viewpoint-dependent colors). Scene rendering is achieved through a differentiable projection process: a 3D Gaussian ellipsoid is projected according to the camera pose. When projected onto a 2D image plane, its projection center Covariance The calculation is as follows:

[0026]

[0027] in It is a perspective projection function. It is the Jacobian matrix of the projection. Pixel color. and depth Calculated using the forward Alpha synthesis formula:

[0028]

[0029] in and It is the color and depth of the i-th Gaussian. It is its opacity, cumulative opacity Used to evaluate rendering reliability.

[0030]

[0031] In SLAM systems, 3DGS is embedded in a parallel tracking and mapping (PTAM) framework: the tracking front end estimates the camera pose, and the mapping back end optimizes the Gaussian parameters. However, traditional 3DGS-SLAM methods (such as SplaTAM and MonoGS) assume that the scene is static, and in dynamic environments, they are prone to tracking drift and rendering artifacts due to moving objects disrupting consistency.

[0032] To address the aforementioned limitations, this application proposes PDDG-SLAM, whose core innovation lies in coupling motion-aware Gaussian pyramid rendering with adaptive densification based on a monotonic decay function. The method flow is as follows: Figure 1 As shown (although the document does not contain images, the description is based on the original text), the steps include:

[0033] 1. Dynamic Mask Generation: Real-time semantic segmentation networks such as OneFormer are used to identify dynamic objects, and accurate semantic masks are generated by combining reprojection depth consistency checks. This is to distinguish between static backgrounds and dynamic areas.

[0034] 2.3D Gaussian Map Initialization: Based on the RGB-D data of the first frame, initialize the Gaussian ellipsoid set, with parameters including location. Covariance Opacity And the SH coefficient.

[0035] 3. Motion-Aware Gaussian Pyramid Rendering: Constructing a pyramid representation of the input image, where... Original image resolution This is the top layer (lowest resolution). For each layer... Render the Gaussian map to the corresponding resolution to obtain the color map. and depth map . It is a level The total number of pixels below, It is a level The number of effective depth pixels. and It is a level The motion region mask and cumulative opacity mask below are used to filter dynamic regions and unreliable rendered regions. Hierarchy The RGB rendering loss function, depth rendering loss function, and rendering loss function are as follows:

[0036]

[0037]

[0038]

[0039] in At the level The calculated SSIM loss is optimized layer by layer from the top of the pyramid, with each layer training a Gaussian ellipsoid set based on the aforementioned loss function. The process involves using multiple iterations to minimize the graph-building loss function at each layer, thereby achieving the optimal optimization effect.

[0040]

[0041] 4. Adaptive Denseization Based on Monotonically Decreasing Function: To optimize the Gaussian ellipsoid density, a radius adjustment strategy based on a monotonically decaying function is proposed. Current Training Progress Dynamic radius under (number of iterations to total number of iterations) The calculation is as follows:

[0042]

[0043] in This is the initial maximum radius (e.g., 0.1), and b controls the descent rate (set to 7). In the early stages of training ( Use a larger radius to cover flat areas, later ( This paper employs a smaller radius to capture high-curvature details. The adaptive radius strategy effectively complements the native adaptive density controls (such as cloning and splitting) of 3D Gaussian Splashing (3DGS). The monotonically decaying function handles the continuous gradation at the macroscopic scale, while 3DGS density control processes the discrete addition and deletion of microscopic geometry. This collaborative approach avoids the limitations of single control; for example, while the function guides the smooth contraction of the radius, the cloning operation can supplement new Gaussian kernels in high-gradient regions to capture details, thus achieving a more efficient and accurate scene representation evolution overall.

[0044] 5. Coarse-to-fine pose optimization: First, use dense odometry such as DROID-VO to provide coarse pose. Then, the rendered image is combined with an opacity mask and a dynamic region mask, and the pose is finely optimized.

[0045] Opacity mask The cumulative opacity of the rendered image pixels exceeds a threshold. All pixels.

[0046] The pose loss function is:

[0047]

[0048]

[0049]

[0050] in It's a loss of color and resolution compared to the original image. It is a loss of deep consistency.

[0051] In this application's embodiments, terms such as "high-level rendering" and "low-level rendering" are used only to distinguish different resolution levels and do not restrict the execution order. For example, pyramid rendering can start from the top layer and gradually refine, or it can process each level in parallel. Similarly, parameters such as "initial radius" and "dynamic radius" can be adjusted as appropriate (e.g., scaled according to hardware performance).

[0052] The method described in this application can be applied to various hardware platforms (such as NVIDIA GPUs and mobile processors), and the software implementation can be based on the PyTorch or TensorFlow framework. Parameters include weights. These variations can be dynamically adjusted according to the scenario, and all of them fall within the scope of protection of this application. For example, when computational resources are limited, the number of pyramid levels L or the number of Gaussian training rounds can be reduced without affecting the core innovation.

[0053] In one embodiment, the system runs on an NVIDIA RTX 4090 GPU, with a software environment of Ubuntu 18.04 and CUDA 11.3. Key parameter settings are as follows:

[0054] • Number of pyramid levels L = 2 (original image, 1 / 2 downsampled).

[0055] • Gaussian training rounds = 50.

[0056] • Loss weights: = 0.9, = 0.1, = 0.2.

[0057] • Opacity threshold = 0.95.

[0058] To verify the effectiveness of the PDDG-SLAM method proposed in this application, experiments were conducted on the publicly available TUM and BONN datasets, and the method of this application was compared and analyzed with several existing mainstream SLAM methods. Evaluation metrics include Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), and Learning-Aware Patch Similarity (LPIPS), which measure rendering quality, and Absolute Trajectory Error (ATE), which measures localization accuracy.

[0059] 1. Rendering quality analysis

[0060] As shown in Tables 1 and 2 (where "Table 1" and "Table 2" are numbers in the description and should be distinguished from the figures), the method of this application demonstrates excellent rendering fidelity in dynamic scenes.

[0061] Table 1 Rendering accuracy of the TUM dataset

[0062] Table 2 Rendering accuracy of the BONN dataset

[0063]

[0064] As shown in Table 1, the proposed method (PDDG-SLAM) achieves an average PSNR of 20.65 dB, an SSIM of 0.786, and an LPIPS of 0.170 across four dynamic sequences. Compared to the comparative methods MonoGS (PSNR 15.89, SSIM 0.621) and SplaTAM (PSNR 15.55, SSIM 0.633), this application demonstrates significant improvements in both PSNR and SSIM. In particular, compared to the baseline method DG-SLAM, which is also designed for dynamic scenes, this application maintains a similar PSNR (20.65 vs 20.01) while achieving a significant reduction of approximately 54% in the LPIPS metric, which better reflects visual perception quality (0.170 vs 0.373). This fully demonstrates the positive effect of the proposed motion-aware Gaussian pyramid rendering method on eliminating motion artifacts and improving visual quality.

[0065] As shown in Table 2, the proposed method also maintains leading rendering performance on more challenging high dynamic sequences (such as crowd, crowd2), with an average PSNR superior to all comparative methods, further verifying the effectiveness and generalization ability of the proposed method.

[0066] 2. Positioning accuracy analysis

[0067] The positioning accuracy of this application remains stable even in extreme dynamic environments, as shown in Tables 3 and 4.

[0068] Table 3. Localization accuracy of the TUM dataset

[0069]

[0070] Table 4. Localization accuracy of the BONN dataset

[0071]

[0072] As shown in Table 3, the method in this application achieved the best ATE accuracy (0.015m and 0.002m, respectively) on highly dynamic sequences such as fr3_wk_xyz and fr3_wk_hf, with an average ATE of 0.012m. Compared with methods such as RoDyn-SLAM (0.050m) and DGS-SLAM (0.036m), the accuracy improvement is significant. Compared with DG-SLAM (0.013m) which incorporates dynamic processing, the method in this application still achieves a further improvement in accuracy, demonstrating the synergistic advantages of the coarse-to-fine pose optimization strategy combined with the Gaussian pyramid and adaptive densification model.

[0073] As shown in Table 4, the localization accuracy of the method in this application is comparable to or better than that of DG-SLAM on most sequences, with an average ATE of 0.052m, demonstrating the robustness of the algorithm in different dynamic scenarios.

[0074] 3. Visual Comparison Analysis of Rendering Effects

[0075] To visually demonstrate the advantages of the proposed method in improving visual quality, Figure 4 A comparison of rendering results with baseline methods in typical dynamic scenes is provided. As shown in the figure, compared with methods such as DG-SLAM and MonoGS, the proposed PDDG-SLAM in this application shows significant improvements in two aspects: First, at the edges of moving objects (such as the silhouette of a walking pedestrian), this method effectively suppresses artifacts and blurring, obtaining clear and sharp boundaries, which directly corresponds to a significant reduction of approximately 54% in the LPIPS index in Table 1, demonstrating excellent perceptual quality. Second, in high-texture detail areas (such as ground texture and distant objects), this method benefits from the fine optimization of high curvature areas by Gaussian pyramids and adaptive densification strategies, enabling better recovery and preservation of scene details, which is corroborated by the improvement in the PSNR index. This visual comparison strongly demonstrates that this method not only accurately estimates pose in high-dynamic environments but also outputs high-quality, high-fidelity scene rendering results.

[0076] It is worth noting that, in the comparison images, both PDDG-SLAM and DG-SLAM exhibit a certain degree of blurring in the ceiling region of the TUM dataset's fr3_wk_xyz and fr3_wk_hf sequences. This is due to a shared characteristic of both methods: their optimization strategies primarily focus on local Gaussian maps, using depth maps to filter out nearby local regions for optimization. For large, distant regions such as the ceiling, the lack of sufficient image gradient changes means their Gaussian ellipsoids are not trained, resulting in insufficient reconstruction details.

[0077] 4. Ablation Experiment Analysis

[0078] Table 5 Ablation experiments on the TUM dataset

[0079]

[0080] To quantitatively verify the effectiveness of the two core modules proposed in this application—motion-aware Gaussian pyramid rendering (P) and adaptive densification strategy (AD)—an ablation study was conducted on the TUM dataset. As shown in Table 5, we compared the performance of four configurations: the baseline method (DG-SLAM), PDDG-SLAM without the motion-aware Gaussian pyramid rendering module (woP), PDDG-SLAM without the adaptive densification strategy module (woAD), and the complete method of this application (PDDG-SLAM).

[0081] Experimental results show that, compared with the baseline DG-SLAM, the woP and woAD configurations have shown stable improvements in rendering quality metrics such as PSNR, SSIM, and LPIPS. This proves that both motion-aware Gaussian pyramid rendering and adaptive densification strategies contribute positively to system performance.

[0082] The complete model, PDDG-SLAM, achieved the best results across all evaluation metrics. Particularly noteworthy is its significantly lower LPIPS (0.170) score compared to woAD (0.252) and woP (0.303), which measures perceptual quality. This indicates a synergistic effect between the two modules: motion-aware Gaussian pyramid rendering improves pose estimation accuracy (ATE from 0.017 to 0.016) and global rendering stability through multi-scale optimization, while providing more accurate gradient guidance for optimizing the Gaussian ellipsoidal distribution using adaptive densification strategies. This collectively achieves a simultaneous leap in rendering quality and localization accuracy.

[0083] Ablation experiments fully demonstrate that the motion-aware Gaussian pyramid rendering and adaptive densification strategies proposed in this application are both key to improving the performance of dynamic SLAM. Both are indispensable and together constitute the advancement and reliability of this method.

[0084] 5. Performance Summary

[0085] Based on the experimental data above, it can be seen that the PDDG-SLAM method provided in this application, by introducing motion-aware Gaussian pyramid rendering and an adaptive densification strategy based on a monotonic decay function, effectively solves the problems of rendering artifacts and insufficient detail rendering accuracy in dynamic SLAM. It has achieved industry-leading levels in both rendering quality and positioning accuracy, while ensuring the real-time operating efficiency of the system.

Claims

1. A dynamic 3DGS-SLAM method and system based on Gaussian pyramids and adaptive densification, characterized in that, Includes the following steps: S1: Receives the input image sequence and corresponding depth information, uses a real-time semantic segmentation network to identify dynamic objects, and generates a dynamic region semantic mask; S2: Based on the image sequence and depth information, construct and maintain an explicit scene map model composed of multiple 3D Gaussian ellipsoids; S3: The scene map model is rendered using a motion-aware Gaussian pyramid rendering method. The rendering method includes: rendering color images and depth images layer by layer at different resolution levels, and using the dynamic region semantic mask to fuse the rendering results of different levels to output the final rendered image. S4: The 3D Gaussian ellipsoid is optimized using an adaptive densification strategy based on a monotonically decaying function. The strategy includes: dynamically calculating the initial radius of the Gaussian ellipsoid based on the training progress function, performing high-density initialization of the map, and then performing redundant Gaussian ellipsoid pruning based on gradient and window density. S5: Based on the final rendered image, the final displayed scene map model, the dynamic region semantic mask, and the rendering reliability mask, optimize the camera pose from coarse to fine and output the precise 6-DOF camera pose of the current frame.

2. The method according to claim 1, characterized in that, The motion-aware Gaussian pyramid rendering method in step S3 specifically includes: Construct a Gaussian pyramid for the input image; For each level l in the pyramid, the global 3D Gaussian map is rendered to the resolution corresponding to that level to obtain the rendered color image and rendered depth image of that level; The dynamic region semantic mask is downsampled to the same resolution as level l; Calculate the color and depth losses between the rendering result at this level and the real image at the corresponding resolution; The loss functions at multiple levels are summed to jointly optimize the parameters of the 3D Gaussian map.

3. The method according to claim 1, characterized in that, The adaptive densification strategy based on the monotonic decay function in step S4 specifically includes: The radius of the Gaussian ellipsoid at the current training progress is dynamically calculated using a radius adjustment formula based on a monotonically decaying function. In the early stages of training, a larger radius is used to initialize the Gaussian ellipsoid to cover flat areas; In the later stages of training, a smaller radius is used to precisely capture high curvature detail areas; Cloning or splitting operations are performed based on the gradient magnitude and spatial density of the Gaussian ellipsoid, and redundant Gaussian ellipsoids with transparency below the threshold or excessive volume are pruned.

4. The method according to claim 1, characterized in that, The coarse-to-fine camera pose optimization in step S5 specifically includes: First, a short-term pose tracking is performed using a dense visual odometry module to obtain a robust coarse estimate of the pose. Then, using the coarsely estimated pose as the initial value, a fusion loss function is minimized, which includes: color reconstruction loss based on the final rendered image and the real observed image, depth consistency loss, and a weighted mask composed of the dynamic region semantic mask and the rendering cumulative opacity mask.

5. A dynamic SLAM system for implementing the method of any one of claims 1-4, characterized in that, include: The dynamic mask generation module processes the input image and outputs a semantic mask for the dynamic region. The pose calculation module is used to perform coarse camera pose calculation. The scene map building module is used to build and maintain explicit scene maps based on a 3D Gaussian ellipsoid. The Gaussian pyramid rendering module is used to execute the motion-aware Gaussian pyramid rendering method. An adaptive densification module is used to execute the adaptive densification strategy based on a monotonically decaying function; The pose optimization module is used to perform the coarse-to-fine camera pose optimization and output the camera pose.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-4.

Images