A robust 3D gaussian-spraying slam method based on adaptive kernel smoothing

Abstract

This invention discloses a robust 3D Gaussian sputtering SLAM method based on adaptive kernel smoothing, comprising: Step 1, capturing images of the target area to be constructed, and constructing an original 3D Gaussian map based on the previous t frames of RGB-D images; Step 2, performing CB-KNN-enhanced differentiable rendering on the original 3D Gaussian map; Step 3, performing iterative optimization of the original 3D Gaussian map using a CB-KNN-based SLAM system to complete map updates. The significant advantages of this invention are: First, extremely high rendering fidelity, closely matching the RGB image of the real scene, accurately restoring the details and colors of indoor furniture and decorations; Second, excellent depth accuracy, providing a reliable geometric basis for pose tracking and scene understanding; Third, outstanding robustness, consistently outputting high-quality reconstruction results even under complex indoor layouts and changing viewpoints.

Description

Technical Field

[0001] This invention relates to the field of Simultaneous Localization and Mapping (SLAM) technology, and in particular to a robust 3D Gaussian sputtering SLAM method based on adaptive kernel smoothing. Background Technology

[0002] In the field of SLAM (Simultaneous Localization and Mapping), 3D Gaussian sputtering (3DGS) technology has become one of the core technologies for real-time dense reconstruction in recent years due to its ability to achieve millisecond-level high-fidelity radiation field rendering. Existing 3DGS-based SLAM systems achieve photometric and geometric alignment to estimate camera pose by quickly rendering color and depth maps. However, it has the following key drawbacks: First, Gaussian parameter errors lead to rendering artifacts. The core parameters of 3DGS (position, orientation, scale, and color) are optimized from noisy camera observations, and the edges of the Gaussian kernel often have significant errors. During rasterization, these errors produce artifacts such as incorrect pixel colors and scene discontinuities, introducing a large amount of noise into photometric alignment for pose tracking. Second, pose tracking robustness is insufficient. In complex scenarios such as rapid camera movement and drastic changes in viewpoint, the aforementioned rendering artifacts can lead to photometric alignment failure, causing pose drift or even tracking loss, which cannot meet the reliability requirements of real-time moving scenarios (such as robot navigation). Secondly, the limitations of traditional design concepts mean that existing technologies generally adhere to the traditional understanding that "high-fidelity visual rendering is the core of pose estimation accuracy," excessively pursuing rendering clarity while neglecting the impact of Gaussian parameter noise on tracking robustness. Furthermore, the lack of a decoupling mechanism between "scene fidelity" and "tracking robustness" makes it impossible to guarantee pose tracking stability under parameter errors. Finally, existing robustness methods are inapplicable. Traditional SLAM robustness strategies (such as ORB-SLAM3's dynamic feature point weighting and robust loss function) only address observation noise and cannot solve the unique 3DGS problem of "map representation error (Gaussian parameter error) → rendering artifacts → alignment noise." Existing kernel smoothing techniques are only applied to 2D pixels or 3D point clouds and do not involve regularization optimization in the 3DGS rendering process. These problems severely restrict the application of 3DGS-SLAM in real-time, complex scenes, necessitating a new method that can mitigate the impact of Gaussian parameter errors and improve pose tracking robustness. Summary of the Invention

[0003] Purpose of the invention: The technical problem to be solved by the present invention is to provide a robust 3D Gaussian sputtering SLAM method based on adaptive kernel smoothing, which addresses the shortcomings of the existing technology.

[0004] To address the aforementioned technical problems, this invention discloses a robust 3D Gaussian sputtering SLAM method based on adaptive kernel smoothing, comprising the following steps:

[0005] Step 1: Take pictures of the target area to be constructed, and complete the construction of the original 3D Gaussian map based on the RGB-D images of the previous t frames;

[0006] Step 2: Perform CB-KNN enhanced differentiable rendering on the original 3D Gaussian map;

[0007] Step 3: Iteratively optimize the original 3D Gaussian map using a SLAM system based on CB-KNN to complete the map update.

[0008] Step 1 specifically includes: constructing an initial 3D Gaussian map based on the RGB-D images of the previous t frames, back-projecting the pixels of the first frame image into an initial Gaussian set, defining the core parameters of each Gaussian (including center position, opacity, radius, and view-independent color), minimizing the RGB and depth errors through differentiable rendering, iteratively optimizing the initial Gaussian parameters, and forming the original 3D Gaussian map for subsequent SLAM processes (all subsequent temporary parameter modifications will not change this original map).

[0009] Step 2 specifically includes: For the input t+1 frame image, a differentiated rendering strategy is adopted according to whether it is a keyframe: For non-keyframes, the original 3D Gaussian map is directly used to render the RGB image and depth map to meet the real-time requirements of fast pose tracking; For keyframes, the visible Gaussians in the current frame are first filtered and sorted by camera distance, and then the K neighboring Gaussians with the largest contribution are selected for each pixel. The CB-KNN strategy is used to temporarily correct these neighboring Gaussians (including position correction of centroid shift in 2D projection and color smoothing based on contribution weights) to generate a temporary Gaussian set. Finally, a smooth RGB image, depth map and contour map are rendered based on this temporary set to provide reliable input for high-precision pose optimization.

[0010] Step 3 specifically includes: based on the original 3D Gaussian map, with keyframe CB-KNN correction as the core and non-keyframe fast tracking as a supplement, to complete the iterative optimization of the SLAM system: including frame-by-frame camera pose optimization, keyframe optimization based on smoothed rendering maps, and non-keyframe fast optimization based on the original rendering maps; identifying low-density or new geometric structure regions through densification masks, supplementing with new Gaussians to improve the map; adaptively determining the K value of CB-KNN by combining local Gaussian density and scene complexity, selecting keyframes and applying CB-KNN only to them; selecting the keyframe group with the highest overlap, optimizing the original map parameters through differentiable rendering and pruning invalid Gaussians to complete the map update and form a complete SLAM loop.

[0011] The initial map is constructed based on the first t frames of RGB-D images. The specific steps are as follows:

[0012] Step 1-1. Initial Gaussian (in this text, "Gaussian" refers to Gaussian primitives) generation: All pixels of the first frame RGB-D image are back-projected into 3D points using camera intrinsics. Each 3D point corresponds to an initialized Gaussian, forming a Gaussian set. The parameters of a single Gaussian are defined as follows: ,in: : The 3D center position of Gauss, i.e., the coordinates of the 3D point obtained by back projection; The Gaussian opacity, initially set to 0.5; The radius of the Gaussian shape is adaptively set based on the depth value corresponding to the pixel (the greater the depth, the larger the initial radius value, ranging from 0.1 to 0.5). Gaussian perspective is independent of RGB color and is directly taken from the RGB value of the corresponding pixel in the first frame;

[0013] Step 1-2. Gaussian parameter optimization: Render the initial Gaussian set into an RGB image and a depth image using differentiable rendering. Calculate the RGB and depth errors between the rendered result and the ground truth image of the first frame. Iteratively optimize the Gaussian set using the Adam optimizer. The parameters are set with a learning rate of 1e-4 until the error converges, resulting in the original 3D Gaussian map.

[0014] For non-keyframes, with "real-time performance" as the core objective, the original 3D Gaussian map is used directly to render the RGB image and depth map. The specific steps are as follows:

[0015] 2-1-1. Visible Gaussian Filtering: Gaussians outside the current camera's field of view are removed by frustum clipping, retaining only visible Gaussians;

[0016] 2-1-2. Direct Rendering: Using the original Gaussian set G, RGB and depth maps are generated based on the standard differentiable rendering pipeline of 3DGS (without CB-KNN correction);

[0017] 2-1-3. Fast Pose Initialization: Based on a Constant Velocity Model ( + ( - ), Initialize the pose of frame t+1 for the camera pose of frame t. Optimize the pose by minimizing the L1 loss of color and depth between the rendered image and the ground truth image of the input frame. The processing time for a single frame is controlled within 23ms (RTX A40 graphics card).

[0018] For keyframes, with "robustness" as the core objective, CB-KNN correction is introduced. The specific steps are as follows:

[0019] 2-2-1. Visible Gaussian filtering and sorting: Same as step 2-1-1 for non-keyframes, filter visible Gaussians, and then sort the visible Gaussians in ascending order according to "Euclidean distance from the center of the Gaussian to the optical center of the camera" to ensure that near-field Gaussians participate in subsequent calculations first;

[0020] 2-2-2. K-Nearest Neighbor Gaussian Selection: For each pixel in the current keyframe ( This represents the horizontal axis coordinate of a pixel in the image coordinate system, i.e., the horizontal pixel position. This represents the vertical axis coordinate of a pixel in the image coordinate system, i.e., the vertical pixel position. The contribution of all visible Gaussian elements to this pixel is calculated using the Gaussian decay function.

[0021]

[0022] in, The 3D coordinates corresponding to the pixels; then select the one with the largest contribution value. A set of Gaussians ;

[0023] 2-2-3. For Gaussian sets Perform CB-KNN (corrected fuzzy K-nearest neighbor) correction;

[0024] 2-2-4. Rendering smoothing: based on a corrected temporary Gaussian set Each of these is rendered to generate a smooth RGB image, depth image, and contour image:

[0025] Step 2-2-3 specifically includes:

[0026] Position correction: First, adjust the Gaussian set. 3D center of each Gaussian Converted to 2D pixel coordinates using a projection function, the projection function... (·)for:

[0027] ( )=

[0028] in, The 3D center location of a single neighboring Gaussian. Camera external parameters, For camera internal parameters;

[0029] Calculate the Gaussian set again 2D projection centroid :

[0030]

[0031] Finally, project each Gaussian 2D onto its 2D projection centroid. Offset, to obtain the corrected 2D coordinates:

[0032]

[0033] [0.1, 0.3] are offset coefficients. = To avoid the denominator being zero, This is a temporary 3D position correction, used only for rendering the current frame and does not update the original map. ;

[0034] Color correction: Calculating Gaussian sets based on the Gaussian decay function Each Gaussian pair of pixels Contribution weight:

[0035]

[0036] in, : refers to the first The corresponding pixel The contribution intensity function value of the neighboring Gaussians to the pixel.

[0037] Then, the corrected color is calculated using a weighted average:

[0038]

[0039] This weighting ensures that the large Gaussian contribution dominates the color, while suppressing anomalous Gaussian color noise.

[0040] The camera pose optimization is performed according to the principle of "high-precision optimization of key frames and fast optimization of non-key frames":

[0041] 3-1-1. Non-keyframe pose optimization: Using the original rendering image generated in step 2-1-2 as input, minimize the "color loss and depth loss" and iteratively optimize the pose to ensure real-time performance;

[0042] 3-1-2. Keyframe pose optimization: Using the smoothed map generated in step 2-2-4 as input, only the contour map coverage (calculated in step 2-2-4) is used. For pixels with a threshold, minimize the weighted loss function. :

[0043]

[0044] in, , These are rendering depth and color, respectively. , Given the ground truth depth and color of the input frame, This refers to the pixel coverage of the outline map. This is due to the loss in depth rendering and color rendering.

[0045] Low-density or new geometrically structured regions are identified through densification masks, and new Gaussian simplification is used to supplement the map. The specific steps for supplementing the Gaussian simplification of low-density or new geometrically structured regions in the map include:

[0046] 3-2-1. Compacted Mask Generation: Calculate the compacted mask based on the camera pose obtained from tracking. The formula is:

[0047]

[0048] The error threshold is denoted as MDE, where MDE is the median depth error. When the pixel is in the "low-density region", the corresponding 3D area is marked as the "region to be densified"; in the "low-density region" marked by the mask ( ) or “new geometric structure region” (depth error exceeds threshold);

[0049] 3-2-2. Adding a New Gaussian: Back-project the pixels of the "region to be densified" into 3D points, initialize the new Gaussian parameters according to the initial Gaussian parameter rules in step 1-1, and add them to the original map. middle.

[0050] The specific method for determining "whether it is a keyframe" is adaptive K-value keyframe selection:

[0051] 3-3-1. Keyframe filtering frequency: From the continuously input RGB-D sequence, candidate keyframes are filtered at a fixed frequency of "selecting 1 frame every n frames";

[0052] 3-3-2. Adaptive Calculation of K Value: The K value is the number of neighboring Gaussians in CB-KNN (corrected fuzzy K-nearest neighbors), determined by the local Gaussian density. and scene complexity Joint decision:

[0053] The local Gaussian density Divide the candidate keyframe image into an 8×8 pixel grid and count the number of Gaussian projections within each grid. ,but (N = 64 is the number of grid pixels);

[0054] The complexity of the scenario : Calculate the pose difference between the candidate keyframe and the previous keyframe (translation distance and rotation angle normalized to the [0,1] interval), as... ;

[0055] The K value is calculated as follows:

[0056]

[0057] As the baseline value, To adjust the coefficient, Avoid division by zero;

[0058] 3-3-3. Keyframe Confirmation: The depth back-projection point cloud of the candidate keyframe is smoothed by CB-KNN, the overlap between the candidate keyframe and the current frame is calculated (the proportion of points in the current frame point cloud that fall within the view frustum of the candidate keyframe is counted), and the candidate keyframe with the highest overlap is selected as the final keyframe, and CB-KNN correction is applied only to it.

[0059] The map update described in step 3 specifically includes

[0060] Select the "current frame + latest keyframe + k-2 keyframes with the highest overlap with the current frame", apply CB-KNN correction to these keyframes; iteratively update the original Gaussian map parameters by minimizing RGB error (including SSIM loss) and depth error through differentiable rendering. Pruning invalid Gaussians (opause close to 0 or size too large) completes map update.

[0061] Beneficial effects:

[0062] 1. Innovative Design Concept: For the first time, a 3DGS-SLAM design perspective of "pose tracking robustness takes precedence over visual fidelity" is proposed, establishing a decoupling mechanism between "scene reconstruction quality" and "pose tracking robustness". Through controllable rendering regularization, pose tracking stability in complex scenes is significantly improved without reducing map fidelity.

[0063] 2. Adaptive advantages of the CB-KNN method:

[0064] Dynamic correction: The position and color of Gaussian are temporarily corrected during the rendering process, directly mitigating rendering artifacts caused by parameter noise at the source;

[0065] High compatibility: No need to rebuild the existing 3DGS rendering pipeline, can be seamlessly integrated into existing frameworks such as SplaTAM;

[0066] Computationally efficient: CB-KNN is applied only to key frames, while non-key frames maintain fast rendering, reducing single-frame tracking time by approximately 21% compared to the existing SplaTAM (from 29ms to 23ms, RTX A40 graphics card).

[0067] 3. Significantly improved pose tracking accuracy:

[0068] On the Replica dataset, the mean absolute trajectory error (ATE RMSE) decreased from 0.39 cm in the existing SplaTAM to 0.32 cm, and the relative pose error (RPE) decreased from 0.10 cm to 0.08 cm.

[0069] On the TUM-RGBD dataset, the ATE RMSE decreased from 3.31cm to 3.18cm, and the trajectory drift in the fr1 / desk scene was reduced by 15%.

[0070] On the ScanNet dataset, the ATE RMSE for the 0181 scene decreased from 12.08cm to 11.91cm, and the number of tracking losses decreased by 60%.

[0071] 4. Map reconstruction quality maintenance:

[0072] The rendering metrics are comparable to existing methods: on the Replica dataset, the peak signal-to-noise ratio (PSNR) reaches 34.17 dB (SplaTAM is 34.11 dB), and the learned perceptual patch similarity (LPIPS) reaches 0.09 (SplaTAM is 0.10).

[0073] Novel perspective rendering without blur: Comparative experiments show that the fidelity of color and depth maps under both novel and training perspectives deviates less from the true values, and there is no loss of detail due to excessive smoothing.

[0074] 5. High engineering application value: It is suitable for real-time mobile scenarios such as autonomous driving and robot navigation, and can work stably under complex conditions such as rapid movement and drastic changes in perspective; moreover, it has low computing cost and can run in real time on consumer-grade GPUs, and has broad industrialization prospects. Attached Figure Description

[0075] Figure 1 A trajectory comparison image for the TUM-RGBD dataset fr1 / desk scene;

[0076] Figure 2 A comparison chart of rendering results for the Replica dataset R0 scene;

[0077] Figure 3 A comparison chart of trajectories from multiple datasets;

[0078] Figure 4 Comparison images of 3D reconstruction of the room0 scene in the synthetic dataset Replica. Detailed Implementation

[0079] This invention discloses a robust 3D Gaussian sputtering (3DGS) simultaneous localization and mapping (SLAM) method based on adaptive kernel smoothing, which belongs to the field of SLAM technology and is applicable to scenarios requiring real-time high-precision pose tracking, such as autonomous driving and robot navigation.

[0080] Existing 3DGS-SLAM systems suffer from drawbacks such as rendering artifacts caused by Gaussian parameter errors, insufficient robustness in pose tracking, and the inapplicability of traditional robust strategies. The purpose of this invention is to mitigate the impact of Gaussian parameter noise, improve tracking robustness and accuracy, while maintaining scene reconstruction quality.

[0081] The core of the technical solution is the proposed "Corrective Blurry K-Nearest Neighbors (CB-KNN)" strategy, which includes the following steps: 1. Initial 3D Gaussian map construction: generating and optimizing an initial Gaussian set based on the previous t frames of RGB-D images to form the original map; 2. CB-KNN-enhanced differentiable rendering: generating corresponding images for key frames (after CB-KNN temporarily corrects the position and color before rendering) and non-key frames (direct and fast rendering); 3. Iterative optimization of the SLAM system based on CB-KNN, including frame-by-frame pose optimization, Gaussian compaction to complete the map, adaptive K-value key frame selection, original map parameter optimization and pruning, forming a complete SLAM loop.

[0082] The innovation of this invention lies in the introduction of controllable regularization into the CB-KNN dynamic correction rendering process. This regularization is applied only to keyframes to balance efficiency and robustness, while the adaptive K-value balances smoothness and detail preservation. The CB-KNN strategy introduces controllable regularization through dynamic correction of the Gaussian position and color of keyframes, suppressing rendering artifacts caused by Gaussian parameter noise at the source. This solves the problem of "map representation error → rendering distortion → pose tracking failure" in traditional 3DGS-SLAM. Applying this correction strategy only to keyframes while maintaining fast rendering for non-keyframes ensures the robust input required for high-precision pose optimization and avoids computational redundancy caused by full-frame correction, achieving a dynamic balance between efficiency and robustness. The adaptive K-value adjusts the number of neighboring Gaussian selections in real time based on the local Gaussian density and scene complexity. When the scene texture is sparse or the motion is intense, the K-value is increased to enhance the smoothing effect and resist noise interference. When the scene has rich details or a stable structure, the K-value is decreased to preserve fine geometric features. This achieves a precise balance between "noise suppression" and "detail fidelity" in 3D reconstruction, avoiding the problems of "oversmoothing and loss of details" or "insufficient smoothing and residual artifacts" caused by a fixed K-value.

[0083] The scenario in this embodiment is: performing 3D reconstruction of the room0 scene in the synthetic dataset Replica. Figure 4(As shown). 3D reconstruction of the Replica's room0 scene serves two main purposes: firstly, it verifies the accuracy of SLAM algorithms in pose tracking and map building within complex indoor environments, providing quantitative evidence for algorithm iteration; secondly, the high-precision indoor 3D scene model provides a realistic environment for virtual simulation platforms, aiding in the research and testing of technologies such as autonomous driving and robot navigation, and thus holding significant importance for algorithm verification and technology implementation. This invention demonstrates significant advantages in the 3D reconstruction of this scene: firstly, extremely high rendering fidelity, closely matching the RGB image of the real scene, accurately reproducing the details and colors of indoor furniture and decorations; secondly, excellent depth accuracy, with an L1 error of 0.00 in the "Rasterized Depth" and a very high proportion of blue in the "Diff Depth L1" image, indicating almost no difference between the reconstructed depth and the actual depth, providing a reliable geometric basis for pose tracking and scene understanding; and thirdly, outstanding robustness, consistently outputting high-quality reconstruction results even under complex indoor layouts and changing perspectives, meeting the stringent requirements of "high precision and high reliability" for scene reconstruction in fields such as autonomous driving simulation and robot navigation.

[0084] A robust 3D Gaussian sputtering SLAM method based on adaptive kernel smoothing includes the following steps:

[0085] Step 1: Take pictures of the target area to be constructed, and complete the construction of the original 3D Gaussian map based on the RGB-D images of the previous t frames;

[0086] Step 2: Perform CB-KNN enhanced differentiable rendering on the original 3D Gaussian map;

[0087] Step 3: Iteratively optimize the original 3D Gaussian map using a SLAM system based on CB-KNN to complete the map update.

[0088] Step 1 specifically includes: constructing an initial 3D Gaussian map based on the RGB-D images of the previous t frames, back-projecting the pixels of the first frame image into an initial Gaussian set, defining the core parameters of each Gaussian (including center position, opacity, radius, and view-independent color), minimizing the RGB and depth errors through differentiable rendering, iteratively optimizing the initial Gaussian parameters, and forming the original 3D Gaussian map for subsequent SLAM processes (all subsequent temporary parameter modifications will not change this original map).

[0089] Step 2 specifically includes: For the input t+1 frame image, a differentiated rendering strategy is adopted according to whether it is a keyframe: For non-keyframes, the original 3D Gaussian map is directly used to render the RGB image and depth map to meet the real-time requirements of fast pose tracking; For keyframes, the visible Gaussians in the current frame are first filtered and sorted by camera distance, and then the K neighboring Gaussians with the largest contribution are selected for each pixel. The CB-KNN strategy is used to temporarily correct these neighboring Gaussians (including position correction of centroid shift in 2D projection and color smoothing based on contribution weights) to generate a temporary Gaussian set. Finally, a smooth RGB image, depth map and contour map are rendered based on this temporary set to provide reliable input for high-precision pose optimization.

[0090] Step 3 specifically includes: based on the original 3D Gaussian map, with keyframe CB-KNN correction as the core and non-keyframe fast tracking as a supplement, to complete the iterative optimization of the SLAM system: including frame-by-frame camera pose optimization, keyframe optimization based on smoothed rendering maps, and non-keyframe fast optimization based on the original rendering maps; identifying low-density or new geometric structure regions through densification masks, supplementing with new Gaussians to improve the map; adaptively determining the K value of CB-KNN by combining local Gaussian density and scene complexity, selecting keyframes and applying CB-KNN only to them; selecting the keyframe group with the highest overlap, optimizing the original map parameters through differentiable rendering and pruning invalid Gaussians to complete the map update and form a complete SLAM loop.

[0091] (I) Detailed Implementation of Initial 3D Gaussian Map Construction

[0092] 1. Definition and parameter initialization of Gaussian sets

[0093] The initial map is constructed based on the first t frames of RGB-D images. The specific steps are as follows:

[0094] 1-1. Initial Gaussian Generation: Backproject all pixels of the first frame of the RGB-D image into 3D points using camera intrinsics. Initialize a Gaussian set for each 3D point. The parameters of a single Gaussian are defined as follows: ,in:

[0095] : The 3D center position of Gauss, i.e., the coordinates of the 3D point obtained by back projection;

[0096] The Gaussian opacity, initially set to 0.5;

[0097] The radius of the Gaussian shape is adaptively set based on the depth value corresponding to the pixel (the greater the depth, the larger the initial radius value, ranging from 0.1 to 0.5).

[0098] Gaussian perspective is independent of RGB color and is directly taken from the RGB value of the corresponding pixel in the first frame.

[0099] 1-2. Gaussian Parameter Optimization: The initial Gaussian set is rendered into an RGB image and a depth image using differentiable rendering. The RGB error and depth error between the rendered result and the ground truth image of the first frame are calculated. The Adam optimizer is then used to iteratively optimize the Gaussian set. The parameters are set with a learning rate of 1e-4 until the error converges, resulting in the original 3D Gaussian map.

[0100] (II) Detailed Implementation of CB-KNN Enhanced Differentiable Rendering

[0101] For non-critical frames, with "real-time performance" as the core objective, the specific steps are as follows:

[0102] 2-1-1. Visible Gaussian Filtering: Gaussians outside the current camera's field of view are removed by frustum clipping, retaining only visible Gaussians;

[0103] 2-1-2. Direct Rendering: Using the Original Gaussian Set RGB and depth maps are generated based on the standard differentiable rendering pipeline of 3DGS (without CB-KNN correction);

[0104] 2-1-3. Fast Pose Initialization: Based on a Constant Velocity Model ( + ( - ), Initialize the pose of frame t+1 (for the camera pose of frame t) by minimizing the color and depth of the rendered image compared to the ground truth image of the input frame. "Loss" optimization pose, single frame processing time is controlled within 23ms (RTX A40 graphics card).

[0105] 2. CB-KNN Correction Rendering Process for Keyframes

[0106] For keyframes, with "robustness" as the core objective, CB-KNN correction is introduced. The specific steps are as follows:

[0107] 2-2-1. Visible Gaussian filtering and sorting: Same as step 2-1-1 for non-keyframes, filter visible Gaussians, and then sort the visible Gaussians in ascending order according to "Euclidean distance from the center of the Gaussian to the optical center of the camera" to ensure that near-field Gaussians participate in subsequent calculations first;

[0108] 2-2-2. K-Nearest Neighbor Gaussian Selection: For each pixel in the current keyframe ( ), Calculate the contribution of all visible Gaussians to the pixel using the Gaussian decay function:

[0109]

[0110] These are the 3D coordinates corresponding to the pixels.

[0111] Then select the one with the largest contribution value. A set of Gaussians .

[0112] 2-2-3. CB-KNN Correction:

[0113] Position correction: First, 3D center of each Gaussian Converted to 2D pixel coordinates using a projection function, the projection function... (·)for:

[0114] ( )=

[0115] in, Camera external parameters, camera internal parameters .

[0116] Recalculate 2D projection centroid:

[0117]

[0118] Finally, the 2D projection of each Gaussian is directed to... Offset, to obtain the corrected 2D coordinates:

[0119]

[0120] [0.1, 0.3] are offset coefficients. = To avoid the denominator being zero, This is a temporary 3D position correction, used only for rendering the current frame and does not update the original map. .

[0121] Color correction: calculated based on Gaussian decay function. Each Gaussian pair of pixels Contribution weight:

[0122]

[0123] Then, the corrected color is calculated using a weighted average:

[0124]

[0125] This weighting ensures that the large Gaussian contribution dominates the color, while suppressing anomalous Gaussian color noise.

[0126] 2-2-4. Rendering smoothing: based on a corrected temporary Gaussian set Each of these is rendered to generate a smooth RGB image, depth image, and contour image:

[0127] RGB image: uses an alpha synthesis mechanism, the formula is:

[0128]

[0129] Depth map: Employs a weighted average mechanism, the formula is as follows:

[0130]

[0131] To correct the depth of the Gaussian in the camera coordinate system;

[0132] Contour map: A coverage calculation mechanism is used, the formula is:

[0133]

[0134] (III) Detailed Implementation of Iterative Optimization of SLAM System Based on CB-KNN

[0135] 1. Camera pose optimization

[0136] The optimization will be carried out according to the principle of "high-precision optimization of key frames and rapid optimization of non-key frames".

[0137] 3-1-1. Non-keyframe pose optimization: Using the original rendering image generated in step 2-1-2 as input, minimize the "color loss and depth loss" and iteratively optimize the pose to ensure real-time performance;

[0138] 3-1-2. Keyframe Pose Optimization: Using the smoothed map generated in step 2-2-4 as input, only the contour map coverage is used. For each pixel, minimize the weighted loss function:

[0139]

[0140] , Given the ground truth depth and color of the input frame, This refers to the pixel coverage of the outline map. This is due to the loss in depth rendering and color rendering.

[0141] 2. Gaussian compaction

[0142] The specific steps for using Gaussian maps to supplement low-density or new geometrically structured regions are as follows:

[0143] 3-2-1. Compacted Mask Generation: Calculate the compacted mask based on the camera pose obtained from tracking. The formula is:

[0144]

[0145] The error threshold is denoted as MDE, where MDE is the median depth error. When the pixel is in the "low-density region", the corresponding 3D area is marked as the "region to be densified"; in the "low-density region" marked by the mask ( ) or “new geometric structure region” (depth error exceeds threshold).

[0146] 3-2-2. Adding a New Gaussian: Back-project the pixels of the "region to be densified" into 3D points, initialize the new Gaussian parameters according to the initial Gaussian parameter rules in step 1-1, and add them to the original map. middle.

[0147] 3. Adaptive K-value keyframe selection

[0148] 3-3-1. Keyframe filtering frequency: From the continuously input RGB-D sequence, candidate keyframes are filtered at a fixed frequency of "selecting 1 frame every n frames".

[0149] 3-3-2. Adaptive Calculation of K Value: The K value (number of neighboring Gaussians in CB-KNN) is derived from the local Gaussian density. and scene complexity Joint decision:

[0150] Local Gaussian density Divide the candidate keyframe image into an 8×8 pixel grid and count the number of Gaussian projections within each grid. ,but (64 is the number of grid pixels);

[0151] Scene complexity : Calculate the pose difference between the candidate keyframe and the previous keyframe (translation distance and rotation angle normalized to the [0,1] interval), as... ;

[0152] K value calculation:

[0153]

[0154] As the baseline value, To adjust the coefficient, Avoid dividing by zero.

[0155] 3-3-3. Keyframe Confirmation: The depth back-projection point cloud of the candidate keyframe is smoothed by CB-KNN, the overlap between the candidate keyframe and the current frame is calculated (the proportion of points in the current frame point cloud that fall within the view frustum of the candidate keyframe is counted), and the candidate keyframe with the highest overlap is selected as the final keyframe, and CB-KNN correction is applied only to it.

[0156] 4. Map Update

[0157] Select the "current frame + latest keyframe + k-2 keyframes with the highest overlap with the current frame", apply CB-KNN correction to these keyframes; iteratively update the original Gaussian map parameters by minimizing RGB error (including SSIM loss) and depth error through differentiable rendering. Pruning invalid Gaussians (opause close to 0 or size too large) completes map update.

[0158] The examples cover both synthetic and real datasets to verify the universality and advantages of the technology from multiple dimensions: For the synthetic dataset Replica, the room0 scenario ( Figure 4 As shown in the figure, focusing on 3D reconstruction accuracy analysis, this invention demonstrates three core advantages: First, its rendering fidelity is leading, with the PSNR of the reconstructed RGB images reaching 34.71dB and 33.52dB respectively, perfectly aligned with the colors, textures, and furniture details of the real scene, almost indistinguishable to the naked eye; Second, its depth accuracy is nearly lossless, with an L1 error of 0.00 in "Rasterized Depth" and blue accounting for over 99% in the "Diff Depth L1" image, making the difference between the reconstructed depth and the real depth negligible, providing millimeter-level reliable geometric support for subsequent pose tracking; Third, it exhibits strong robustness to complex perspectives, even when there are many similar textures in the scene (such as sofas and walls) or when the perspective changes drastically, it can still stably output high-fidelity reconstruction results, breaking through the bottleneck of "detail loss and depth drift" in the traditional 3DGS method in the composite scene.

[0159] Figure 2 In the image, (a) shows the renderings of the novel and training views of the existing SplaTAM method, (b) shows the rendering of the method of the present invention, and (c) shows the ground truth image. It can be seen that the present invention does not have image blurring under both views, and the fidelity of the color image and the depth image is closer to the ground truth.

[0160] Figure 3 The trajectories of Replica (R1, Of1 scenarios), TUM-RGBD (fr1 / desk, fr3 / off scenarios), and ScanNet (0106, 0207 scenarios) are shown respectively. Compared with the existing SplaTAM method, the trajectory of the present invention is closer to the true value in all scenarios and has no obvious pose drift.

[0161] For the real-world dataset TUM-RGBD (fr1 / desk, fr3 / off scenario), Figure 1 As shown, Figure 1 The left side of the image shows the trajectory of the existing SplaTAM method, and the right side shows the trajectory of the method of this invention. It can be seen that the trajectory of this invention is smoother and has a smaller deviation from the ground-truth trajectory. ScanNet (0106 and 0207 scenes, as shown in Figure 3) achieves a breakthrough in the robustness of the focusing and localization performance: In the dynamic noise and fast motion scene of TUM-RGBD, the ATE RMSE (mean absolute trajectory error) of this invention is reduced from 3.31cm to 3.18cm in the existing SplaTAM method, the trajectory drift is reduced by 15%, and no tracking loss is achieved throughout the fr1 / desk scene; In the large-scale, texture-poor scene of ScanNet, the ATE RMSE of the 0181 scene is reduced from 12.08cm to 11.91cm, the number of tracking losses is reduced by 60%, and even in the 0207 scene with a large number of dynamic objects (such as pedestrians and moving objects) interference, the stability of pose tracking can still be maintained. This invention demonstrates a dual advantage: "top-tier accuracy in synthetic scene reconstruction + breakthrough in robust positioning in real-world scenarios." It solves the industry problem of "insufficient decoupling between reconstruction and tracking" in 3DGS-SLAM synthetic data, and overcomes the pain point of "pose drift caused by dynamic interference and texture loss" in real-world scenarios. It provides an integrated technical solution that combines "high-precision reconstruction" and "high-reliability positioning" for fields such as autonomous driving simulation testing and robot indoor navigation.

[0162] This invention provides a robust 3D Gaussian sputtering SLAM method based on adaptive kernel smoothing. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A robust 3D Gaussian sputtering SLAM method based on adaptive kernel smoothing, characterized in that, Includes the following steps: Step 1: Take pictures of the target area to be constructed, and complete the construction of the original 3D Gaussian map based on the RGB-D images of the previous t frames; Step 2: Perform CB-KNN enhanced differentiable rendering on the original 3D Gaussian map; Step 3: Iteratively optimize the original 3D Gaussian map using a SLAM system based on CB-KNN to complete the map update.

2. The robust 3D Gaussian sputtering SLAM method based on adaptive kernel smoothing according to claim 1, characterized in that, Step 1 specifically includes: constructing an initial 3D Gaussian map based on the first t frames of RGB-D images, back-projecting the pixels of the first frame image into an initial Gaussian set, defining the core parameters of each Gaussian and minimizing the RGB and depth errors through differentiable rendering, iteratively optimizing the initial Gaussian parameters, and forming the original 3D Gaussian map for subsequent SLAM processes.

3. The robust 3D Gaussian sputtering SLAM method based on adaptive kernel smoothing according to claim 1, characterized in that, Step 2 specifically includes: For the input t+1 frame image, a differentiated rendering strategy is adopted according to whether it is a key frame: For non-key frames, the original 3D Gaussian map is used directly to render the RGB image and depth map; for key frames, the visible Gaussians in the current frame are first filtered and sorted by camera distance, and then the K neighboring Gaussians with the largest contribution are selected for each pixel. The CB-KNN strategy is used to temporarily correct these neighboring Gaussians to generate a temporary Gaussian set. Finally, a smooth RGB image, depth map and contour map are rendered based on the temporary Gaussian set, providing input for high-precision pose optimization.

4. The robust 3D Gaussian sputtering SLAM method based on adaptive kernel smoothing according to claim 1, characterized in that, Step 3 specifically includes: based on the original 3D Gaussian map, with keyframe CB-KNN correction as the core and non-keyframe fast tracking as a supplement, to complete the iterative optimization of the SLAM system: including frame-by-frame camera pose optimization, keyframe optimization based on smoothed rendering maps, and non-keyframe fast optimization based on the original rendering maps; identifying low-density or new geometric structure regions through densification masks, supplementing with new Gaussians to improve the map; adaptively determining the K value of CB-KNN by combining local Gaussian density and scene complexity, selecting keyframes and applying CB-KNN only to them; selecting the keyframe group with the highest overlap, optimizing the original map parameters through differentiable rendering and pruning invalid Gaussians to complete the map update and form a complete SLAM loop.

5. A robust 3D Gaussian sputtering SLAM method based on adaptive kernel smoothing according to claim 1, characterized in that, The initial map is constructed based on the first t frames of RGB-D images. The specific steps are as follows: Step 1-1: Initial Gaussian Generation: Back-project all pixels of the first frame RGB-D image into 3D points using camera intrinsics. Initialize a Gaussian primitive for each 3D point, forming a set of Gaussian primitives. Steps 1-2: Gaussian parameter optimization: Render the initial Gaussian set into an RGB image and a depth image using differentiable rendering. Calculate the RGB and depth errors between the rendered result and the ground truth image of the first frame. Iteratively optimize the Gaussian set using the Adam optimizer. The parameters are adjusted until the error converges, resulting in the original 3D Gaussian map.

6. A robust 3D Gaussian sputtering SLAM method based on adaptive kernel smoothing according to claim 3, characterized in that, For non-keyframes, the original 3D Gaussian map is used directly to render the RGB image and depth map. The specific steps are as follows: 2-1-1 Visible Gaussian Filtering: By using the view frustum clipping method, Gaussians outside the current camera's field of view are removed, and only visible Gaussians are retained; 2-1-2 Direct Rendering: Using the original Gaussian set G, RGB and depth maps are generated based on the standard differentiable rendering process of 3DGS; 2-1-3. Fast pose initialization: The pose of frame t+1 is initialized based on the constant velocity model. The pose is optimized by minimizing the L1 loss of color and depth between the rendered graph and the ground truth graph of the input frame.

7. A robust 3D Gaussian sputtering SLAM method based on adaptive kernel smoothing according to claim 3, characterized in that, For keyframes, CB-KNN correction is introduced, and the specific steps are as follows: 2-2-1. Visible Gaussian filtering and sorting: Same as step 2-1-1 for non-keyframes, filter visible Gaussians, and then sort the visible Gaussians in ascending order according to the Euclidean distance from the center of the Gaussian to the optical center of the camera, to ensure that near-field Gaussians participate in subsequent calculations first. 2-2-2, K-Nearest Neighbor Gaussian Selection: For each pixel in the current keyframe, calculate the contribution value of all visible Gaussians to that pixel; 2-2-3. For Gaussian sets Perform CB-KNN correction; 2-2-4. Rendering smoothing: Based on the corrected temporary Gaussian set The system renders smooth RGB images, depth images, and contour images respectively.

8. A robust 3D Gaussian sputtering SLAM method based on adaptive kernel smoothing according to claim 7, characterized in that, Step 2-2-3 specifically includes: Position correction: First, adjust the Gaussian set. 3D center of each Gaussian The coordinates are converted to 2D pixel coordinates using a projection function, and then the Gaussian set is calculated. 2D projection centroid Finally, project each Gaussian 2D onto its 2D projection centroid. Offset, to obtain the corrected 2D coordinates: Color correction: Calculating Gaussian sets based on the Gaussian decay function Each Gaussian pair of pixels Contribution weight The corrected color is then calculated using a weighted average.

9. A robust 3D Gaussian sputtering SLAM method based on adaptive kernel smoothing according to claim 4, characterized in that, The camera pose optimization is performed according to the principle of "high-precision optimization of key frames and fast optimization of non-key frames": 3-1-1. Non-keyframe pose optimization: Using the original rendering image generated in step 2-1-2 as input, minimize color loss and depth loss, and iteratively optimize the pose; 3-1-2. Keyframe pose optimization: Using the smoothed map generated in step 2-2-4 as input, only the contour map coverage is used. For pixels with a threshold, minimize the weighted loss function. .

10. A robust 3D Gaussian sputtering SLAM method based on adaptive kernel smoothing according to claim 3, characterized in that, The specific method for determining whether a frame is a keyframe is adaptive K-value keyframe selection: 3-3-1 Keyframe Filtering Frequency: Select candidate keyframes from the continuously input RGB-D sequence at a fixed frequency of 1 frame every n frames; 3-3-2 Adaptive Calculation of K Value: K value is the number of neighboring Gaussians in CB-KNN, derived from the local Gaussian density. and scene complexity Joint decision: The local Gaussian density Divide the candidate keyframe image into a pixel grid and count the number of Gaussian projections within each grid. ,but N is the number of grid pixels; The complexity of the scenario Calculate the pose difference between the candidate keyframe and the previous keyframe, as... ; The K value is calculated as follows: ； in, As the baseline value, To adjust the coefficient, To avoid division by zero constants; 3-3-3 Keyframe Confirmation: The depth back-projection point cloud of the candidate keyframe is smoothed by CB-KNN, the overlap between the candidate keyframe and the current frame is calculated, and the candidate keyframe with the highest overlap is selected as the final keyframe. CB-KNN correction is applied only to it.

Images