A semantic Gaussian sputtering dynamic RGB-D SLAM method and system with loop closure optimization

Abstract

The application relates to a semantic Gaussian splatting dynamic RGB-D SLAM method with loop closure optimization and a system thereof, and belongs to the technical field of embodied intelligence, which comprises the following steps: a semantic segmentation model is used to preliminarily extract a mask of dynamic objects and a motion area; based on semantic segmentation and a contour mask, dynamic and uncertain areas are removed, high-confidence static pixels are screened out, depth loss, color loss and semantic loss are jointly optimized, and robust iterative estimation of a current camera pose is realized; a DBoW2 bag-of-words retrieval method is used to retrieve loop candidate frames and suppress false matching; geometric fine registration is carried out based on ICP, and a robust loop detection constraint is generated; global calibration of a full-frame pose is completed based on PGO; a three-dimensional Gaussian primitive is fully reconstructed based on an optimal pose, a dense semantic three-dimensional Gaussian map is constructed, a dynamic point mask is constructed based on a semantic label, a Gaussian splatting element that is dynamic and geometrically invalid is pruned, and a high-precision static scene map is constructed.

Description

Technical Field

[0001] This invention belongs to the field of embodied intelligence technology and relates to a semantic Gaussian sputtering dynamic RGB-D SLAM method and system with loop closure optimization. Background Technology

[0002] Traditional SLAM (Simultaneous Localization and Mapping) systems typically employ explicit representations, such as voxel grids and point clouds. While these methods excel in tracking accuracy and efficiency, they have limitations in generating high-fidelity maps. To improve tracking robustness, some methods introduce loop closure detection to reduce accumulated errors. For example, ORB-SLAM2 uses a visual bag-of-words approach to retrieve loop closure candidate frames, performs geometric verification, adds pose constraints to the pose graph, and eliminates drift and reduces accumulated errors through graph optimization methods such as G2O (General Graph Optimization). SLAM++, on the other hand, identifies and tracks semantic targets in the scene, utilizing the geometric and semantic information of the targets to achieve loop closure detection and optimization.

[0003] In recent years, NeRF (Neural Radiance Fields) has received widespread attention in the field of SLAM algorithms. Methods such as iMAP (Implicit Mapping and Positioning), NICE-SLAM (Neural Implicit Scalable Encoding SLAM), and ESLAM utilize neural implicit representations to achieve accurate and dense 3D surface reconstruction. However, these neural implicit methods are limited by the high computational cost of volume rendering, making robust tracking difficult in large-scale environments. To improve tracking robustness, some methods introduce loop closure detection and online bundle adjustment (BA) to reduce accumulated errors. For example, MIPS-Fusion uses multiple implicit subgraph representations, achieving global optimization through subgraph refinement and fusion; while Go-SLAM combines loop closure detection with online global BA for all keyframes to ensure global consistency in large-scale environments.

[0004] SLAM methods based on 3D Gaussian representation have received significant attention in recent years due to their ability to combine the advantages of explicit and implicit representations. Gaussian-SLAM organizes the scene into a 3D Gaussian subgraph, thereby achieving efficient optimization and avoiding catastrophic forgetting. SplaTAM employs a simplified 3D Gaussian representation, supporting real-time efficient optimization and high-quality rendering. SGS-SLAM is the first semantic visual SLAM system based on 3D Gaussian sputtering. It uses a differentiable Gaussian kernel to represent the scene from the bottom layer and combines online semantic segmentation with a semantically guided keyframe selection strategy. Through multi-channel constraint optimization, this method achieves robust tracking and real-time high-quality rendering in dynamic environments. However, these methods all lack online loop closure correction, leading to error accumulation and map drift. In contrast, Photo-SLAM is based on ORB-SLAM and integrates loop closure correction to reduce accumulated errors and enhance tracking robustness, but its design decouples tracking and mapping, reducing the effectiveness of loop closure correction and increasing communication overhead. The loop closure detection and pose graph optimization frameworks widely used in SLAM systems typically focus primarily on optimizing and correcting keyframe poses, while neglecting the posterior update and refinement of map elements. This can lead to a certain degree of inconsistency between the map representation and the optimized pose, resulting in a deviation problem after optimization: the pose has been corrected, but the map is still distorted. Even if the camera pose is adjusted to global consistency through pose graph optimization, map elements generated earlier in the drift pose still retain their original errors and cannot be accurately aligned with the corrected pose, ultimately reducing mapping accuracy. Taking the classic RGB-D SLAM system ORB-SLAM2 as an example, its loop closure process typically includes loop closure candidate frame retrieval based on visual bag-of-words, geometric consistency verification, and global graph optimization. Its global bundle adjustment mechanism can update sparse feature point coordinates, but full joint optimization of millions of Gaussian elements is computationally infeasible; while directly skipping map updates causes the explicit dense representation of 3D Gaussian Splatting (3DGS) to lose its global consistency advantage. Therefore, a pose-map decoupled reconstruction mechanism adapted to the characteristics of 3DGS is urgently needed. However, existing 3DGS-SLAM methods have not fully utilized the advantages of the classic loop closure optimization framework, which consists of visual bag-of-words loop closure candidate retrieval, geometric registration, and pose graph optimization, in terms of efficient retrieval and global consistency optimization. They have also failed to address the shortcomings of pose-map optimization decoupling, making it difficult to adapt to the dense representation characteristics of 3D Gaussian sputtering.

[0005] Dynamic target filtering is crucial for robustness in static scene reconstruction and pose estimation. Existing methods mainly fall into two categories: the first relies on resampling and residual optimization strategies to remove outliers, such as ORB-SLAM2, ORB-SLAM3, and Refusion. However, these methods typically only handle small-scale motion, and their performance often degrades significantly when faced with large-scale, continuous object motion. The second category of methods leverages additional prior knowledge (such as semantic segmentation or object detection priors) to remove dynamic targets. However, these methods often suffer from domain-specific limitations in practical applications, leading to prediction errors.

[0006] Neural implicit scene representations have garnered significant attention in the RGB-D SLAM field due to their superior expressive power and low memory footprint. In recent years, existing methods such as Vox-Fusion, Co-SLAM, and ESLAM have explored sparse coding or triplane representation strategies to improve scene reconstruction quality and system execution efficiency. Point-SLAM, drawing inspiration from Point-NeRF, uses neural points to encode spatial geometric and color features, constructing a spatial map explicitly, effectively improving localization and mapping accuracy. These methods are all based on strong assumptions about static scenes and have achieved remarkable results. However, the robustness of these systems significantly decreases when dynamic objects appear in the environment. Recently, Dn-SLAM uses optical flow estimation to cull dynamic feature points at the front end, correcting tracking errors in a manner similar to traditional methods, but neglecting the rendering artifacts caused by noise input at the back end. Ddn-SLAM combines deep learning-based detection with depth segmentation and introduces an additional rendering penalty term.

[0007] Compared to the NeRF-based SLAM methods mentioned above, 3D Gaussian sputtering has garnered significant research attention due to its high-fidelity reconstruction, differentiable parameters, and ability to represent fine-grained scene details. Some researchers have explored using 3DGS to replace the implicit representation NeRF in the map building stage. However, these methods are currently still limited by the assumption of a static environment and perform poorly in dynamic scenes, severely restricting the practical application of Gaussian SLAM systems in real-world environments. Summary of the Invention

[0008] In view of this, the purpose of this invention is to provide a semantic Gaussian sputtering dynamic RGB-D SLAM method and system with loop closure optimization.

[0009] To achieve the above objectives, the present invention provides the following technical solution: On one hand, this invention provides a semantic Gaussian sputtering dynamic RGB-D SLAM method with loop closure optimization, comprising the following steps: S1: Use a semantic segmentation model to initially extract the masks of dynamic objects and motion regions; based on semantic segmentation and contour masks, remove dynamic and uncertain regions, select high-confidence static pixels, and jointly optimize depth loss, color loss and semantic loss to achieve robust iterative estimation of the current camera pose; S2: Use the Distributed Bag of Words 2 (DBoW2) retrieval method to retrieve loop closure candidate frames and suppress false matches; S3: Perform fine geometric registration based on the Iterative Closest Point (ICP) algorithm to generate robust loop closure detection constraints; S4: Global calibration of the entire frame pose is completed based on Pose Graph Optimization (PGO); S5: Perform full posterior reconstruction of 3D Gaussian primitives with fixed optimal pose to construct a dense semantic 3D Gaussian map; S6: Construct a dynamic dot mask based on semantic tags, prune dynamic and geometrically invalid Gaussian sputtering elements, retain only valid static Gaussian sputtering elements, and finally construct a high-precision static scene map.

[0010] Furthermore, in step S1, for the initial frame, the camera pose is initialized as an identity matrix and used as the reference coordinate system for subsequent tracking and mapping processes; when estimating the camera pose for the RGB-D (Red Green Blue - Depth) frame at the new timestamp, the displacement of the previous frame is accumulated based on the constant velocity assumption to calculate the initial pose, following the relational formula. , Indicates the first Initial estimation of camera pose for a frame. Indicates the first The camera pose of the frame. Indicates the first Initial estimation of camera pose for each frame; Based on semantic segmentation and contour masking to remove dynamic and uncertain regions, high-confidence static pixels are selected. The Adam adaptive optimization algorithm based on gradient descent is used to optimize the factor graph, minimizing the tracking loss caused by depth, color and semantic errors. The depth loss, color loss and semantic loss are jointly optimized to achieve robust iterative estimation of the current camera pose. The factor graph takes the camera pose as the core state node and the multimodal tracking loss as the constraint factor.

[0011] Furthermore, step S1 specifically includes the following steps: First, based on the semantic segmentation results, pixels corresponding to preset dynamic categories are labeled as dynamic pixels, generating pixel-level dynamic masks. The dynamic mask is then fused with the depth-valid mask and the result of the NaN filtering. If the contour visibility mask is enabled, it is also included in the fusion process to obtain the final effective mask. In the pixel-level loss function used for camera pose optimization, depth, color, and semantic errors are only considered when the effective mask satisfies... Accumulation is performed within a pixel range to ensure that the pixels of moving objects do not generate gradient information that misleads pose estimation; Iterative optimization of the current camera pose is achieved by minimizing the tracking loss. The tracking loss function measures the deviation between the ground truth values ​​of depth, color, and semantic map and their differentiable rendered values, where the ground truth value of depth is... The true value of color The true value of the semantic map The loss function expression is as follows:

[0012] in This represents the pixel color rendered from the Gaussian map at the current pose. This represents the depth rendered at the current pose. This represents the semantic result obtained by rendering in the current pose. Represents the outline value of a pixel. , These represent the depth loss weight, color loss weight, and semantic loss weight, respectively. In this weighted summation mechanism, the rationality of multimodal fusion is based on a unified rendering space and probabilistic optimization underlying design: firstly, by assigning an independent channel to each 3D Gaussian volume, abstract semantic information is explicitly transformed into a semantic color vector isomorphic to the appearance, enabling color and semantic errors to be calculated using the same metric on the 2D image plane, achieving dimensionlessness and physical space unification at the feature level; the weight coefficients for each... , It also serves as a scale conversion factor, which, within the probabilistic optimization framework, is equivalent to compensating for the inverse of the variance of the observation noise of each sensor signal. This offsets the residual numerical magnitude differences between depth, color, and semantics, allowing multimodal observation errors to be fully and legitimately mapped to a unified error energy space for same-scale accumulation and joint optimization; contour threshold The settings are designed to utilize a map that has been optimized in advance.

[0013] Furthermore, step S2 specifically includes the following steps: Insert a keyframe when frame With initial frame and fixed time interval or the second to last frame in the sequence. When they meet, insert them into the keyframe set, where For preset intervals, Total number of frames; each keyframe records the camera-world-human transformation. The corresponding RGB / grayscale image and optional semantic mask; where Indicates the first The camera pose corresponding to a keyframe represents a rigid body transformation from the world coordinate system to the camera coordinate system. For keyframe grayscale images First, key points are detected and a set of binary descriptors are extracted. in Indicates the first The first frame One key point, Indicates the first Frames One key point, ; using offline training to obtain A hierarchical dictionary of visual words, with each descriptor Quantize to the nearest leaf node and count the frequency of each word. Then, a bag-of-words (BoW) vector is constructed, and a weighted term frequency-inverse document frequency (WF-IF) is calculated to generate a sparse vector:

[0014] In the formula, It represents the set of visual words, containing all possible visual words; For visual words In the keyframe set Document frequency in It is a dummy variable used to iterate over the collection. All words, avoid and current words Confusing.

[0015] Furthermore, step S3 specifically includes the following steps: Calculate the current number Frame keyframe bag-of-words vector With all historical keyframes bag-of-words vectors Cosine similarity:

[0016] like And inter-frame interval Then the frame pair Add to the candidate set of loops ,in The threshold for cosine similarity is... The keyframe time interval is used as the filtering threshold; for candidate loop closure pairs Generate source point cloud With target point cloud Source cloud Corresponding keyframe 3D point cloud, For the first point cloud Points, target point cloud Corresponding keyframe 3D point cloud, For the first point cloud 1 point, and calculate the initial relative pose:

[0017] in Keyframe The world-to-camera transformation matrix, Keyframe The world-to-camera transformation matrix describes the camera's pose in the world coordinate system, and this result provides initial values ​​for subsequent error minimization. Iteratively update the transformation matrix Align the source point cloud with the target point cloud after transformation, where For rotation matrix, The translation vector is used; in each iteration, the source point cloud is minimized. With target point cloud The transformation matrix is ​​updated based on the point-to-point error between points. The optimization objective is to minimize the sum of squared errors of all point pairs between point clouds. The objective function is:

[0018] In the formula, The source point after transformation , As the source The corresponding target point Let be the transformation matrix to be optimized; Iteratively update the rotation matrix Translation vector To minimize the error, the source point cloud is first calculated to solve for the updated transformation matrix. With target point cloud center of mass and ,Right now:

[0019] The optimal rotation matrix is ​​then calculated using the centroid. For symmetric matrices Performing Singular Value Decomposition (SVD) yields the rotation matrix:

[0020] right SVD decomposition Then the rotation matrix is:

[0021] Then, the translation vector is calculated from the centroid. :

[0022] The updated transformation matrix Used for source point cloud transformation, and continues to the next round of iteration optimization; After each iteration, the error is calculated to determine whether the termination condition is met. An error threshold is set, and the iteration is terminated when the current error is less than the threshold or the change in error is less than the preset tolerance. If the condition is not met, the iteration continues until the error converges. Finally, when the optimized transformation matrix When the error between the source point cloud and the target point cloud is lower than a set threshold and the matching quality meets the standard, the loop is deemed valid and incorporated into the global optimization process.

[0023] Furthermore, step S4 specifically includes the following steps: Constructing a pose graph , where nodes Corresponding keyframe set, edge set Includes the odometer edge and the loopback edge, where Indicates the odometer side, The covariance matrix of the odometer edge, representing the loop edge, is:

[0024] The covariance of the loop edge is , The coefficients are between 0 and 1; then pose graph optimization is performed to minimize the weighted pose error function:

[0025] In the formula, These are adjacent keyframe index pairs for the odometer edge. These are non-adjacent keyframe index pairs for loop edges. The index of the keyframe, that is, the index of the first keyframe. The keyframe number; , and Representing keyframes and The current pose, Keyframe arrive The measurement pose constraints are determined; the Gauss-Newton method is used for global optimization to obtain the optimized keyframe pose; the optimized pose matrix is ​​decomposed into rotation matrix and translation vector, converted into quaternion form and synchronously updated to the system state parameters to complete the loop closure and drift correction.

[0026] Furthermore, in step S5, with the optimized pose as a rigid constraint, full-pixel Gaussian initialization is performed based on RGB-D semantic prior. Frame-by-frame dynamic densification is triggered by differentiable contour rendering and depth inconsistency masking to complete the full posterior reconstruction of the map and completely eliminate historical accumulated errors.

[0027] Furthermore, in step S6, after mapping, a dynamic point mask is constructed based on semantic labels to mark Gaussian points belonging to the dynamic category; a preset dynamic category set is set as follows. ,in for A preset dynamic category is used to determine the Gaussian point. The dynamic attribute is determined by the following formula:

[0028] in, Indicates the first The semantic label corresponding to each Gaussian point indicates which object category this Gaussian point is identified as; This indicates an indicator function that returns 1 if the condition within the parentheses is true, and 0 if the condition is false. For semantic dynamic masking, when When, it indicates the first Each Gaussian point belongs to the dynamic category; The joint pruning rules are constructed, including the following three types of decision conditions: Semantic dynamic conditions: through semantic dynamic masks Mark dynamic Gaussian points; Transparency validity conditions: Set a transparency threshold The determination formula is:

[0029] in, Indicates the first A Gaussian point transparency parameter, For transparency mask; Scale validity condition: If a scale threshold is set The judgment is as follows:

[0030] in, , and These are three preset scale thresholds, which respectively limit... , , Maximum permissible dimensions of the shaft; For scale mask, symbols Represents a logical OR operation; By integrating three types of decision conditions, a total pruning mask is defined. Its expression is:

[0031] when At that time, the first A Gaussian point was marked as a point to be pruned; Pruning is triggered periodically only within a preset iteration interval, with a total number of iterations of [number missing]. The initial iteration step for pruning is The final iteration step is The pruning cycle is The trigger condition for the pruning operation is:

[0032] When the triggering condition is met, perform a pruning operation: retain the total pruning mask. The Gaussian points are then updated, and the Gaussian point parameter set is updated as follows:

[0033] in Indicates the first One Gaussian point; At the same time, the parameters of the optimizer are adjusted to match the number of Gaussian points after pruning, so as to ensure the continuity and stability of the model training process.

[0034] On the other hand, the present invention provides a semantic Gaussian sputtering dynamic RGB-D SLAM system with loop closure optimization, including a semantic mask tracking module, a loop closure optimization module and a mapping module; The semantic mask tracking module is used to initially extract the masks of dynamic objects and motion regions using a semantic segmentation model; based on semantic segmentation and contour masking, dynamic and uncertain regions are eliminated, high-confidence static pixels are selected, and depth loss, color loss and semantic loss are jointly optimized to achieve robust iterative estimation of the current camera pose. The loop closure optimization module is used to efficiently retrieve loop closure candidate frames and suppress false matches using the DBoW2 bag-of-words retrieval method; to perform geometric fine registration based on the iterative nearest point algorithm to generate robust loop closure detection constraints; and to complete the global calibration of the full-frame pose based on pose graph optimization. The mapping module is used to perform Gaussian initialization, Gaussian densification, and multi-channel joint optimization. Finally, it performs dynamic Gaussian point pruning based on semantics to construct a dense semantic 3D Gaussian map.

[0035] Furthermore, the mapping module also constructs a dynamic point mask based on semantic tags, prunes dynamic and geometrically invalid Gaussian sputtering elements, retains only valid static Gaussian sputtering elements, and finally constructs a high-precision static scene map.

[0036] The beneficial effects of this invention are as follows: The entire process of this invention forms a closed-loop mechanism of pose-first optimization and full-scale posterior reconstruction of the map, which fundamentally solves the defect of traditional loop closure methods where the pose has been corrected but the map is still distorted, and significantly improves the tracking stability of the algorithm in long-term operation and the global consistency of scene reconstruction; This invention achieves full-link elimination of dynamic interference, has dual robustness in both tracking and mapping stages, and introduces lightweight constraints to ensure the actual deployment capability of the system.

[0037] This invention addresses the problems of error accumulation and map drift caused by the lack of online loop closure correction in existing 3DGS-SLAM methods, as well as the issues of pose-map inconsistency and reduced mapping accuracy caused by traditional loop closure optimization frameworks failing to adapt to the dense representation characteristics of 3DGS, which only correct pose without posterior reconstruction of map elements. Furthermore, 3DGS-SLAM is limited by static environment assumptions and performs poorly in dynamic scenes, with dynamic targets easily interfering with pose estimation and rendering artifacts easily occurring in the backend. By solving these problems and adapting to the dense representation characteristics of 3DGS, this invention achieves global calibration of camera pose and posterior reconstruction of 3D Gaussian elements, improving the long-term tracking stability of the SLAM system and the global consistency of scene reconstruction. It effectively suppresses the interference of dynamic targets on localization and mapping, eliminates dynamic and geometrically invalid Gaussian points in the map, and improves the reconstruction accuracy and fidelity of dense 3D maps in dynamic scenes.

[0038] Experiments on the Replica dataset and real-world datasets show that, in loop closure detection scenarios, the proposed method reduces pose error by 3.18% compared to SGS-SLAM. In dynamic environments, the proposed method reduces pose error by 79.77% compared to SGS-SLAM.

[0039] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0040] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 Flowchart of a semantic Gaussian sputtering dynamic RGB-D SLAM method with loop closure optimization; Figure 2 Optimize the corresponding factor graph for the pose graph; Figure 3 The factor graph corresponding to the semantic mask is used for tracking. Detailed Implementation

[0041] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0042] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0043] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0044] Example 1: like Figure 1 As shown, this invention provides a semantic Gaussian sputtering dynamic RGB-D SLAM method with loop closure optimization, comprising two parts: a 3DGS-SLAM framework based on SGS-SLAM and incorporating two-stage DBoW2+ICP+PGO loop closure optimization, and a semantic-based end-link dynamic interference cancellation method. Details are as follows: The 3DGS-SLAM framework, based on SGS-SLAM and incorporating two-stage DBoW2+ICP+PGO loop closure optimization, first achieves full-frame global pose calibration through visual bag-of-words retrieval, geometric registration, and pose graph optimization. Then, it performs posterior reconstruction of 3D Gaussian primitives by fixing the optimal pose. Based on accurate camera pose, 3D Gaussian density is achieved, and the 3D Gaussian primitive parameters are jointly optimized under multi-channel constraints to reconstruct a dense semantic 3D Gaussian map.

[0045] In the 3DGS-SLAM framework based on SGS-SLAM and incorporating two-stage DBoW2+ICP+PGO loop closure optimization, 3D Gaussian sputtering SLAM adopts a frame-to-model tracking paradigm. It constructs a multi-constraint loss function that integrates photometric, depth, and semantic information through differentiable rendering to iteratively optimize camera pose. However, due to the inevitable observation residuals and gradient optimization errors in single-frame pose estimation, these errors gradually accumulate during continuous frame iterations and propagate and amplify over time, leading to deviations between the system-constructed 3D Gaussian map and the real environment, causing camera trajectory drift and map distortion. Loop closure detection and optimization, by identifying the similarity relationships between the current image and historical keyframes, provide long-distance constraints across time for backend pose map optimization. This constraint effectively suppresses error accumulation and corrects camera trajectory deviations, resulting in more accurate camera trajectories and maps.

[0046] The loop closure detection and pose graph optimization frameworks widely used in SLAM systems, such as methods combining loop closure detection and pose graph optimization based on the visual bag-of-words model, typically focus primarily on optimizing and correcting keyframe poses, while paying insufficient attention to posterior updates and refinement of map elements. This can lead to a certain degree of inconsistency between the map representation and the optimized pose. This results in a disconnect after optimization, where the pose is corrected, but the map remains distorted. Even if the pose is adjusted to global consistency through pose graph optimization, map elements generated earlier based on drifted poses (such as sparse feature points and voxels) still retain their original errors and cannot accurately match the corrected pose, ultimately affecting mapping accuracy. Taking the classic RGB-DSLAM system ORB-SLAM2 as an example, its loop closure process typically includes loop closure candidate frame retrieval based on the visual bag-of-words model, geometric consistency verification, and global graph optimization. Its global bundle adjustment mechanism can update the coordinates of sparse feature points, but full joint optimization of millions of Gaussian elements is computationally infeasible; while directly skipping map updates causes the explicit dense representation of 3DGS to lose its global consistency advantage. Therefore, a pose-map decoupled reconstruction mechanism adapted to the characteristics of 3DGS is urgently needed. However, existing 3DGS-SLAM methods have not fully utilized the advantages of the classic loop closure optimization framework, which consists of loop closure candidate frame retrieval, geometric registration, and pose graph optimization based on the visual bag-of-words model, in terms of efficient retrieval and global consistency optimization. They have also failed to address the defect of pose-map optimization being disconnected, making them unsuitable for the dense representation characteristics of 3DGS.

[0047] To address the aforementioned problems, this invention proposes a two-stage DBoW2+ICP+PGO loop closure detection optimization scheme: First, DBoW2 achieves efficient global keyframe retrieval and mismatch suppression; then, ICP performs geometric fine registration to generate robust loop closure detection constraints; finally, PGO completes global calibration of the full-frame pose. Its core innovation lies in fixing the optimized globally consistent pose and performing full posterior map reconstruction on 3DGS Gaussian cells, forming a complete pose-priority optimization-posterior map reconstruction closed loop. This fundamentally solves the core defect of traditional pose correction where the map remains distorted, perfectly meeting the requirements of 3DGS-SLAM for high-density, high-fidelity map construction. Specifically, it includes the following steps: Step 1: Keyframe selection and feature representation Keyframe insertion follows this strategy: when frame With the initial frame Fixed time interval ( (The preset interval, with a default value of 1) or the second to last frame in the sequence. When they meet, insert them into the keyframe set. Each keyframe records the camera-world-human transformation. (No. The keyframes include the camera pose (representing the rigid body transformation from the world coordinate system to the camera coordinate system), the corresponding RGB / grayscale image, and an optional semantic mask.

[0048] For keyframe grayscale images First, key points are detected and a set of binary descriptors are extracted. ( Indicates the first The first frame One key point, Indicates the first Frames (a few key points), among which Using offline training to obtain... A hierarchical dictionary of visual words, with each descriptor Quantize to the nearest leaf node and count the frequency of each word. Then, a bag-of-words (BoW) vector is constructed, and a weighted term frequency – inverse document frequency is calculated to generate a sparse vector:

[0049] In the formula, A dictionary representing a set of visual words, containing all possible visual words; For visual words In the keyframe set Document frequency in the text; It is a dummy variable used to iterate over the collection. All words, avoid and current words Confusing.

[0050] Step 2: Coarse search of loopback candidate frames Current number Frame keyframe bag-of-words vector With all historical keyframes Cosine similarity of bag-of-words vectors:

[0051] like And inter-frame interval Then the frame pair Add to the candidate set of loops For candidate loop pairs Generate source point cloud (Source point cloud set, corresponding keyframes) 3D point cloud, For the first point cloud (points) and target point cloud (Target point cloud set, corresponding keyframes) 3D point cloud, For the first point cloud (points), and calculate the initial relative pose:

[0052] in Keyframe World-to-Camera transformation matrix. Keyframe The world-to-camera transformation matrix describes the camera's pose in the world coordinate system. This result provides initial values ​​for subsequent error minimization. Step 3: Geometric verification and precise pose estimation Iteratively update the transformation matrix (including rotation matrix) With translation vector This process aligns the transformed source point cloud with the target point cloud. In each iteration, the source point cloud is minimized. With target point cloud The transformation matrix is ​​updated based on the point-to-point error between points. The optimization objective is to minimize the sum of squared errors of all point pairs between point clouds. The objective function is:

[0053] In the formula, The source point after transformation , As the source The corresponding target point Let be the transformation matrix to be optimized; Iteratively update the rotation matrix Translation vector To minimize the error, the source point cloud is first calculated to solve for the updated transformation matrix. With target point cloud center of mass and ,Right now:

[0054] The optimal rotation matrix is ​​then calculated using the centroid. For symmetric matrices Performing singular value decomposition (SVD) yields the rotation matrix:

[0055] right SVD decomposition Then the rotation matrix is:

[0056] Then, the translation vector is calculated from the centroid. :

[0057] The updated transformation matrix Used for source point cloud transformation, and then continue to the next round of iteration optimization.

[0058] After each iteration, the error is calculated, and it is determined whether the termination condition is met. The algorithm typically sets an error threshold. When the current error is less than the threshold, or the change in error is less than a preset tolerance, the iteration terminates; if the condition is not met, the iteration continues until the error converges.

[0059] Finally, when the optimized transformation matrix When the error between the source point cloud and the target point cloud is lower than a set threshold (usually measured by root mean square error RMSE) and the matching quality meets the standard, the loop is deemed valid and incorporated into the global optimization process.

[0060] Step 4: Global pose graph optimization and map posterior reconstruction Factor graphs for pose graph optimization, such as Figure 2 As shown, the camera pose of each keyframe is the core state node, and the odometry constraint and loop closure constraint are the constraint factors. Its structure clearly reflects the inherent logic of the collaborative optimization of local motion continuity and global pose consistency.

[0061] Constructing a pose graph , where nodes Corresponding keyframe set, edge set It includes the odometer edge and the loop edge. This represents the odometry edge (pose constraint between adjacent keyframes). The covariance matrix of the odometry edge represents the loop closure edge (the non-adjacent keyframe constraint obtained from loop closure detection).

[0062] The covariance of the loop edge is （ (The coefficients are between 0 and 1). Then, pose graph optimization is performed to minimize the weighted pose error function:

[0063] In the formula, These are adjacent keyframe index pairs for the odometer edge. These are non-adjacent keyframe index pairs for loop edges. Indicates the index of the keyframe (representing the first keyframe). (The keyframe number). , and Representing keyframes and The current pose, Keyframe arrive The measurement pose constraints are determined; the Gauss-Newton method is used for global optimization to obtain the optimized keyframe pose; the optimized pose matrix is ​​decomposed into rotation matrix and translation vector, converted into quaternion form and synchronously updated to the system state parameters to complete the loop closure and drift correction.

[0064] Subsequently, with the globally consistent full-frame pose fixed and optimized, and based on the SGS-SLAM 3D Gaussian mapping framework, Gaussian density reconstruction and 3D Gaussian kernel parameter optimization were carried out again to complete the full a posteriori reconstruction and fine-grained update of the 3D Gaussian map.

[0065] For the initial frame, the camera pose is initialized to an identity matrix and used as the reference coordinate system for subsequent tracking and mapping. When estimating the camera pose for the RGB-D frame at the new timestamp, the displacement of the previous frame is accumulated based on the constant velocity assumption to calculate the initial pose, following the relational formula. Factor graphs are commonly used optimization problem-solving tools in the field of simultaneous localization and mapping (SLAM), and are graph structures containing variable nodes and factor nodes. This invention employs the Adam adaptive optimization algorithm based on gradient descent to optimize the factor graph. This algorithm effectively minimizes the tracking loss caused by depth, color, and semantic errors (i.e., the total constraint error in the factor graph), while filtering out dynamic interference through static masks, thereby improving the accuracy of camera pose estimation and providing a more reliable foundation for scene reconstruction. The factor graph structure used in this method is as follows: Figure 3 As shown, this factor graph uses camera pose as the core state node and multimodal tracking loss as the constraint factor. Its structure clearly reflects the correlation logic between pose optimization and multi-source error constraints.

[0066] First, based on the semantic segmentation results, pixels corresponding to preset dynamic categories (such as pedestrians, vehicles, etc.) are labeled as dynamic pixels, generating pixel-level dynamic masks. The dynamic mask is then fused with the depth-valid mask and the NaN filtering result. If the contour visibility mask is enabled, it is also included in the fusion process to obtain the final valid mask. In the pixel-level loss function used for camera pose optimization, depth, color, and semantic errors are only considered when the effective mask satisfies... Accumulation is performed within a pixel range to ensure that the pixels of moving objects do not generate gradient information that misleads pose estimation.

[0067] Iterative optimization of the current camera pose is achieved by minimizing the tracking loss. This loss function measures the deviation between the ground truth values ​​of depth, color, and semantic map and their differentiable rendered values, where the ground truth value of depth is... The true value of color The true value of the semantic map The loss function expression is as follows:

[0068] in This represents the pixel color rendered from the Gaussian map at the current pose. This represents the depth rendered at the current pose. This represents the semantic result obtained by rendering in the current pose. Represents the outline value of a pixel. , These represent the depth loss weight, color loss weight, and semantic loss weight, respectively. In this weighted summation mechanism, the rationality of multimodal fusion is built upon an underlying design of unified rendering space and probabilistic optimization: the system first explicitly transforms abstract semantic information into a "semantic color" vector isomorphic to appearance by assigning an independent channel to each 3D Gaussian volume. This allows color and semantic errors to be calculated using the same metric on the 2D image plane, achieving dimensionlessness and physical space unification at the feature level. Furthermore, the weight coefficients of the above-mentioned items... , This doesn't merely indicate the importance of constraints, but rather, as scale factors, within a probabilistic optimization framework, it's equivalent to compensating for the inverse of the variance of the observation noise of each sensor signal. This offsets the remaining numerical magnitude differences between depth, color, and semantics, allowing multimodal observation errors to be legitimately mapped to a unified "error energy" space for same-scale accumulation and joint optimization. Only rendered pixels with sufficiently large contour values ​​and static attributes are included in the loss calculation. Threshold The settings are designed to utilize a previously optimized map that has a high probability of visibility from the current camera's perspective.

[0069] After the tracking module, it enters the loop closure optimization module above to perform loop closure optimization, map densification, and then multi-channel constraint joint optimization of 3D Gaussian meta-parameters to construct a dense semantic 3D Gaussian map.

[0070] After mapping, a dynamic point mask is constructed based on semantic labels to mark Gaussian points belonging to the dynamic category. A preset set of dynamic categories is set as follows. (e.g., pedestrians, vehicles, etc.), based on this, determine the Gaussian point. The dynamic attribute is determined by the following formula:

[0071] in, Indicates the first The semantic label corresponding to each Gaussian point indicates which object category this Gaussian point is identified as; This indicates an indicator function that returns 1 if the condition within the parentheses is true, and 0 if the condition is false. For semantic dynamic masking, when When, it indicates the first Each Gaussian point belongs to the dynamic category; To simultaneously remove dynamic objects and geometrically invalid Gaussian points, this invention constructs a joint pruning rule that combines semantic dynamics and geometric validity, specifically including the following three types of judgment conditions: 1) Semantic dynamic conditions: As shown in formula (12), through semantic dynamic masking Mark dynamic Gaussian points; 2) Transparency validity condition: If the transparency of the Gaussian point... If the value is too low, its contribution to the rendering result is negligible. Set the transparency threshold. The determination formula is:

[0072] in, Indicates the first The opacity / alpha parameter at Gaussian points. For transparency mask; 3) Scale validity condition: If the scale of the Gaussian point... Too large a scale may introduce spatial redundancy. Set a scale threshold. The judgment is as follows:

[0073] in, , and These are three preset scale thresholds, which respectively limit... , , The maximum permissible dimension of the shaft. For scale mask, symbols Represents a logical OR operation; By combining the above three types of judgment conditions, a total pruning mask is defined. Its expression is:

[0074] when At that time, the first A Gaussian point was marked as a point to be pruned.

[0075] To avoid premature pruning affecting model convergence, this invention only triggers pruning periodically within a preset iteration interval. Let the total number of iterations be... The initial iteration step for pruning is The final iteration step is The pruning cycle is The trigger condition for the pruning operation is:

[0076] When the above triggering conditions are met, perform the pruning operation: retain the total pruning mask. The Gaussian points are then updated, and the Gaussian point parameter set is updated as follows:

[0077] in Indicates the first One Gaussian point; At the same time, the parameters of the optimizer are adjusted to match the number of Gaussian points after pruning, so as to ensure the continuity and stability of the model training process.

[0078] Example 2: An electronic device, comprising a memory and a processor; The memory is used to store computer programs; The processor is configured to implement the method described in Embodiment 1 when executing the computer program.

[0079] Example 3: A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in Embodiment 1.

[0080] Example 4: A computer program product includes a computer program that, when executed by a processor, implements the method described in Example 1.

[0081] In the above embodiments, the reference to "this embodiment" in the specification indicates that a specific feature, structure, or characteristic described in connection with the embodiment is included in at least some embodiments, but not necessarily all embodiments. Multiple appearances of "this embodiment" do not necessarily refer to the same embodiment.

[0082] In the above embodiments, although the invention has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory structures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed. The embodiments of the invention are intended to cover all such substitutions, modifications, and variations falling within the broad scope of the appended claims.

[0083] As will be understood by those skilled in the art, the computer-readable storage medium described in this embodiment allows for the implementation of all or part of the steps in the above method embodiments by computer program-related hardware. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0084] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication between them. The memory is used to store computer programs, the communication interface is used to perform communication, and the processor and the transceiver are used to run the computer programs, so that the electronic terminal performs the steps of the above method.

[0085] In this embodiment, the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0086] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0087] This invention can be used in a wide range of general-purpose or special-purpose computing system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0088] This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A semantic Gaussian-spraying dynamic RGB-D SLAM method with loop closure optimization, characterized in that: Includes the following steps: S1: Use a semantic segmentation model to initially extract the masks of dynamic objects and motion regions; based on semantic segmentation and contour masks, remove dynamic and uncertain regions, select high-confidence static pixels, and jointly optimize depth loss, color loss and semantic loss to achieve robust iterative estimation of the current camera pose; S2: Use the DBoW2 bag-of-words retrieval method to retrieve loop closure candidate frames and suppress false matches; S3: Geometric fine registration based on the Iterative Closest Point (ICP) algorithm to generate robust loop closure detection constraints; S4: Global calibration of full-frame pose is completed by optimizing PGO based on pose graph; S5: Perform full posterior reconstruction of 3D Gaussian primitives with fixed optimal pose to construct a dense semantic 3D Gaussian map. Specifically, this includes: using optimized pose as a rigid constraint, performing full-pixel Gaussian initialization based on RGB-D semantic prior, triggering frame-by-frame dynamic densification through differentiable contour rendering and depth inconsistency masking, and completing the full posterior reconstruction of the map. S6: Construct a dynamic dot mask based on semantic tags, prune dynamic and geometrically invalid Gaussian sputtering elements, retain only valid static Gaussian sputtering elements, and finally construct a high-precision static scene map.

2. The semantic Gaussian sputtering dynamic RGB-D SLAM method with loop closure optimization according to claim 1, characterized in that: In step S1, for the initial frame, the camera pose is initialized as an identity matrix and used as the reference coordinate system for subsequent tracking and mapping processes. When estimating the camera pose for the RGB-D frame at the new timestamp, the displacement of the previous frame is accumulated based on the constant velocity assumption to calculate the initial pose, following the relational formula. , Indicates the first Initial estimation of camera pose for a frame. Indicates the first The camera pose of the frame. Indicates the first Initial estimation of camera pose for each frame; Based on semantic segmentation and contour masking to remove dynamic and uncertain regions, high-confidence static pixels are selected. The Adam adaptive optimization algorithm based on gradient descent is used to optimize the factor graph, minimizing the tracking loss caused by depth, color and semantic errors. The depth loss, color loss and semantic loss are jointly optimized to achieve robust iterative estimation of the current camera pose. The factor graph takes the camera pose as the core state node and the multimodal tracking loss as the constraint factor.

3. The semantic Gaussian sputtering dynamic RGB-D SLAM method with loop closure optimization according to claim 2, characterized in that: Step S1 specifically includes the following steps: First, based on the semantic segmentation results, pixels corresponding to preset dynamic categories are labeled as dynamic pixels, generating pixel-level dynamic masks. The dynamic mask is then fused with the depth-valid mask and the result of the NaN filtering. If the contour visibility mask is enabled, it is also included in the fusion process to obtain the final effective mask. In the pixel-level loss function used for camera pose optimization, depth, color, and semantic errors are only considered when the effective mask satisfies... Accumulation is performed within a pixel range to ensure that the pixels of moving objects do not generate gradient information that misleads pose estimation; Iterative optimization of the current camera pose is achieved by minimizing the tracking loss. The tracking loss function measures the deviation between the ground truth values ​​of depth, color, and semantic map and their differentiable rendered values, where the ground truth value of depth is... The true value of color The true value of the semantic map The loss function expression is as follows: in This represents the pixel color rendered from the Gaussian map at the current pose. This represents the depth rendered at the current pose. This represents the semantic result obtained by rendering in the current pose. Represents the outline value of a pixel. , These represent the depth loss weight, color loss weight, and semantic loss weight, respectively. In this weighted summation mechanism, the rationality of multimodal fusion is based on a unified rendering space and probabilistic optimization underlying design: firstly, by assigning an independent channel to each 3D Gaussian volume, abstract semantic information is explicitly transformed into a semantic color vector isomorphic to the appearance, enabling color and semantic errors to be calculated using the same metric on the 2D image plane, achieving dimensionlessness and physical space unification at the feature level; the weight coefficients for each... , It also serves as a scale conversion factor, which, within the probabilistic optimization framework, is equivalent to compensating for the inverse of the variance of the observation noise of each sensor signal. This offsets the residual numerical magnitude differences between depth, color, and semantics, allowing multimodal observation errors to be fully and legitimately mapped to a unified error energy space for same-scale accumulation and joint optimization; contour threshold The settings are designed to utilize a map that has been optimized in advance.

4. The semantic Gaussian sputtering dynamic RGB-D SLAM method with loop closure optimization according to claim 1, characterized in that: Step S2 specifically includes the following steps: Insert a keyframe when frame With initial frame, fixed time interval or the second to last frame in the sequence. When they meet, insert them into the keyframe set, where For preset intervals, Total number of frames; each keyframe records the camera-world-human transformation. The corresponding RGB / grayscale image and optional semantic mask; where Indicates the first The camera pose corresponding to a keyframe represents a rigid body transformation from the world coordinate system to the camera coordinate system. For keyframe grayscale images First, key points are detected and a set of binary descriptors are extracted. in Indicates the first The first frame One key point, Indicates the first Frames One key point, ; using offline training to obtain A hierarchical dictionary of visual words, with each descriptor Quantize to the nearest leaf node and count the frequency of each word. Then, a bag-of-words (BoW) vector is constructed, and a weighted term frequency-inverse document frequency (WF-IF) is calculated to generate a sparse vector: In the formula, It represents the set of visual words, containing all possible visual words; For visual words In the keyframe set Document frequency in It is a dummy variable used to iterate over the collection. All words, avoid and current words Confusing.

5. The semantic Gaussian sputtering dynamic RGB-D SLAM method with loop closure optimization according to claim 1, characterized in that: Step S3 specifically includes the following steps: Calculate the current number Frame keyframe bag-of-words vector With all historical keyframes bag-of-words vectors Cosine similarity: like And inter-frame interval Then the frame pair Add to the candidate set of loops ,in The threshold for cosine similarity is... The keyframe time interval is used as the filtering threshold; for candidate loop closure pairs Generate source point cloud With target point cloud Source cloud Corresponding keyframe 3D point cloud, For the first point cloud Points, target point cloud Corresponding keyframe 3D point cloud, For the first point cloud 1 point, and calculate the initial relative pose: in Keyframe The world-to-camera transformation matrix, Keyframe The world-to-camera transformation matrix describes the camera's pose in the world coordinate system. Provides initial values ​​for minimizing subsequent errors; Iteratively update the transformation matrix Align the source point cloud with the target point cloud after transformation, where For rotation matrix, The translation vector is used; in each iteration, the source point cloud is minimized. With target point cloud The transformation matrix is ​​updated based on the point-to-point error between points. The optimization objective is to minimize the sum of squared errors of all point pairs between point clouds. The objective function is: In the formula, The source point after transformation , As the source The corresponding target point Let be the transformation matrix to be optimized; Iteratively update the rotation matrix Translation vector To minimize the error, the source point cloud is first calculated to solve for the updated transformation matrix. With target point cloud center of mass and ,Right now: The optimal rotation matrix is ​​then calculated using the centroid. For symmetric matrices Perform singular value decomposition (SVD) to obtain the rotation matrix: right SVD decomposition Then the rotation matrix is: Then, the translation vector is calculated from the centroid. : The updated transformation matrix Used for source point cloud transformation, and continues to the next round of iteration optimization; After each iteration, the error is calculated to determine whether the termination condition is met. An error threshold is set, and the iteration is terminated when the current error is less than the threshold or the change in error is less than the preset tolerance. If the condition is not met, the iteration continues until the error converges. Finally, when the optimized transformation matrix When the error between the source point cloud and the target point cloud is lower than a set threshold and the matching quality meets the standard, the loop is deemed valid and incorporated into the global optimization process.

6. The semantic Gaussian sputtering dynamic RGB-D SLAM method with loop closure optimization according to claim 1, characterized in that: Step S4 Specifically, the following steps are included: Constructing a pose graph , where nodes Corresponding keyframe set, edge set Includes the odometer edge and the loopback edge, where Indicates the odometer side, The covariance matrix of the odometer edge, representing the loop edge, is: The covariance of the loop edge is , The coefficients are between 0 and 1; then pose graph optimization is performed to minimize the weighted pose error function: In the formula, These are adjacent keyframe index pairs for the odometer edge. These are non-adjacent keyframe index pairs for loop edges. The index of the keyframe, that is, the index of the first keyframe. The keyframe number; , and Representing keyframes and The current pose, Keyframe arrive The measurement pose constraints are determined; the Gauss-Newton method is used for global optimization to obtain the optimized keyframe pose; the optimized pose matrix is ​​decomposed into rotation matrix and translation vector, converted into quaternion form and synchronously updated to the system state parameters to complete the loop closure and drift correction.

7. The semantic Gaussian sputtering dynamic RGB-D SLAM method with loop closure optimization according to claim 1, characterized in that: In step S6, after the map is built, a dynamic point mask is constructed based on semantic labels to mark Gaussian points belonging to the dynamic category; Set the preset dynamic category set as ,in for A preset dynamic category is used to determine the Gaussian point. The dynamic attribute is determined by the following formula: in, Indicates the first The semantic label corresponding to each Gaussian point indicates which object category this Gaussian point is identified as; This indicates an indicator function that returns 1 if the condition within the parentheses is true, and 0 if the condition is false. For semantic dynamic masking, when When, it indicates the first Each Gaussian point belongs to the dynamic category; The joint pruning rules are constructed, including the following three types of decision conditions: Semantic dynamic conditions: through semantic dynamic masks Mark dynamic Gaussian points; Transparency validity conditions: Set a transparency threshold The determination formula is: in, Indicates the first A Gaussian point transparency parameter, For transparency mask; Scale validity condition: If a scale threshold is set The judgment is as follows: in, , and These are three preset scale thresholds, which respectively limit... , , Maximum permissible dimensions of the shaft; For scale mask, symbols Represents a logical OR operation; By integrating three types of decision conditions, a total pruning mask is defined. Its expression is: when At that time, the first A Gaussian point was marked as a point to be pruned; Pruning is triggered periodically only within a preset iteration interval, with a total number of iterations of 1. The initial iteration step for pruning is The final iteration step is The pruning cycle is The trigger condition for the pruning operation is: When the triggering condition is met, perform a pruning operation: retain the total pruning mask. The Gaussian points are then updated, and the Gaussian point parameter set is updated as follows: in Indicates the first One Gaussian point; At the same time, the parameters of the optimizer are adjusted to match the number of Gaussian points after pruning, so as to ensure the continuity and stability of the model training process.

8. A semantic Gaussian sputtering dynamic RGB-D SLAM system with loop closure optimization, characterized in that: It includes a semantic mask tracking module, a loop closure optimization module, and a graph construction module; The semantic mask tracking module is used to initially extract the masks of dynamic objects and motion regions using a semantic segmentation model; based on semantic segmentation and contour masking, dynamic and uncertain regions are eliminated, high-confidence static pixels are selected, and depth loss, color loss and semantic loss are jointly optimized to achieve robust iterative estimation of the current camera pose. The loop closure optimization module is used to efficiently retrieve loop closure candidate frames and suppress false matches using the DPoW2 bag-of-words retrieval method. Geometric fine registration is performed based on the iterative nearest point algorithm to generate robust loop closure detection constraints; global calibration of the full-frame pose is completed based on pose graph optimization. The mapping module initializes, densifies, and performs multi-channel joint optimization of 3D Gaussian sputtering elements based on the optimal pose, and reconstructs a dense semantic 3D Gaussian map from scratch. It also constructs a dynamic point mask based on semantic labels, performs joint pruning on dynamic and geometrically invalid Gaussian sputtering elements, retains only valid static Gaussian sputtering elements, and generates a static map.

Images