A visual positioning and Gaussian splashing scene reconstruction algorithm in a dynamic scene, a storage medium and a system

Abstract

This invention belongs to the field of simultaneous localization and mapping (SLAM) technology, specifically disclosing a visual localization and Gaussian splash scene reconstruction algorithm, storage medium, and system for dynamic scenes. The algorithm includes: Step S1, acquiring monocular RGB image sequences, predicting and measuring depth for each keyframe, and generating a coarse dynamic prior; Step S2, refining the coarse dynamic prior into a soft dynamic mask based on a mask refinement network, and then mapping the soft dynamic mask to pixel-wise static weights; Step S3, injecting the static weights into dense bundle adjustment, multiplying them pixel-wise with the pixel-wise matching confidence predicted by the network, and minimizing the reprojection error to estimate the keyframe pose; Step S4, representing the scene with a 3D Gaussian splash during the mapping stage, generating rendering residuals by fusing rendering color, depth, and structural residuals, and then performing convergent perceptual gating and morphological processing to reconstruct a static Gaussian map. This invention effectively solves the pose drift problem in dynamic scenes while achieving high-quality static map reconstruction.

Description

Technical Field

[0001] This invention belongs to the field of Simultaneous Location And Mapping (SLAM) technology, specifically relating to a visual positioning and Gaussian splash scene reconstruction algorithm, storage medium and system in dynamic scenes. Background Technology

[0002] Simultaneous Localization and Mapping (SLAM) technology aims to enable robots to achieve autonomous navigation and dense 3D reconstruction in unknown environments, and is one of the core technologies for applications such as autonomous navigation, autonomous driving, and augmented reality. When a camera is used as a sensor, it is called visual SLAM. Due to the limitations of laser sensors in terms of information acquisition and size cost, visual SLAM has gradually become a research hotspot due to its advantages of low cost, small size, and rich environmental information. In recent years, 3DGS scene representation methods have opened up new avenues for dense reconstruction in visual SLAM. Existing Gaussian splash SLAM systems implicitly assume that the scene is a static world. Dynamic objects, which are difficult to completely avoid in real-world scenes, will disrupt the photometric and geometric consistency on which SLAM depends at both the tracking and mapping levels. This can lead to pose drift or even tracking failure, and dynamic objects can embed themselves in the map as artifacts or trails, destroying the spatiotemporal consistency of the map and its downstream reuse value.

[0003] To address the aforementioned issues, existing dynamic SLAM methods can be broadly categorized into several technical approaches, but all have limitations. Semantic detection-based methods remove predefined dynamic categories through segmentation or detection, i.e., applying explicit dynamic cue detection. However, these techniques are limited by the closed-set assumption, cannot handle unknown moving objects outside the vocabulary, and rely heavily on segmentation accuracy. Geometric optical flow-based methods identify dynamic regions using depth residuals or optical flow inconsistencies, resulting in coarse masks, and most rely on RGB-D input. Methods based on online uncertainty avoid explicit dynamic priors and have strong open-set generalization capabilities, thus adapting to unknown moving objects outside the training set. However, pure geometric uncertainty fields converge slowly, and object boundaries are blurred. Existing methods often employ hard masks for binary removal (either moving or static), which may mistakenly remove reliable static pixels at object edges and in slowly moving regions. This weakens the geometric constraints of bundle adjustment and reduces effective observations for mapping, making it difficult to meet the pixel-by-pixel soft suppression and consistency requirements between tracking and mapping required for monocular Gaussian splashing. Summary of the Invention

[0004] The purpose of this invention is to provide a visual localization and Gaussian splash scene reconstruction algorithm for dynamic scenes, which solves the technical problems of existing technologies that use hard masks for non-dynamic (either static or dynamic) binary culling, which may mistakenly remove reliable static pixels at object edges and in slowly moving areas, making it difficult to meet the pixel-by-pixel soft suppression required for monocular Gaussian splashing and the consistency requirements between tracking and mapping.

[0005] The aforementioned visual localization and Gaussian splash scene reconstruction algorithm for dynamic scenes includes the following steps: Step S1: Acquire monocular RGB image sequences, select keyframes and estimate poses, predict and measure depth for each keyframe, and generate coarse dynamic priors; Step S2: Based on the mask refinement network, the coarse dynamic prior is refined into a soft dynamic mask, and then the soft dynamic mask is mapped to a pixel-wise static weight. The mask refinement network uses the rendering residual as a self-supervised signal. Step S3: During the tracking phase, the static weights are injected into the dense bundle adjustment and multiplied pixel by pixel with the pixel-by-pixel matching confidence predicted by the network to minimize the reprojection error and estimate the key frame pose. Step S4: In the mapping stage, the scene is represented by a 3D Gaussian splash. The rendering residuals are generated by blending the rendering color, depth and structural residuals and then undergoing convergence perception gating and morphological processing to reconstruct the static Gaussian map.

[0006] Preferably, in step S1, for each keyframe, the corresponding metric depth is predicted from the RGB image by the monocular metric depth estimation model and normalized. This is then combined with a common input dynamic prior extraction module to obtain an initial coarse dynamic prior. An S-shaped function transformation with threshold and temperature is applied to the initial coarse dynamic prior to obtain a normalized coarse dynamic prior. The formula is: , in, , The corresponding threshold parameters and temperature parameters are listed in order. It is an S-shaped function.

[0007] Preferably, in step S2, the mask thinning network is a lightweight U-shaped network. Its input consists of a coarse dynamic prior, an RGB image, and a concatenation of a normalized depth metric along the channel dimension. The encoder uses average pooling for downsampling, and the decoder uses bilinear interpolation for upsampling. Multi-scale features are fused through skip connections, and the output after encoding and decoding is a soft dynamic mask. The refined soft dynamic mask is converted into pixel-by-pixel static weights. .

[0008] Preferably, in step S2, the online training signal of the mask refinement network consists of rendering consistency loss, prior consistency loss and sparsity loss. The soft target is obtained by remapping the fused rendering residual of the current keyframe using a sigmoid function. The rendering consistency loss is then obtained by calculating the binary cross-entropy between the soft dynamic mask and the soft target using residual gating. The expression is as follows:

[0009] in, For pixel domain, For pixels, It is a residual gate, and BCE is a binary cross-entropy; The corresponding loss for the positive sample term is obtained by applying a weighted binary cross-entropy to the positive and negative sample seeds. and negative sample term loss The expression is as follows:

[0010]

[0011] Among them, weight , Gating is performed based on whether the residuals are consistent with the coarse dynamic prior; positive sample terms are gated at the strong seed of the coarse dynamic prior. Pulling towards 1, the negative sample term will be at a point where the coarse dynamic prior is empty and the residuals are also inactive. Pull towards 0; The ultimate training goal of mask refinement networks The formula is: , in, Loss for positive sample terms Negative sample term loss Rendering consistency loss sparsity loss The weight.

[0012] Preferably, in step S2, when the coarse dynamic prior cannot be stably trained by relying solely on prior consistency loss across the entire frame, the supervision path is switched to be dominated by rendering residuals, allowing soft dynamic masks only at residual confidence points. The background error is increased and limited by sparsity loss; at the same time, a dual-evidence veto is introduced: the fused rendering residual is used as the first evidence, and the structural similarity residual calculated separately based on structural similarity is used as the second evidence. The first evidence and the second evidence are binarized into regions R and S, respectively, and the overlap rate of these two regions is calculated: When the overlap rate Below the set threshold, or in the area number of pixels When there is insufficient residual evidence, the frame is determined to contain no reliable dynamics and a soft dynamics mask is applied. Set the entire frame to zero; otherwise, it indicates that the evidence from the fusion rendering residual and the structural similarity residual is consistent and strong enough to allow dynamic discrimination driven by the residual.

[0013] Preferably, in step S3, the keyframes are... static weights The static weights of dense bundle adjustment are obtained by downsampling to the low-resolution grid used in dense bundle adjustment. Then match the pixel-by-pixel confidence score predicted by the network. Pixel-by-pixel multiplication yields the dynamic sensing weights. The expression is: ; Use dynamic sensing weights to assign weights to each edge in the keyframe factor graph Weighted reprojection errors are used to construct a dynamically perceived dense bundle adjustment optimization objective. The corresponding formula is:

[0014] in, To predict the target pixel position, and For projection and back projection functions, , For the first i The and the first j Keyframe pose Inverse depth; dense bundle adjustment optimization objective Minimize the solution of keyframe pose.

[0015] Preferably, in step S4, a three-dimensional Gaussian splash is used as a representation of a dense scene, and the rendering color, rendering depth, and cumulative opacity obtained by the three-dimensional Gaussian splash rendering are used. Based on these rendering results, map iterative optimization is performed within a sliding keyframe window, and the color residuals are obtained by comparing the rendering results with observations. Depth residual With structural residuals And the three are merged into the initial residual. The formula is: ,in, Color residuals Depth residual Structural residuals The corresponding weights are then applied; a convergence-aware gating based on cumulative opacity is then applied, retaining the fused rendering residual only on pixels where the map has converged. The expression is: ,in Opacity threshold For characteristic functions, operators Median filtering, morphological opening, and small connected component removal are performed sequentially, and frame-level rejection is triggered when the median of the frame-level residual is abnormally high.

[0016] Preferably, in step S4, dynamic content is suppressed using static weights in four stages: backprojection initialization, rendering loss weighting, densification scoring, and dynamic Gaussian culling, including: In the backprojection initialization phase, dynamic filtering is performed before generating the new Gaussian mask from the keyframe depth backprojection. Pixels whose soft dynamic mask depth exceeds the mask threshold are set to zero. The filtering algorithm is as follows: ,in Represents the corresponding pixel , In order, they are the keyframe depth after dynamic filtering, the normalized metric depth, and the mask threshold; In the rendering loss weighting process, pixels still involved in map building are given static weights. Pixel-by-pixel weighted construction of photometric loss With depth loss The details are as follows: , and isotropic regularization terms The loss function for jointly constructing the overall mapping objective: , in, , , The weights of the structural residual, photometric loss, and isotropic regularization term are calculated in sequence. In the densification process, each Gaussian center is projected onto the current viewpoint and sampled. Its accumulated gradient in screen space is weighted and scored using static weights, as shown in the following formula: , in, The result is a compacted score, i.e., a Gaussian score. Let Gaussian center be the projected pixel value at the current viewpoint. The cumulative number of times within a densification cycle. The number of iterations for densification. For corresponding projected pixels Spatial gradient; In the dynamic Gaussian culling process, an active culling is performed after each keyframe optimization. The center of the Gaussian is projected onto the current viewpoint for sampling, and Gaussians that fall into the dynamic region and are close to the surface of the observed foreground are directly deleted.

[0017] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a visual positioning and Gaussian splash scene reconstruction algorithm in a dynamic scene as described above.

[0018] The present invention also provides a computer system, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, it implements the steps of a visual positioning and Gaussian splash scene reconstruction algorithm in a dynamic scene as described above.

[0019] The present invention has the following advantages: 1. This invention employs a residual-driven self-supervised scheme, using prior consistency, rendering consistency, and sparsity losses to train an online mask refinement network. The online mask refinement network is then extended into a soft dynamic mask through rendering residual self-supervision, and this soft dynamic mask is uniformly mapped to static weights shared by tracking and mapping. When the coarse dynamic prior is silenced, the rendering residual extends dynamic recognition to cover unknown moving objects, and double-evidence rejection avoids false detections in purely static scenes. Compared to the either-or culling of hard masks, this invention, based on continuous soft static weights, allows reliable pixels at object edges and in slowly moving regions to participate in pose optimization and Gaussian fitting in a decaying manner, while simultaneously improving tracking stability and rendering quality. This satisfies the pixel-by-pixel soft suppression required for monocular Gaussian splashing and the consistency requirements between tracking and mapping.

[0020] 2. This invention generates coarse dynamic priors based on monocular RGB input. The coarse dynamic priors provide explicit and accurate initial dynamic cues, which are then extended into soft dynamic masks by rendering residual self-supervision. The rendering residuals are used to extend dynamic recognition to cover unknown moving objects, thereby achieving a balance between explicit dynamic cues, online adaptation to unknown dynamic objects, and monocular availability, and realizing a joint robust solution for localization and reconstruction in dynamic environments.

[0021] 3. This invention adopts an improved Gaussian mapping strategy for dynamic scenes, introducing soft dynamic masks into four stages: backprojection initialization, rendering loss weighting, densification scoring, and dynamic Gaussian culling. This allows the limited Gaussian budget to prioritize the static background, avoiding the seeding of Gaussian on moving objects from the source and reducing the solidification of dynamic artifacts in the densification stage. It mainly improves the integrity and perception quality of static area reconstruction, resulting in a cleaner static map with fewer artifacts. Attached Figure Description

[0022] Figure 1 This is a basic flowchart of a visual positioning and Gaussian splash scene reconstruction algorithm in a dynamic scene according to the present invention.

[0023] Figure 2 This is an overall framework diagram of the SLAM system for applying the present invention.

[0024] Figure 3 This is a schematic diagram of the mask thinning network in this invention.

[0025] Figure 4This is a schematic diagram of the coarse dynamic prior and soft dynamic mask refinement results in this invention.

[0026] Figure 5 This is a schematic diagram comparing the results before and after the improvement of the Gaussian graphing strategy for dynamic scenes in this invention.

[0027] Figure 6 This is a comparison chart of the dynamic mask estimation performance of the present invention and existing methods.

[0028] Figure 7 This is a comparison chart of ATE data between the present invention and existing algorithms on dynamic sequences.

[0029] Figure 8 This is a comparison diagram of the trajectories of the present invention and existing algorithms on dynamic sequences.

[0030] Figure 9 This is a comparison chart of the static area rendering effects of the present invention and existing algorithms.

[0031] Figure 10 This is a comparison chart of PSNR data between the present invention and existing algorithms in the static region.

[0032] Figure 11 This is a data comparison chart of the present invention and existing algorithms in the static region SSIM.

[0033] Figure 12 This is a data comparison chart of the present invention and existing algorithms in the static region LPIPS. Detailed Implementation

[0034] The following detailed description of the embodiments, with reference to the accompanying drawings, will further illustrate the specific implementation of the present invention, in order to help those skilled in the art to have a more complete, accurate, and in-depth understanding of the inventive concept and technical solution of the present invention.

[0035] Example 1.

[0036] like Figures 1-4 As shown, the present invention provides a visual localization and Gaussian splash scene reconstruction algorithm in dynamic scenes, including the following steps.

[0037] Step S1: Acquire monocular RGB image sequences, select keyframes and estimate poses, predict and measure depth for each keyframe, and generate coarse dynamic priors.

[0038] In this step, the system uses only image sequences acquired by a monocular RGB camera as input, without relying on any depth sensor.

[0039] The front end adopts a dense bundle adjustment structure to maintain a keyframe factor graph with keyframes as nodes and dense matching constraints between keyframes as edges, thus completing keyframe selection and initial pose estimation.

[0040] For each keyframe, a monocular metric depth estimation model predicts and normalizes the metric depth corresponding to each pixel from the RGB image. Then, the RGB image and the metric depth are input together into the dynamic prior extraction module to obtain the initial coarse dynamic prior. The RGB image provides color information, and the initial coarse dynamic prior takes values ​​between 0 and 1 and is marked in the potential dynamic region. This coarse dynamic prior is used to provide a spatial seed for mask refinement.

[0041] The dynamic prior extraction module uses pre-trained priors with frozen parameters, which have weak responses and blurred boundaries to out-of-distribution objects, and therefore are not directly used as the final soft dynamic mask. To construct prior consistency constraints, an S-shaped function transformation with threshold and temperature is applied to the initial coarse dynamic prior to obtain a normalized coarse dynamic prior. The formula is: , in, , The corresponding threshold parameters and temperature parameters are listed in order. It is a sigmoid function with coarse dynamic priors. Used to guide network output on reliable positive and negative sample seeds.

[0042] Step S2: Based on the mask refinement network, the coarse dynamic prior is refined into a soft dynamic mask, and then the soft dynamic mask is mapped to pixel-wise static weights. The mask refinement network uses the rendering residual as a self-supervised signal.

[0043] This step employs a coarse-to-fine dynamic masking scheme: the coarse dynamic prior provides the initial spatial seed, and the masking refinement network receives the coarse dynamic prior, the RGB image, and the normalized depth metric during the mapping loop. It uses the rendering residual as a self-supervised signal to extend the mask along the boundaries of real objects to the areas previously missed by the coarse dynamic prior, thereby identifying the real dynamic objects in the scene.

[0044] Specifically, the masking refinement network is a lightweight U-shaped network. Its input is a concatenation of channel dimensions. , , The sequence is as follows: coarse dynamic prior, RGB image, and normalized metric depth; the masking refinement network outputs a soft dynamic mask after encoding and decoding. , The encoder uses average pooling downsampling, the decoder uses bilinear interpolation upsampling, and multi-scale features are fused through skip connections.

[0045] The online training signal for the mask refinement network consists of rendering consistency loss, prior consistency loss, and sparsity loss. Specifically: Rendering consistency loss: Self-supervised signal from the fused rendering residual of the current keyframe First, remap it to a soft target. The expression is , , The corresponding threshold parameters and temperature parameters are listed in order. It is a sigmoid function; hence, the rendering consistency loss is defined. The expression is as follows:

[0046] in, For pixel domain, For pixels, Indicates the corresponding pixel u. For residual gating, BCE is the binary cross-entropy; this term promotes Growth occurs in regions with high residuals, thereby covering moving objects for which coarse dynamic priors fail to respond.

[0047] Prior consistency loss: Applying weighted binary cross-entropy to the positive and negative sample seeds yields the corresponding loss for the positive sample term. and negative sample term loss .

[0048] Positive Sample Term Loss and negative sample term loss The formula is:

[0049]

[0050] Among them, weight , Gating is performed based on whether the residuals are consistent with the coarse dynamic prior; positive sample terms are gated at the strong seed of the coarse dynamic prior. Pulling towards 1, the negative sample term will be at a point where the coarse dynamic prior is empty and the residuals are also inactive. Pulling towards 0. Prior consistency loss avoids both labeling a stable, static background as dynamic and suppressing true motion in the prior blind zone.

[0051] Sparsity loss: Sparsity loss Suppress spurious dynamic responses globally.

[0052] The ultimate training goal of mask refinement networks The formula is: , The losses for positive and negative samples are weighted independently. Loss for positive sample terms Negative sample term loss Rendering consistency loss sparsity loss The weights are then calculated. The refined soft dynamic mask is converted into pixel-by-pixel static weights. It is used to uniformly suppress dynamic pixels in tracking and mapping.

[0053] When the coarse dynamic prior has almost no response across the entire frame, and training cannot be stable by relying solely on prior consistency loss, the system switches to a supervision path dominated by rendering residuals, allowing only the rendering residuals to be supervised at residual confidence points. Increase, at the same time by Limit background error.

[0054] To prevent the misinterpretation of high residuals caused by insufficient reconstruction or complex textures in purely static scenes, a dual-evidence rejection mechanism is introduced: the fused rendering residual is used as the first evidence, and the structural similarity residual calculated separately based on structural similarity is used as the second evidence. The first and second evidences are binarized into regions R and S, respectively, and the overlap rate of these two regions is calculated. , When overlap rate Below the threshold, or When the residual evidence is too small and insufficient, the frame is determined to contain no credible dynamics and... The entire frame is zeroed out; dynamic discrimination driven by residuals is only allowed when the two types of evidence are consistent and strong enough. The coarse dynamic prior, the mask refinement network, and the prior silence fallback together constitute a dynamic mask generation and refinement chain from the known to the open set, from coarse localization to fine boundary. They share a unified static weight as output, which integrates dynamic pixel processing from mask generation to subsequent tracking and mapping.

[0055] Step S3: During the tracking phase, the static weights are injected into the dense bundle adjustment and multiplied pixel by pixel with the pixel-by-pixel matching confidence predicted by the network to minimize the reprojection error and estimate the keyframe pose.

[0056] In this step, the keyframe factor map is maintained through dense bundle adjustment. , It is a set of nodes, that is, a set of keyframes. Let be the set of edges, and for each edge... The network iteratively predicts the target pixel position by combining feature correlation with the current motion estimate. and the confidence level of the corresponding pixel matching It optimizes keyframe pose by minimizing reprojection error, assuming by default that all pixels come from a static scene.

[0057] This invention employs explicit static weight injection into dense bundle adjustment: keyframes static weights Downsampling to the low-resolution grid used for adjustment yields the static weights for dense bundle adjustment. The dynamic perception weights are then multiplied pixel by pixel with the pixel-by-pixel matching confidence predicted by the network. The expression is Dynamically perceptual weights are used to assign weights to each edge in the keyframe factor graph. Weighted reprojection errors are used to construct a dynamically perceived dense bundle adjustment optimization objective. The corresponding formula is:

[0058] in, and For projection and back projection functions, , For the first i The and the first j Keyframe pose This represents the inverse depth.

[0059] After static weight injection, pixels with higher soft dynamic mask values ​​retain non-zero weights after static weight mapping, but their impact on pose estimation is weakened. Static background pixels retain higher effective confidence and provide the main geometric constraints.

[0060] Step S4: In the mapping stage, the scene is represented by a 3D Gaussian splash. The rendering residuals are generated by blending the rendering color, depth and structural residuals and then undergoing convergence perception gating and morphological processing to reconstruct the static Gaussian map.

[0061] In this step, a 3D Gaussian splash is used as a representation of the dense scene, and the scene is modeled as a set of anisotropic 3D Gaussian splashes. To ensure positive semidefiniteness, , No. Gaussian From the center Opacity ,color With 3D covariance Parametric formation, three-dimensional covariance Decomposed into: ,in, These are the rotation matrix and the scaling matrix, respectively. Given the camera pose, the 3D covariance... Projecting onto the image plane yields the two-dimensional covariance. , ,in Let be the Jacobian matrix of the affine approximation of the projection transformation. From this, we obtain the ___ A projected Gaussian at pixel The density at that location is The formula is:

[0062] in, For the first i Gaussian three-dimensional center The two-dimensional center obtained by projecting onto the image plane For opacity, It is a natural exponential function.

[0063] Gaussian spherical vectors covering the same pixel are sorted by depth from nearest to farthest and then α-synthesized to obtain the rendered color, rendered depth, and cumulative opacity. The entire rendering process is differentiable, so the Gaussian parameters can be directly optimized by backpropagating the error between the rendered amount and the observation, which is the basis of incremental mapping. Whenever the pose of a new keyframe is tracked, the system backprojects a new Gaussian spherical vector from its RGB image and metric depth to expand the uncovered areas; when loop closure or global adjustment updates the pose and depth of historical keyframes, the affected Gaussian spherical vectors undergo rigid motion and scale rescaling to maintain map and trajectory consistency.

[0064] Map iterative optimization is performed within a sliding keyframe window. In each iteration, the rendering result of one keyframe is sampled and compared with the observation to obtain the color residual. Depth residual With structural residuals And the three are merged into the initial residual. The formula is: ,in, Color residuals Depth residual Structural residuals The corresponding weights; SSIM is for structural similarity calculation; the rendering result includes rendering color. Rendering depth With cumulative opacity The observations included RGB images. with normalized metric depth RGB images provide color information.

[0065] Since high residuals are not necessarily caused by dynamic objects, the system applies a convergence-aware gating based on cumulative opacity, retaining the fused rendering residuals only for pixels where the map has converged. The expression is: ,in Opacity threshold For characteristic functions, operators Median filtering, morphological opening, and small connected component removal are performed to suppress salt-and-pepper noise and distant speckle. A frame-level rejection is triggered when the median of the frame-level residual is abnormally high; that is, the entire image frame is rejected when the calculated median of the frame-level residual exceeds a set threshold. In static scenes, the above-mentioned fused rendering residual can be directly used as an optimization target and also serves as a self-supervised signal for step S2.

[0066] In dynamic scenes, allowing moving objects to participate in mapping will pollute the static map and leave ghosting. Therefore, this invention uses static weights in four stages—backprojection initialization, rendering loss weighting, densification scoring, and dynamic Gaussian culling (corresponding to the four stages of the dynamic Gaussian lifecycle: admission, weighting, densification, and culling)—to suppress dynamic content and reconstruct a static Gaussian map. The fused rendering residual then serves as the basis for dynamic suppression in this step.

[0067] Backprojection initialization: Before generating the new Gaussian image from the keyframe depth backprojection, dynamic filtering is performed. Pixels whose soft dynamic mask exceeds the mask threshold have their depth set to zero. The filtering algorithm is as follows: ,in Represents the corresponding pixel , In order, these are the dynamically filtered keyframe depth, the normalized metric depth, and the mask threshold. In this step, only... The pixel back projection generates a new Gaussian, thus avoiding the ghosting caused by seeding Gaussians on moving objects from the source.

[0068] Rendering loss weighting process: For pixels still involved in map building, render loss is weighted pixel-by-pixel with static weights to construct luminance loss and depth loss, as detailed below: , in, For luminance loss, For deep loss, The weights corresponding to the structural residuals are used to balance L1 and structural similarity. The overall mapping objective loss function is as follows: , in, The total loss during mapping. This is a weight for luminance loss, used to balance color and depth. To suppress the isotropic regularization term of the needle-shaped Gaussian, so that the occasionally seeded dynamic Gaussian contributes very little to the fitting, For the corresponding The weight.

[0069] In the densification scoring process: a long-axis splitting and budget constraint strategy is adopted instead of random splitting. Before densification, each Gaussian center is projected onto the current viewpoint for sampling, and its screen space cumulative gradient is weighted and scored using static weights. The formula is as follows: , in, The result is a compacted score, i.e., a Gaussian score. Let Gaussian center be the projected pixel value at the current viewpoint. The cumulative number of times within a densification cycle. The number of iterations for densification. For corresponding projected pixels Spatial gradient. Gaussian scores are lower on moving objects, they are less likely to be cloned or split, and are more easily pruned because their opacity is not enhanced, thus reducing floating objects and dynamic ghosting.

[0070] Dynamic Gaussian culling: After optimization of each keyframe, active culling is performed once, the center of Gaussian is projected onto the current view sampling, and Gaussian that falls into the dynamic region and is close to the surface of the observed foreground is directly deleted.

[0071] The following section will further verify and explain the execution flow and actual performance of the monocular Gaussian splash SLAM algorithm for dynamic environments by combining specific scenario experiments.

[0072] like Figure 5 As shown, this paper compares the reconstruction results before and after the improvement of the Gaussian mapping strategy for dynamic scenes. Before the improvement, dynamic objects left artifacts and incomplete details in the static map. After the improvement, a more complete static background with fewer artifacts was obtained, which verifies the effectiveness of the improved Gaussian mapping strategy in suppressing dynamic content.

[0073] like Figure 6 As shown, to verify the performance of dynamic mask estimation, representative frames were derived from the dynamic sequence, and the masks of existing methods based on semantic priors, the uncertainty maps of existing methods based on online uncertainty, and the soft dynamic mask of the present invention were compared. Methods based on semantic priors are prone to missing or fragmenting out-of-distribution or non-semantic targets, and misclassify occlusion boundaries caused by viewpoint changes as moving regions. Methods based on online uncertainty exhibit blurred boundaries at object contours and false high uncertainty in background areas with sparse static textures, brief occlusions, or insufficient reconstruction. In contrast, the soft dynamic mask of the present invention has no false responses on static frames, provides clear masks with well-fitted boundaries for real moving objects, and can still detect unknown moving targets through rendering inconsistencies across keyframes even when coarse dynamic priors are silent, demonstrating a category-independent generalization capability that relies on rendering consistency rather than single-frame semantics.

[0074] like Figure 7As shown, the absolute trajectory error (ATE) of the present invention and existing algorithms is quantitatively compared on dynamic sequences such as Bonn, TUM and MoCap; the present invention achieves lower ATE on most sequences, indicating that the proposed method has better positioning accuracy in dynamic environments.

[0075] like Figure 8 As shown, the trajectory of each algorithm is compared on a dynamic sequence. The pose estimation result of the present invention is in high agreement with the true value and the trajectory error is smaller, which verifies the effect of static weight injection dense bundle adjustment on dynamic interference.

[0076] like Figure 9 As shown, under a unified static region evaluation protocol, the static region rendering of this invention is compared with existing algorithms such as MonoGS, Splat-SLAM, and WildGS-SLAM. Static assumption methods leave significant trails and artifacts in areas where moving objects have appeared; some dynamic baselines leave faint shadows or holes at the contours and exhibit overall blurring. This invention effectively suppresses dynamic objects in Gaussian mapping and rendering, resulting in cleaner static backgrounds, more accurate colors, more coherent structures, and sharper textures, making it more realistic in complex distant scenes.

[0077] like Figures 10-12 As shown, under a unified static region-only evaluation protocol, the present invention is quantitatively compared with existing algorithms such as MonoGS, Splat-SLAM, and WildGS-SLAM in terms of three rendering quality indicators: PSNR, SSIM, and LPIPS. The present invention outperforms or approaches the best existing methods in all three indicators, further verifying the improvement of static region rendering quality by the dynamic suppression strategy.

[0078] Example 2.

[0079] Corresponding to Embodiment 1 of the present invention, Embodiment 2 of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps in accordance with the method of Embodiment 1.

[0080] Step S1: Acquire monocular RGB image sequences, select keyframes and estimate poses, predict and measure depth for each keyframe, and generate coarse dynamic priors.

[0081] Step S2: Based on the mask refinement network, the coarse dynamic prior is refined into a soft dynamic mask, and then the soft dynamic mask is mapped to pixel-wise static weights. The mask refinement network uses the rendering residual as a self-supervised signal.

[0082] Step S3: During the tracking phase, the static weights are injected into the dense bundle adjustment and multiplied pixel by pixel with the pixel-by-pixel matching confidence predicted by the network to minimize the reprojection error and estimate the keyframe pose.

[0083] Step S4: In the mapping stage, the scene is represented by a 3D Gaussian splash. The rendering residuals are generated by blending the rendering color, depth and structural residuals and then undergoing convergence perception gating and morphological processing to reconstruct the static Gaussian map.

[0084] The aforementioned storage media include: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), optical discs, and other media capable of storing program code. The specific limitations regarding the steps implemented after program execution in a computer-readable storage medium can be found in Embodiment 1, and will not be described in detail here. Example 3.

[0085] Corresponding to Embodiment 1 of the present invention, Embodiment 3 of the present invention provides a computer system, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, it performs the following steps in accordance with the method of Embodiment 1. Step S1: Acquire monocular RGB image sequences, select keyframes and estimate poses, predict and measure depth for each keyframe, and generate coarse dynamic priors.

[0086] Step S2: Based on the mask refinement network, the coarse dynamic prior is refined into a soft dynamic mask, and then the soft dynamic mask is mapped to pixel-wise static weights. The mask refinement network uses the rendering residual as a self-supervised signal.

[0087] Step S3: During the tracking phase, the static weights are injected into the dense bundle adjustment and multiplied pixel by pixel with the pixel-by-pixel matching confidence predicted by the network to minimize the reprojection error and estimate the keyframe pose.

[0088] Step S4: In the mapping stage, the scene is represented by a 3D Gaussian splash. The rendering residuals are generated by blending the rendering color, depth and structural residuals and then undergoing convergence perception gating and morphological processing to reconstruct the static Gaussian map.

[0089] The specific limitations regarding the computer system implementation steps described above can be found in Example 1, and will not be described in detail here. It should be noted that each block in the block diagrams and / or flowcharts in the accompanying drawings of this invention, as well as combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified function or action, or by a combination of dedicated hardware and computer instructions.

[0090] The present invention has been described above by way of example with reference to the accompanying drawings. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any non-substantial improvements made using the inventive concept and technical solution of the present invention, or the direct application of the inventive concept and technical solution of the present invention to other occasions without modification, are all within the protection scope of the present invention.

Claims

1. A visual localization and Gaussian splash scene reconstruction algorithm for dynamic scenes, characterized in that: Includes the following steps: Step S1: Acquire monocular RGB image sequences, select keyframes and estimate poses, predict and measure depth for each keyframe, and generate coarse dynamic priors; Step S2: Based on the mask refinement network, the coarse dynamic prior is refined into a soft dynamic mask, and then the soft dynamic mask is mapped to a pixel-wise static weight. The mask refinement network uses the rendering residual as a self-supervised signal. Step S3: During the tracking phase, the static weights are injected into the dense bundle adjustment and multiplied pixel by pixel with the pixel-by-pixel matching confidence predicted by the network to minimize the reprojection error and estimate the key frame pose. Step S4: In the mapping stage, the scene is represented by a 3D Gaussian splash. The rendering residuals are generated by blending the rendering color, depth and structural residuals and then undergoing convergence perception gating and morphological processing to reconstruct the static Gaussian map.

2. The visual localization and Gaussian splash scene reconstruction algorithm in a dynamic scene according to claim 1, characterized in that: In step S1, for each keyframe, the corresponding depth measurement is predicted from the RGB image by the monocular depth estimation model and normalized. This is then combined with a common input dynamic prior extraction module to obtain an initial coarse dynamic prior. An S-shaped function transformation with threshold and temperature is applied to the initial coarse dynamic prior to obtain a normalized coarse dynamic prior. The formula is: ， in, , The corresponding threshold parameters and temperature parameters are listed in order. It is an S-shaped function.

3. The visual positioning and Gaussian splash scene reconstruction algorithm in a dynamic scene according to claim 2, characterized in that: In step S2, the mask thinning network is a lightweight U-shaped network. Its input consists of a coarse dynamic prior, an RGB image, and a concatenation of a normalized depth metric along the channel dimension. The encoder uses average pooling for downsampling, and the decoder uses bilinear interpolation for upsampling. Multi-scale features are fused through skip connections, and the output after encoding and decoding is a soft dynamic mask. The refined soft dynamic mask is converted into pixel-by-pixel static weights. .

4. The visual positioning and Gaussian splash scene reconstruction algorithm in a dynamic scene according to claim 3, characterized in that: In step S2, the online training signal of the mask refinement network consists of rendering consistency loss, prior consistency loss and sparsity loss. The soft target is obtained by remapping the fused rendering residual of the current keyframe using a sigmoid function. The rendering consistency loss is then obtained by calculating the binary cross-entropy between the soft dynamic mask and the soft target using residual gating. The expression is as follows: in, For pixel domain, For pixels, It is a residual gate, and BCE is a binary cross-entropy; The corresponding loss for the positive sample term is obtained by applying a weighted binary cross-entropy to the positive and negative sample seeds. and negative sample term loss The expression is as follows: Among them, weight , Gating is performed based on whether the residuals are consistent with the coarse dynamic prior; positive sample terms are gated at the strong seed of the coarse dynamic prior. Pulling towards 1, the negative sample term will be at a point where the coarse dynamic prior is empty and the residuals are also inactive. Pull towards 0; The ultimate training goal of mask refinement networks The formula is: ， in, Loss for positive sample terms Negative sample term loss Rendering consistency loss sparsity loss The weight.

5. The visual localization and Gaussian splash scene reconstruction algorithm in a dynamic scene according to claim 3, characterized in that: In step S2, when the coarse dynamic prior cannot be stably trained by relying solely on prior consistency loss across the entire frame, the supervision path is switched to be dominated by rendering residuals, allowing soft dynamic masks only at residual confidence points. The background error is increased and limited by sparsity loss; at the same time, a dual-evidence veto is introduced: the fused rendering residual is used as the first evidence, and the structural similarity residual calculated separately based on structural similarity is used as the second evidence. The first evidence and the second evidence are binarized into regions R and S, respectively, and the overlap rate of these two regions is calculated: When the overlap rate Below the set threshold, or in the area number of pixels When there is insufficient residual evidence, the frame is determined to contain no reliable dynamics and a soft dynamics mask is applied. Set the entire frame to zero; otherwise, it indicates that the evidence from the fusion rendering residual and the structural similarity residual is consistent and strong enough to allow dynamic discrimination driven by the residual.

6. The visual localization and Gaussian splash scene reconstruction algorithm in a dynamic scene according to claim 3, characterized in that: In step S3, the keyframes static weights The static weights of dense bundle adjustment are obtained by downsampling to the low-resolution grid used in dense bundle adjustment. Then match the pixel-by-pixel confidence score predicted by the network. Pixel-by-pixel multiplication yields the dynamic sensing weights. The expression is: ; Use dynamic sensing weights to assign weights to each edge in the keyframe factor graph Weighted reprojection errors are used to construct a dynamically perceived dense bundle adjustment optimization objective. The corresponding formula is: in, To predict the target pixel position, and For projection and back projection functions, , For the first i The and the first j Keyframe pose Inverse depth; dense bundle adjustment optimization objective Minimize the solution of keyframe pose.

7. The visual positioning and Gaussian splash scene reconstruction algorithm in a dynamic scene according to claim 6, characterized in that: In step S4, a dense scene is represented by a 3D Gaussian splash, and the rendered color, rendering depth, and cumulative opacity are obtained from the 3D Gaussian splash rendering. Based on these rendering results, map iterative optimization is performed within a sliding keyframe window, and the color residuals are obtained by comparing the rendering results with observations. Depth residual With structural residuals And the three are merged into the initial residual. The formula is: ,in, Color residuals Depth residual Structural residuals The corresponding weights are then applied; a convergence-aware gating based on cumulative opacity is then applied, retaining the fused rendering residual only on pixels where the map has converged. The expression is: ,in Opacity threshold For characteristic functions, operators Median filtering, morphological opening, and small connected component removal are performed sequentially, and frame-level rejection is triggered when the median of the frame-level residual is abnormally high.

8. The visual positioning and Gaussian splash scene reconstruction algorithm in a dynamic scene according to claim 7, characterized in that: In step S4, static weights are used to suppress dynamic content in four stages: backprojection initialization, rendering loss weighting, densification scoring, and dynamic Gaussian culling. These include: In the backprojection initialization phase, dynamic filtering is performed before generating the new Gaussian mask from the keyframe depth backprojection. Pixels whose soft dynamic mask depth exceeds the mask threshold are set to zero. The filtering algorithm is as follows: ,in Represents the corresponding pixel , In order, they are the keyframe depth after dynamic filtering, the normalized metric depth, and the mask threshold; In the rendering loss weighting process, pixels still involved in map building are given static weights. Pixel-by-pixel weighted construction of photometric loss With depth loss The details are as follows: ， and isotropic regularization terms The loss function for jointly constructing the overall mapping objective: ， in, , , The weights of the structural residual, photometric loss, and isotropic regularization term are calculated in sequence. In the densification process, each Gaussian center is projected onto the current viewpoint and sampled. Its accumulated gradient in screen space is weighted and scored using static weights, as shown in the following formula: ， in, The result is a compacted score, i.e., a Gaussian score. Let Gaussian center be the projected pixel value at the current viewpoint. The cumulative number of times within a densification cycle. The number of iterations for densification. For corresponding projected pixels Spatial gradient; In the dynamic Gaussian culling process, an active culling is performed after each keyframe optimization. The center of the Gaussian is projected onto the current viewpoint for sampling, and Gaussians that fall into the dynamic region and are close to the surface of the observed foreground are directly deleted.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the visual positioning and Gaussian splash scene reconstruction algorithm in a dynamic scene as described in any one of claims 1-8.

10. A computer system comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: When the processor executes the computer program, it implements the steps of the visual positioning and Gaussian splash scene reconstruction algorithm in a dynamic scene as described in any one of claims 1-8.

Images