Double-layer dynamic probability SLAM method for high-fidelity mapping in dynamic environment
By employing a two-layer dynamic probabilistic SLAM method, combining pixel-level geometric cues and object-level semantic information, and utilizing a static map feedback mechanism, the problem of inaccurate recognition of dynamic objects in dynamic environments is solved, achieving high-fidelity mapping and stable localization, thereby improving the system's robustness and map quality.
Patent Information
- Application Number
- CN202511255330.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-04
- Publication Date
- 2025-12-12
AI Technical Summary
Existing technologies struggle to accurately identify the geometric constraints of temporarily stationary objects in dynamic environments, leading to inaccurate dynamic object recognition. This results in the erroneous merging of dynamic objects into static maps, producing artifacts. Furthermore, the lack of semantic-level assistance affects the system's positioning accuracy and map integrity.
A two-layer dynamic probabilistic SLAM method is adopted, which combines pixel-level geometric cues and object-level semantic information. The camera tracking module performs pixel-level dynamic probability estimation and the map building module constructs a 3D map. The static map feedback mechanism is used to perform closed-loop perception and correction of dynamic objects, remove dynamic objects, and optimize camera pose and map quality.
It significantly improves the system's robustness and map consistency in dynamic environments, effectively eliminates dynamic artifacts, and enhances the accuracy and completeness of static map reconstruction.
Smart Images

Figure CN121120827A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a two-layer dynamic probabilistic SLAM (Simultaneous Localization and Mapping) method for high-fidelity mapping in dynamic environments, belonging to the field of dynamic 3D map construction and localization technology. Background Technology
[0002] Currently, researchers mainly employ two technical approaches to address the 3D reconstruction and localization problem in dynamic environments. One approach is based on geometric consistency detection, which uses optical flow estimation and geometric constraint analysis to infer pixel regions with motion conflicts from images, and then weakens or removes them during pose optimization and map reconstruction. The other approach uses semantic prior information for region filtering, removing predefined dynamic categories (such as people and vehicles) entirely from the map building process. These approaches are widely used in SLAM systems based on emerging representation methods such as neural implicit fields or 3D Gaussian rendering. However, the former, relying entirely on geometric information, is prone to misjudgment in object boundary regions, thus weakening the accuracy of dynamic region recognition; while the latter, by indiscriminately removing dynamic objects, fails to utilize the geometric constraints provided by targets that are stationary for a period of time (i.e., "temporarily static objects"), thus exhibiting a decline in tracking robustness in dynamic-dominated scenes. In addition, some methods attempt to introduce pixel-level uncertainty estimation as a measure of the degree of dynamics, thereby assigning different weights to different pixels during optimization and improving the robustness of the system. However, this type of method usually has two drawbacks: first, because dynamic objects are not removed, dynamic objects are mistakenly merged into static maps, resulting in artifacts; second, the dynamic degree estimation used usually lacks semantic assistance, making it difficult to obtain clear object boundaries.
[0003] Existing technologies for dynamic region recognition generally rely on geometric consistency cues, lacking semantic prior guidance, leading to ambiguous judgments of object boundaries and frequent mislabeling. Furthermore, dynamic objects are often uniformly excluded from the mapping process, ignoring situations where some objects are in a short-term static state and can provide effective geometric constraints, thus affecting the overall positioning accuracy and map integrity. While some methods employing pixel-level weighting strategies alleviate local interference during optimization, the lack of object-level modeling and processing mechanisms still makes it difficult to avoid the introduction of dynamic artifacts and the accumulation of long-term errors. Summary of the Invention
[0004] Existing methods often directly remove predefined dynamic objects, ignoring the geometric constraints of temporarily stationary objects in pose estimation. Furthermore, the uncertainty maps generated based on purely geometric information exhibit ambiguity at object boundaries, leading to inaccurate dynamic object recognition. To address the issues of dynamic object interference, artifact generation, and insufficient semantic consistency in dense 3D mapping for mobile robots in dynamic scenes, this invention proposes a two-layer dynamic probabilistic SLAM method for high-fidelity mapping in dynamic environments. By constructing a collaborative two-layer dynamic probabilistic framework, combining pixel-level geometric cues and object-level semantic information, it fully utilizes the geometric constraints of temporarily stationary targets while avoiding the risk of long-term map contamination. By introducing a dynamic feedback update mechanism and a Gaussian map-level dynamic object removal method, it achieves closed-loop perception and continuous correction of dynamic information, effectively overcoming the limitations of existing methods in terms of dynamic object recognition granularity, response timeliness, and other aspects. Figure 1 This addresses multiple defects in consistency, thereby significantly improving the system's robustness and mapping quality.
[0005] The present invention provides a two-layer dynamic probabilistic SLAM method for high-fidelity mapping in dynamic environments, which mainly includes a camera tracking module and a map building module. The method includes the following steps:
[0006] Step 1: The camera acquires continuous image frames of the target scene, extracts the key frame sequence of the continuous image frames as input to the camera tracking module, and uses optical flow information and semantic labels to perform pixel-level dynamic probability estimation of the image, and optimizes the camera pose solution through a probability weighting mechanism.
[0007] The camera tracking module extracts semantic tags and dense optical flow from each input frame image, calculates the dynamic probability of each pixel based on the optical flow residual, and aggregates the dynamic probabilities of pixels based on the semantic tags to obtain a two-dimensional pixel-level dynamic probability map for each frame image. Then, the pixel-level dynamic probabilities are used as weights to optimize the camera pose solution, obtaining the optimal values of camera pose and depth for each input frame image. The camera tracking module outputs the static region of each frame based on the pixel-level two-dimensional dynamic probability map.
[0008] Step 2: The map building module obtains the camera pose, two-dimensional dynamic probability map, and static regions in each frame of the image calculated by the camera tracking module, and builds a three-dimensional map.
[0009] Each frame of image is input into the map building module in sequence. The map building module converts the 2D image into a 3D map according to the camera pose of the frame. Based on the 2D dynamic probability map of the frame, the 3D dynamic probability of each object in the image is calculated. According to the set threshold, objects judged to be dynamic are removed from the map. In the process of building the 3D map, the map building module also corrects the dynamic probability of objects in the image based on the feedback mechanism of the static map, and corrects and completes the semantic labels of objects.
[0010] In step 1, the camera tracking module uses pixel-level dynamic probabilities as weights to optimize the camera pose solution. The objective function for optimization is as follows:
[0011]
[0012] The above formula represents finding the optimal values of camera pose ω and depth d that minimize the objective function value; E represents the set of edges established by all keyframes in a continuous image frame, and (i,j) represents the edge connecting keyframe i to keyframe j; u ij Π represents the dense optical flow estimated from keyframe i to keyframe j. c ω represents the camera projection function from 3D to 2D. j ω represents the camera pose of keyframe j. i d i Represent the camera pose and depth of keyframe i, respectively; u i P represents the pixel of keyframe i; i A pixel-level dynamic probability map representing keyframe i; the camera tracking module inputs consecutive image frames into a ConvGRU-based optical flow estimation model to extract pixel motion estimation results, Σ ij This represents the confidence level of the ConvGRU output. ConvGRU is a convolutional gated recurrent unit.
[0013] In step 2, when the map building module builds a 3D map, it projects each pixel in the image as a 3D Gaussian ellipsoid to obtain the 3D map. The attributes of each Gaussian ellipsoid are set to include geometric position, color, transparency, semantic label and dynamic probability. The dynamic probability value of the Gaussian ellipsoid on the static area of the image is set to 0. The initial dynamic probability value of the Gaussian ellipsoid on each object in the image is set according to the dynamic probability value of the corresponding pixel in the 2D dynamic probability map of the image.
[0014] In step 2, the map building module corrects the dynamic probability of dynamic objects based on the feedback mechanism of the static map, including: first, generating a geometrically consistent probability map from the current static 3D Gaussian map. Then, the dynamic probability of the image is corrected using the Bayesian update mechanism, and the posterior dynamic probability map of the corrected image is obtained. as follows:
[0015]
[0016] Where k is the current image frame number, γ is a hyperparameter, and P k It is a two-dimensional dynamic probability map of the image calculated by the camera tracking module.
[0017] Compared with the prior art, the present invention has the following advantages and positive effects:
[0018] (1) The method of the present invention can significantly improve the system’s adaptability to dynamic scenes by constructing a two-layer dynamic probability estimation and feedback mechanism that coordinates tracking and mapping. While ensuring tracking stability, it can effectively remove dynamic artifacts and obtain higher fidelity static map reconstruction results.
[0019] (2) The pixel-level weighted optimization, object-level dynamic judgment and semantic tag repair mechanism proposed in this invention can be applied to multiple scenarios and different semantic tag sources. Without relying on fixed object categories, it can achieve robust mapping of dynamic targets and continuous semantic consistency maintenance.
[0020] (3) The method of this invention utilizes optical flow information and semantic tags for pixel-level dynamic probability estimation, and optimizes camera pose solving through a probability weighting mechanism to achieve perception and weakening of dynamic regions, thereby improving the tracking robustness of the system in dynamic environments. Based on the static region information output by the tracking module and the camera pose, the method of this invention performs dynamic probability updates and semantic consistency maintenance on Gaussian points in the map, and introduces a feedback mechanism based on the static map to improve the recognition accuracy of dynamic objects, effectively enhancing the integrity, consistency and anti-interference ability of the map. Attached Figure Description
[0021] Figure 1 This is a flowchart of the two-layer dynamic probabilistic SLAM method for high-fidelity mapping in dynamic environments according to the present invention.
[0022] Figure 2 This is a schematic diagram of the implementation framework of the two-layer dynamic probabilistic SLAM method for high-fidelity mapping in dynamic environments according to the present invention.
[0023] Figure 3 This is a comparison chart of the rendering effect and dynamic probability of the present invention on the BONN dataset.
[0024] Figure 4 This is a tracking accuracy graph of an embodiment of the present invention on the BONN dataset;
[0025] Figure 5 This is a tracking accuracy graph of an embodiment of the present invention on the TUM dataset. Detailed Implementation
[0026] The present invention will now be described in further detail with reference to the accompanying drawings and examples.
[0027] This invention presents a two-layer dynamic probabilistic SLAM method for high-fidelity mapping in dynamic environments, with an implementation flow as follows: Figure 1 As shown, the corresponding implementation framework is as follows: Figure 2 As shown in the figure. This embodiment of the invention illustrates the implementation steps of the method by using a mobile robot to perform 3D mapping in a target scene.
[0028] Step 1. Camera Tracking. The mobile robot acquires continuous image frames of the target scene using its onboard camera and inputs them into the camera tracking module. The camera tracking module uses optical flow information and semantic tags to perform pixel-level dynamic probability estimation on the input images, and optimizes the camera pose solution through a probability weighting mechanism to achieve perception and attenuation of dynamic regions, thereby improving the tracking robustness of the system in dynamic environments.
[0029] The camera tracking module is one of the core modules of this invention, aiming to accurately estimate the camera pose in a dynamic environment and provide a stable preliminary estimate for the subsequent mapping process. To effectively handle interference from dynamic objects, this invention proposes a camera tracking method based on a two-layer dynamic probability framework. The main idea is to use the dynamic probability of each pixel as a weight to guide pose optimization, and combine it with semantic information to improve the recognition accuracy of dynamic objects, ultimately generating an accurate camera pose. The camera tracking module of this embodiment estimates the camera pose using the following steps 11 to 13. This embodiment first extracts a keyframe sequence from each consecutive image frame input to the camera tracking module before proceeding with the following steps. The keyframe sequence extraction rules can be set according to user needs. Alternatively, the entire consecutive image frame can be directly processed as a keyframe sequence. In the following description, the input consecutive RGB image frames are also keyframe sequences.
[0030] Step 11: Extract dense optical flow information from the input continuous RGB image frames, and calculate the dynamic probability of each pixel based on the optical flow residual.
[0031] First, dense optical flow information is extracted from the input RGB image, and the Sampson error metric is used to measure the geometric consistency of each pixel. In this embodiment of the invention, continuous RGB images are input into an optical flow estimation model based on ConvGRU (Convolutional Gated Recurrent Unit) to extract pixel motion estimation results and obtain dense optical flow. Optical flow estimation is performed by matching pixels between adjacent frames to obtain the optical flow residual for each pixel under the assumption of a static scene. The dynamic probability is then calculated based on this residual. Specifically, this embodiment of the invention assumes that the optical flow residual of the pixel follows a chi-square distribution, and utilizes the cumulative distribution function (CDF) of the chi-square distribution. 2 To calculate the dynamic probability of each pixel. Based on the dense optical flow F of the previous frame image. k,k-1 Calculate the current k-th frame RGB image I k The dynamic probability of pixel u as follows:
[0032]
[0033] in, This is the optical flow residual corresponding to pixel u, representing the degree of deviation of that pixel from the static scene assumption. The calculated dynamic probability...
[0034] This reflects the probability that each pixel is a dynamic object; a larger optical flow residual means that the pixel is more likely to belong to a dynamic region.
[0035] Step 12: Extract semantic tags from the image and correct the pixel-level dynamic probability.
[0036] To further improve the recognition accuracy of dynamic regions, this invention introduces semantic information to correct pixel-level dynamic probabilities. In the input image, this invention uses a deep learning model, such as the SAM model, to perform semantic classification on each pixel, obtaining a semantic label map L containing each pixel in the image. k To enhance the robustness of the initial pure geometric probability estimation, this invention performs semantic aggregation on the semantic label map based on the assumption that all pixels on a rigid object share the same motion state. For each pixel with its semantic mask... For a defined object instance o, the initial dynamic probability values of all pixels within its mask region are first collected according to step 11. To ensure that the overall state of the object is determined by any significant motion present within it, the method of this invention sorts these collected probability values in descending order, and then takes the average of the first quartile and the median of the distribution to calculate the final two-dimensional dynamic probability of the object. Finally, by calculating the dynamic probability of the object Assign it the corresponding mask area All pixels within the bounded space are set to zero, while all unlabeled background pixels are set to zero, thus constructing the final robust dynamic probability map P. k Based on the corrected image pixel-level dynamic probability map P k Static regions in an image can be obtained. In this way, the method of the present invention relies not only on geometric information but also on semantic information to more accurately identify dynamic objects.
[0037] Step 13: Use pixel-level dynamic probabilities as weights to guide sparse feature point matching and relative pose optimization, and construct the optimization target.
[0038] Based on pixel-level dynamic probabilities, this invention introduces a weighting term during pose optimization to adjust the traditional bundled adjustment (BA) optimization objective. Specifically, this embodiment of the invention aims to optimize the camera pose by minimizing pixel reprojection error. The optimization objective function is:
[0039]
[0040] The above formula represents finding the optimal values of camera pose ω and depth d for each keyframe in a continuous series of input image frames, minimizing the objective function value. E represents the set of edges established for all keyframes in the continuous image frames, and a keyframe pair (i,j) represents the edge connecting keyframe i to keyframe j; u ij Π represents the dense optical flow estimated from keyframe i to keyframe j. c This represents the camera projection function from 3D to 2D. Represents the pixel u of keyframe i i Combined with depth d i From 2D projection to 3D; ω i d i ω represents the camera pose and depth of keyframe i, respectively; j Represents the camera pose of keyframe j; It is the camera pose transformation from keyframe i to keyframe j; It is the weighted square norm of the error term; Σ ij P represents the confidence level output by ConvGRU. In optical flow estimation, the confidence level indicates the reliability of the model for the estimation results; i The dynamic probability graph represents keyframe i; the superscript -1 indicates the inversion operation.
[0041] This invention reduces the impact of dynamic objects on pose optimization, thereby improving the accuracy of camera pose estimation for both camera pose and depth. Figure 2 The DBA layer in this invention is the BA optimization framework with an adjusted optimization objective. The optimization objective is solved to obtain the optimal values of camera pose ω and depth d for each keyframe.
[0042] Step 2. Map Building. The map building module is another core module of this invention, designed to construct a high-fidelity 3D map in a dynamic environment based on pose information obtained from camera tracking. To address the problems of dynamic object interference and artifacts, this invention employs a scene representation method based on 3D Gaussian rendering and optimizes the construction of the static map through a two-layer dynamic probabilistic framework.
[0043] To further improve the recognition accuracy of dynamic objects and avoid misfusion of dynamic objects, this invention proposes a feedback mechanism based on a static map to correct the dynamic probability map. The key to this method is to use the geometric consistency of the static map to constrain the update of the dynamic probability, ensuring that dynamic objects are correctly identified and eliminated during the optimization process, while improving the reconstruction accuracy of static objects. In traditional dynamic SLAM methods, the estimation of dynamic objects usually depends on pixel-level geometric information. However, due to the movement and changes of dynamic objects, relying solely on geometric information often leads to the failure to completely eliminate dynamic objects, and may even introduce artifacts. This invention, by introducing a feedback mechanism based on a static map, uses the geometric consistency information of the static map to correct the dynamic probability, thereby effectively avoiding these problems. The feedback mechanism based on a static map in this invention is based on two key steps: (1) Static map generation: A static map is generated through an optimization algorithm. After eliminating dynamic objects, the static map reflects the geometric structure of the static environment. The static map contains a three-dimensional Gaussian representation of the static objects, and each Gaussian point has its position information, color, transparency, and semantic label, etc. (2) Dynamic probability feedback: Using the generated static map, the dynamic probability of dynamic objects is checked through geometric consistency, and the dynamic probability is corrected in combination with the static map, thereby avoiding the misfusion of dynamic objects and enhancing the expression of static objects.
[0044] like Figure 1 and Figure 2 As shown, step two of this embodiment of the invention includes the following steps 21 to 25.
[0045] Step 21: The map building module initializes the map based on the camera pose output by the camera tracking module. For each frame of image, a 2D image is projected onto a 3D image according to the optimal camera pose obtained by the camera tracking module to build a 3D map.
[0046] In the map creation process, this invention uses three-dimensional Gaussian points as the representation of the scene. Each three-dimensional Gaussian point is parameterized by its mean, covariance, opacity, RGB color, dynamic probability, and semantic label. The mean and covariance are key parameters describing the spatial location and shape of the three-dimensional Gaussian point. This invention sets the attributes of the three-dimensional Gaussian points to include not only the geometric structure representing the scene but also probabilistic information related to the dynamic characteristics of objects.
[0047] In this embodiment of the invention, for the currently processed image frame, each pixel in the image is projected as a three-dimensional Gaussian ellipsoid to obtain an initial three-dimensional map. Pixels in static areas are initialized as three-dimensional Gaussian ellipsoids through projection, and similarly, pixels representing each object in the image are initialized as three-dimensional Gaussian ellipsoids through projection. Attributes such as geometric position, color, transparency, semantic label, and dynamic probability are set for the three-dimensional Gaussian ellipsoid of each pixel. Initially, the dynamic probability values of the three-dimensional Gaussian ellipsoid are set according to the dynamic probability map output by the camera tracking module.
[0048] Step 22: Perform object-level 3D dynamic probability estimation for each frame of image. During map building, static objects are continuously optimized and updated, while dynamic objects are removed to ensure that the generated map is artifact-free and accurate. This invention designs a removal mechanism based on object-level dynamic probability. First, the 3D dynamic probability of each object needs to be calculated. In this embodiment, the 2D dynamic probability of an object is obtained by aggregating the dynamic probability of each pixel of the object. One implementation is as described in step 12, obtaining the 2D dynamic probability of the object, and then projecting the object-level 2D dynamic probability to obtain the 3D dynamic probability of the object.
[0049] Specifically, for an object in an image with a semantic label attribute of 'o', this embodiment of the invention calculates the three-dimensional dynamic probability of that object. as follows:
[0050]
[0051] in, g represents the set of all Gaussian ellipsoids of objects in the current image whose semantic label attribute is 'o'. i Let p represent the i-th Gaussian ellipsoid in the set. i For the Gaussian ellipsoid g i The dynamic probability, w i For the Gaussian ellipsoid g i The time decay factor reflects the impact of different time steps on the weights of dynamic objects; δ is a hyperparameter, set to δ = 0.3; k represents the frame number of the current image, k i This represents the image frame number where Gaussian points are added to the 3D Gaussian map. The dynamic probabilities of the object's Gaussian points are updated to the calculated 3D dynamic probabilities of the object. The above formula constructs a multi-frame weight fusion strategy using a time decay factor δ, ensuring that more recent frames contribute more, while historical frames gradually decay, enhancing the system's response to rapidly changing objects.
[0052] Step 23: Based on the object's 3D dynamic probability, compare it with the set threshold to determine whether the object is a dynamic object, and delete the dynamic object from the 3D map.
[0053] By using object-level three-dimensional dynamic probabilities, the method of this invention can identify whether an object is dynamic and remove dynamic objects from the map, thus achieving... Figure 2 The method of this invention involves pruning operations. It pre-sets the number of consecutive frames for object detection and a set threshold. It can be configured such that when the dynamic probability of an object exceeds the set threshold, the object is considered a dynamic object and removed from the map. Alternatively, it can be configured such that when the dynamic probability of an object exceeds the threshold for several consecutive frames, the object is removed from the map to avoid the influence of dynamic objects on the static map.
[0054] In this embodiment of the invention, if the 3D dynamic probability of an object exceeds a set threshold for multiple consecutive frames, the object is marked as a dynamic object and removed from the 3D map.
[0055] Step 24: Render the dynamic probability map from the current static 3D map using splatting. Then, the dynamic probability map of the image is updated using the Bayesian update mechanism, and the attributes of the corresponding Gaussian ellipsoid in the map are updated to achieve a time-series closed-loop update of the map.
[0056] In each frame of the image, a dynamic probability map is generated based on the geometric information of the input image. However, relying solely on these dynamic probability maps may result in the incomplete exclusion of dynamic objects. To avoid this, this invention introduces a static map to provide geometric consistency constraints. The static map is obtained by modeling static objects and includes all static Gaussian points. This static map provides a reference for the dynamic probabilities during the optimization process, ensuring that the probability updates of dynamic objects remain consistent with the geometry of the static objects. The true location of the dynamic region is inferred from the static map, and geometric consistency is used to correct the dynamic probabilities, avoiding artifacts of dynamic objects. Specifically, the static map generates a geometric consistency feedback map by comparing it with the current dynamic probability map. This feedback map provides geometric constraints on the static objects, corrects the dynamic probabilities of the dynamic objects, makes the boundaries of the dynamic regions clearer, and effectively reduces the influence of the dynamic objects.
[0057] The embodiments of the present invention obtain a dynamic probability map from a current static 3D map through splatting rendering. as follows:
[0058]
[0059] Where n is the number of Gaussian ellipsoids in the current 3D map, p i It is the dynamic probability of the i-th Gaussian ellipsoid, α i α j T represents the opacity coefficients of the i-th and j-th Gaussian ellipsoids, respectively. iIt is the cumulative transparency factor of the i-th Gaussian ellipsoid; o i The opacity of the Gaussian ellipsoid; α i It is calculated in two-dimensional pixel space using the center μ of the corresponding Gaussian ellipsoid and the pixel coordinate u, Σ 2D The two-dimensional covariance represents the Gaussian ellipsoid.
[0060] To ensure accurate correction of dynamic probabilities, this invention employs a Bayesian update mechanism. The Bayesian update mechanism corrects dynamic probabilities by establishing a relationship between the static map and the current dynamic probability map. The specific formula is as follows:
[0061]
[0062] Where k represents the current image frame number, γ is a hyperparameter, and γ = 0.5; P k It is the initial dynamic probability map obtained from previous camera tracking calculations. This represents the updated posterior dynamic probability graph. This invention employs a power-law reinforcement mechanism for feedback correction of dynamic probabilities, modifying the rendered graph... An exponential enhancement term γ is introduced as a geometric consistency monitoring signal to increase the confidence level of consistent regions. Through this closed-loop feedback mechanism, the dynamic probability is continuously optimized, ultimately resulting in more accurate dynamic region identification results.
[0063] Through this formula, the Bayesian update mechanism can combine information from the static map to correct the dynamic probability map, thereby obtaining a more accurate dynamic probability estimate. The core idea of this process is that when the static map has strong geometric consistency, the dynamic probability map will be corrected based on the geometric information provided by the static map, reducing the probability of dynamic regions and increasing the probability of static objects. In this way, this invention not only eliminates the interference of dynamic objects but also improves the reconstruction accuracy of static objects. Throughout the dynamic probability correction process, this invention continuously updates the dynamic probability map through stepwise optimization, ensuring that the dynamic probability map of each frame gradually approaches the true location of dynamic objects. Whenever a new image frame is processed, the system first corrects the initial dynamic probability map through geometric consistency feedback and the Bayesian update mechanism, then compares the updated dynamic probability map with the dynamic probability map of the previous frame for further correction and optimization. This stepwise optimization process ensures that the system can operate stably in dynamic environments, accurately identify and eliminate dynamic objects, and precisely optimize the map representation of static objects. Over time, the accuracy of the entire map is continuously improved, and the robustness of the system is enhanced.
[0064] By employing a feedback mechanism based on static maps, this invention effectively eliminates dynamic objects, ensuring that the final generated map contains only static objects. Furthermore, the geometry of static objects is enhanced, ensuring more accurate map representation in dynamic environments. Simultaneously, the 3D Gaussian points of static objects are modeled more precisely, improving the map's geometric consistency and stability.
[0065] Step 25: The map building module corrects and completes the semantic labels of objects in the map.
[0066] In addition to geometric constraints, this invention introduces a semantic label optimization strategy to improve the semantic consistency of the map. By correcting and completing the semantic labels of dynamic objects, this invention further improves the fidelity of the 3D map. Specifically, this invention utilizes temporal consistency and occlusion compensation strategies to correct cross-frame semantic labels, thereby eliminating label inconsistencies caused by dynamic object occlusion or occluded areas. In high-fidelity mapping in dynamic environments, semantic information not only helps improve map expressiveness but also enhances map robustness. Due to occlusion or misidentification of dynamic objects, cross-frame semantic labels may become inconsistent, affecting the overall consistency of the map. To address these issues, this invention proposes a dynamically perceived semantic label optimization method, aiming to solve the problems of dynamic object occlusion and label drift through optimization strategies based on spatial consistency and temporal continuity, thereby improving the semantic consistency of the map.
[0067] First, the camera tracking module of the present invention obtains the semantic label L of each pixel by inputting the image. k These semantic labels are predicted using a semantic segmentation model such as a deep neural network, and a label category is assigned to each pixel. In each frame of the image, these labels are passed to a 3D Gaussian map and attached to the Gaussian ellipsoid corresponding to that pixel. To ensure spatial consistency of the labels, this invention optimizes the semantic labels across multiple frames, ensuring that the labels remain consistent over time. The predicted semantic map of the 3D map projected by the map building module for the current frame is obtained.
[0068] To achieve label consistency across frames, this invention first calculates the semantic label map L for each frame of the image. k and The degree of matching between them. To this end, the present invention constructs a matching matrix S. m Used to measure L k and Spatial consistency. Specifically, the matching metric has three main indicators:
[0069] Intersection over Union (IoU): This measures the ratio of the intersection to the union of two labeled regions, reflecting the consistency of semantic regions.
[0070] Prediction accuracy: The accuracy of label matching is evaluated by comparing the area ratio of the overlapping region of two label regions.
[0071] Recall: Evaluates the coverage of two label regions during the matching process to avoid missing labels.
[0072] Construct the cross-label matching matrix S m S m (i,j) is used to measure L k Chinese semantic tags and Chinese semantic tags
[0073] Matching degree S m (i,j) are as follows:
[0074]
[0075] By selecting the maximum value, the optimal tag matching degree S can be obtained. m (i,j).
[0076] For areas with low tag matching accuracy, tag errors or loss may occur due to occlusion by dynamic objects or tag drift. To compensate for these deficiencies, this invention employs a tag repair strategy. Specifically, this invention uses the following methods for tag correction:
[0077] Label completion: For a label region in the current frame, if no matching label was found in the previous frame, the label is completed using spatial consistency rules. The completion process is based on interpolation between the spatial location of the image and the label in the previous frame to infer the label that should be present in the current frame.
[0078] Label replacement: If the label of the current frame is inconsistent with the label of the previous frame, and the label region of the previous frame has a high matching degree, that is, the matching degree is greater than a preset threshold, then this invention replaces the label in the current frame with the label of the previous frame. This can effectively correct the label drift problem caused by dynamic object occlusion.
[0079] The corrected semantic labels are reassigned to the corresponding Gaussian ellipsoids in the 3D map, thus updating the semantic attributes of the 3D map.
[0080] The dynamic perception semantic label optimization method of this invention achieves label maintenance based on temporal consistency. In a multi-frame image sequence, this invention maintains label consistency by propagating labels between consecutive frames. Specifically, this invention uses the label information of the current frame and propagates the labels to the next frame image through spatial and temporal continuity rules. This effectively avoids label misalignment or drift caused by object movement. The basic idea of label propagation is that, in a temporal sequence, the label information of the current frame affects the label estimation of subsequent frames. This invention calculates the similarity of labels between consecutive frames, gradually adjusts and propagates labels, ensuring that the labels of each frame remain consistent. Especially in dynamic scenes, this effectively reduces the impact of occlusion and motion. This invention repeats the above process on consecutive frames: rendering a 3D map of the current frame to obtain a predicted semantic map. Compute two semantic graphs L k and The matching degree of semantic tags in the image is determined by comparing the semantic tags of the previous frame with the semantic tag graph L of the current frame. k Make corrections and additions, using the corrected L k The semantic information of the 3D map in the current frame is updated. The method of this invention ensures the spatial and temporal consistency of semantic labels across consecutive frames.
[0081] like Figure 3 As shown, the mapping performance of the method of this invention is compared with that of existing methods on the BONN dataset. The three dynamic SLAM methods compared are SplaTAM, Spla-SLAM, and WildGS-SLAM. As shown in the red box in the figure, compared with existing methods, this invention avoids the occurrence of artifacts in map rendering. Specifically, other dynamic SLAM algorithms cause artifacts during rendering because they incorrectly integrate temporarily stationary dynamic objects into the map. However, this invention ensures the artifact-free rendering result by removing dynamic objects at the object level. In the mv_box sequence, the method of this invention correctly identifies the cardboard box moving from the table to the ground as a dynamic object and removes the object at its initial and final positions, thus effectively avoiding the occurrence of artifacts. In addition, this invention can accurately estimate the probability of dynamic objects, while the uncertainty estimation of previous methods near the boundaries of dynamic objects is relatively inaccurate, resulting in the inability to effectively identify dynamic regions. Through accurate dynamic probability estimation, this invention effectively eliminates the interference of dynamic objects in the mapping process, providing a more reliable and accurate 3D map.
[0082] The camera tracking experimental results on the BONN and TUM RGB-D dynamic datasets are as follows: Figure 4 and Figure 5As shown, compared with traditional methods, our method, namely the method of this invention, significantly improves performance on both datasets. On the BONN dataset, the average camera pose error of our method is 0.022 meters; on the TUM RGB-D dataset, the average camera pose error is also 0.022 meters, a reduction of 19% compared to existing methods. These results verify the accuracy and efficiency of this invention, demonstrating that it can significantly improve the robustness and accuracy of camera tracking in dynamic scenes.
[0083] This invention generates a dynamic probability map by fusing semantic and geometric information at the pixel level, then elevates it to 3D space and aggregates it to obtain instance-level dynamic probabilities. This enables dynamic Gaussian categorical culling, constructing an artifact-free static map. The static map, in turn, provides geometric consistency constraints for the pixel-level dynamic probabilities, improving their reliability. Based on this, a dynamically perceptive semantic label optimization strategy is proposed, utilizing temporal consistency and occlusion compensation to eliminate cross-frame semantic inconsistencies. Experiments show that this invention achieves up to 19% improvement in camera tracking accuracy on the BONN and TUM RGB-D dynamic datasets compared to existing methods, and can generate high-fidelity, semantically consistent 3D maps, making it suitable for high-precision mapping and localization tasks in dynamic environments.
[0084] Except for the technical features described in the specification, all other technologies are known to those skilled in the art. Descriptions of well-known components and technologies are omitted in this invention to avoid redundancy and unnecessary limitation. The embodiments described above do not represent all embodiments consistent with this application. Various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this invention are still within the protection scope of this invention.
Claims
1. A two-layer dynamic probabilistic SLAM method for high-fidelity mapping in dynamic environments, characterized in that, This method includes a camera tracking module and a map building module. The camera acquires continuous image frames of the target scene, and the keyframe sequence of the continuous image frames is extracted as input to the camera tracking module. The method includes the following steps: Step 1: The camera tracking module extracts semantic labels and dense optical flow for each input frame image, and calculates the dynamic probability of each pixel based on the optical flow residual; it aggregates the dynamic probabilities of pixels based on semantic labels to obtain a two-dimensional pixel-level dynamic probability map for each frame image; then, it uses the pixel dynamic probabilities as weights to optimize the camera pose solution and obtain the optimal values of camera pose and depth for each keyframe; the camera tracking module obtains the static region of each frame image based on the two-dimensional dynamic probability map of each frame image. Step 2: The map building module obtains the camera pose, 2D dynamic probability map, and static regions in each frame of the image calculated by the camera tracking module, and builds a 3D map, including: Each frame of image is input into the map building module in sequence. The map building module converts the 2D image into a 3D map according to the camera pose of the frame. Based on the 2D dynamic probability map of the frame, the 3D dynamic probability of each object in the image is calculated. According to the set threshold, objects judged to be dynamic are removed from the map. In the process of building the 3D map, the map building module also corrects the dynamic probability of objects in the image based on the feedback mechanism of the static map, and corrects and completes the semantic labels of objects.
2. The method according to claim 1, characterized in that, In step 1, the camera tracking module obtains a two-dimensional pixel-level dynamic probability map for each frame of the image. This is achieved by extracting dense optical flow from the input RGB image frame sequence, obtaining the optical flow residual for each pixel under the static scene assumption, and calculating the dynamic probability of each pixel based on the optical flow residual. Assuming the optical flow residual of a pixel follows a chi-square distribution, the dynamic probability is calculated based on the dense optical flow F of the previous frame. k,k-1 Calculate the current frame image I k The dynamic probability of pixel u for: Where CDFχ 2 It is the cumulative distribution function of the chi-square distribution. It is the optical flow residual of pixel u under the assumption of a static scene.
3. The method according to claim 1 or 2, characterized in that, In step 1, the camera tracking module corrects the dynamic probability of pixels based on semantic labels. This is achieved by: obtaining the semantic classification of each pixel in the image through a deep learning model to obtain a semantic label map; assuming all pixels on the same rigid object share the same motion state, performing semantic aggregation on the semantic label map; defining a semantic mask for each object, collecting the dynamic probability values of all pixels within the mask area and arranging them in descending order; taking the average of the first quartile and the median from the arranged pixel dynamic probability values as the dynamic probability of the object; setting the dynamic probability values of all pixels within the semantic mask area of the object to the object's dynamic probability value; and setting the dynamic probability values of background pixels without labeled objects in the image to 0, thus constructing a two-dimensional pixel-level dynamic probability map of the image.
4. The method according to claim 1 or 2, characterized in that, In step 1, the camera tracking module uses pixel-level dynamic probabilities as weights to optimize the camera pose solution. The objective function for optimization is as follows: The above formula represents finding the optimal values of camera pose ω and depth d that minimize the objective function value; E represents the set of edges established by all keyframes in a continuous image frame, and (i,j) represents the edge connecting keyframe i to keyframe j; u ij Π represents the dense optical flow estimated from keyframe i to keyframe j. c ω represents the camera projection function from 3D to 2D. j ω represents the camera pose of keyframe j. i d i Represent the camera pose and depth of keyframe i, respectively; u i P represents the pixel of keyframe i; i A pixel-level dynamic probability map representing keyframe i; The camera tracking module inputs consecutive image frames into a ConvGRU-based optical flow estimation model to extract pixel motion estimation results, Σ ij This represents the confidence level of the ConvGRU output. ConvGRU is a convolutional gated recurrent unit.
5. The method according to claim 1, characterized in that, In step 2, when the map building module builds a 3D map, it projects each pixel in the image as a 3D Gaussian ellipsoid to obtain a 3D map. The attributes of each Gaussian ellipsoid are set, including geometric position, color, transparency, semantic label and dynamic probability. The initial dynamic probability value of the Gaussian ellipsoid on each object in the image is set according to the dynamic probability value of the corresponding pixel in the 2D dynamic probability map of the image.
6. The method according to claim 1 or 5, characterized in that, In step 2, the map building module calculates the three-dimensional dynamic probability of each object in the image, which is achieved as follows: For an object in an image with semantic label 'o', calculate the object's three-dimensional dynamic probability. as follows: in, G represents the set of Gaussian ellipsoids over the object in the current image. i Representative set The i-th Gaussian ellipsoid in p i For the Gaussian ellipsoid g i The dynamic probability, w i For the Gaussian ellipsoid g i The time decay factor, δ is a hyperparameter, and k is the frame number of the current image. i The representative will use the Gaussian ellipsoid g i Add the image frame number of the 3D map.
7. The method according to claim 1 or 5, characterized in that, In step 2, the map building module corrects the dynamic probability of dynamic objects based on the feedback mechanism of the static map, including: First, a geometrically consistent probability map is generated from the current static 3D Gaussian map. as follows: Where k is the current image frame number, n is the number of Gaussian ellipsoids in the current 3D map, and p i It is the dynamic probability of the i-th Gaussian ellipsoid, α i α j T represents the opacity coefficients of the i-th and j-th Gaussian ellipsoids, respectively. i It is the cumulative transparency factor of the i-th Gaussian ellipsoid, o i Σ is the opacity of the i-th Gaussian ellipsoid; μ and u are the center and pixel coordinates of the i-th Gaussian ellipsoid, respectively; 2D Represents the two-dimensional covariance of the Gaussian ellipsoid; Then, the dynamic probabilities of the image are corrected using the Bayesian update mechanism as follows: Where γ is a hyperparameter, P k It is a two-dimensional dynamic probability map of the image calculated by the camera tracking module. This represents the posterior dynamic probability map of the updated image.
8. The method according to claim 1, characterized in that, In step 2, the map building module uses a dynamically perceived semantic label optimization method to correct and complete the semantic labels of objects, including: First, obtain the semantic label map L extracted by the camera tracking module for the current frame. k Obtain the predicted semantic map of the 3D map projected onto the current frame by the map building module. Construct the cross-label matching matrix S m Measure L k and Spatial consistency, where L k Chinese semantic tags and Chinese semantic tags Matching degree S m (i,j) are as follows: in, This represents the intersection-union ratio of two semantically labeled regions. This indicates the percentage of overlap between two semantic tag regions. This indicates the coverage of two semantic label regions; by selecting the maximum value, the optimal label matching degree is obtained. Then, the semantic labels of the current frame are corrected and completed, including: (1) for the region in the current frame where the semantic labels are missing, the semantic labels of the region are completed based on the spatial location and the semantic labels in the previous frame image; (2) for the region in the current frame where the semantic labels are inconsistent with those in the previous frame, if the semantic label matching degree of the region in the previous frame is greater than the set threshold, the semantic label of the region in the current frame is replaced with the semantic label of the region in the previous frame. The corrected semantic labels are repeatedly assigned to the corresponding Gaussian ellipsoids in the 3D map.
Citation Information
Cited By
Dynamic fuzzy scene three-dimensional reconstruction method based on 3DGS technology
CN121482288A
