Robust visual SLAM (Simultaneous Localization and Mapping) method for complex dynamic environment

Abstract

The invention discloses a neural implicit vision SLAM (Simultaneous Localization and Mapping) method based on dynamic perception. The method aims at solving the core technical problems that an existing visual SLAM method is insufficient in robustness, poor in global consistency, large in calculation overhead and the like in challenging environments such as dynamic scenes, weak texture areas and violent illumination changes. According to the method, the feature processing capability of deep learning, efficient dynamic object perception, advanced neural implicit mapping and a global optimization mechanism are integrated, so that more accurate camera pose estimation and higher-quality static environment map construction are realized. In the tracking module, a six-step workflow based on mask guidance is adopted, dynamic objects are filtered from the source, frame-level pre-screening is carried out, and the robustness and the calculation efficiency of the system are remarkably improved. In a dynamic local mapping module, a pixel-level fusion method based on transmission probability and inverse variance weight is innovatively adopted, texture blurring and geometric distortion at the boundary of a plurality of sub-maps are effectively inhibited, and the visual quality of a global map is improved. Besides, by introducing a loop candidate frame reordering strategy based on pose uncertainty weighting in loop detection, visual similarity and geometric credibility can be combined, the false detection rate is effectively reduced, and global consistency and long-term precision of the map are ensured.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of computer vision, robot navigation, and artificial intelligence, specifically a robust neural implicit visual SLAM method for complex dynamic environments. This method aims to provide high-precision and robust localization and mapping solutions for applications such as robot navigation, autonomous driving, drone obstacle avoidance, and augmented reality. This invention deeply integrates the high-quality mapping capabilities of neural implicit fields with the robust tracking and global consistency maintenance mechanisms of traditional SLAM, solving the technical challenge of existing technologies in balancing accuracy, robustness, and global consistency in complex dynamic environments. Background Technology

[0002] Simultaneous localization and mapping (SLAM) technology plays a central role in robotics and computer vision, and has made significant progress in static environments. However, the performance of existing technologies still needs improvement when facing complex dynamic visual environments.

[0003] Traditional visual SLAM methods, such as ORB-SLAM3, primarily rely on hand-designed feature points for pose estimation and map construction. These feature points are poorly robust in scenes with drastic lighting changes, weak textures, and dynamic objects, easily leading to feature point tracking failures, which in turn cause inaccurate pose estimation or global system drift. The bottleneck of these traditional methods in handling dynamic objects lies in their "post-processing" mode; that is, they attempt to remove dynamic interference only after feature point extraction and matching are complete, resulting in low efficiency and poor performance.

[0004] In recent years, emerging neural implicit SLAM methods, such as iMAP and NICE-SLAM, have achieved high-quality dense map reconstruction through techniques like neural radiation fields (NeRF). However, these methods generally lack effective global bundle adjustment (BA) and loop closure detection mechanisms. This leads to the accumulation of significant pose errors in long-term operation or large-scale scenes, eventually causing trajectory drift or tracking failure. This is especially true in scenes with severe dynamic disturbances. Figure 1 Consistency is difficult to guarantee, and it cannot meet the needs of long-term, large-scale applications.

[0005] Existing methods struggle to achieve both high precision and robust stability in low-texture, low-light, or dynamic interference environments, hindering their widespread application in complex real-world environments. This invention combines the advantages of two paradigms, implementing robust processing at the source of the data processing flow and supplementing the neural implicit mapping method with a global consistency maintenance mechanism necessary for long-term operation, thereby achieving a performance balance that existing systems cannot achieve. Summary of the Invention

[0006] This invention aims to address the core technical problems of existing SLAM systems, such as insufficient robustness, poor global consistency, and high computational overhead in complex dynamic environments. The purpose of this invention is to provide a novel robust neural implicit SLAM system suitable for complex dynamic environments. This system integrates the feature processing capabilities of deep learning, efficient dynamic object perception, advanced neural implicit mapping, and global optimization mechanisms to achieve more accurate camera pose estimation and higher-quality static environment map construction.

[0007] The technical solution of the present invention is as follows:

[0008] This invention proposes a robust visual SLAM method for complex dynamic environments. Its core framework consists of three cooperating modules: a tracking module, a dynamic local mapping module, and a loop closure detection module, and includes the following steps:

[0009] Step 1: The tracking module selects keyframes and estimates camera pose based on the RGB-D image stream. First, it detects moving objects using a YOLOv5 network and generates a binary mask, then performs frame-level pre-screening based on the proportion of moving objects and the magnitude of image content changes. Second, it extracts feature points using a SuperPoint network, and combines the output of the original feature point set with the mask for further processing. Then, it uses the LightGlue model to efficiently match the filtered static feature points, achieving robust inter-frame pose estimation. Finally, the tracking module employs a mask-guided keyframe selection strategy and performs local bundle adjustment optimization on the selected keyframes to complete camera pose estimation.

[0010] Step 2: The loop closure detection module utilizes the DBoW3 bag-of-words model for rapid scene recognition to find loop closure candidate frames. Unlike traditional methods, this invention introduces keyframe pose uncertainty as a weighting factor to re-rank the candidate set after initial retrieval. Subsequently, a deep learning network model based on an attention mechanism is used for geometric verification. Once a loop closure is confirmed, the system performs loop closure correction and global bundle adjustment (BA) to eliminate accumulated trajectory drift and ensure global map consistency.

[0011] Step 3: The dynamic local mapping module, which integrates octrees and implicit neural fields, constructs and updates local sub-maps using a hybrid representation method combining dynamic sparse octrees and implicit neural fields. For image rendering at the boundaries of multiple sub-maps, the mapping module employs a pixel-level fusion strategy driven by uncertainty. This strategy combines the transmission probability of pixels with inverse variance weights during fusion, giving higher weights to pixels with low uncertainty, geometric consistency, and visibility, thereby effectively suppressing texture blurring and geometric distortion in boundary regions.

[0012] Preferably, in step 1, the tracking module processes the input image data stream using a deep learning model, and the specific workflow includes the following six steps:

[0013] Step 1.1, Frame-level pre-screening: After the image enters the YOLOv5 module, the proportion of dynamic objects in the image is first detected, and a rapid comparison is performed based on the semantic information and coordinates of each object in the image with the previous keyframe. If the proportion of dynamic objects is too large (exceeding a preset threshold) or the change in the image compared with the previous keyframe is too small, the frame is rejected from entering the subsequent keyframe extraction process, thus saving computing power from the system entry point. This step is the core of the system's robustness pre-design, as it avoids sending invalid data unsuitable for mapping into subsequent time-consuming processing stages.

[0014] Preferably, in step 1.1, the threshold for the proportion of dynamic objects is set to 70%, and the threshold for the change between the image and the previous keyframe is set to 90%.

[0015] Step 1.2, Dynamic Mask Extraction: If the image passes the pre-screening (i.e., the proportion of dynamic objects is less than a certain threshold and the difference from the previous keyframe meets a certain threshold), the YOLOv5 module generates a binary mask for the dynamic objects. This mask accurately marks the dynamic regions in the image, providing crucial guiding information for subsequent feature extraction.

[0016] Step 1.3 involves extracting feature points using the SuperPoint network. This network takes the original image as input and combines the output set of original feature points with a binary mask provided by YOLOv5 as its output. The mask network avoids generating feature points in dynamic regions, thus fundamentally ensuring the reliability of the extracted feature points. The network ultimately outputs a descriptor tensor, serving as the descriptor for each static feature point.

[0017] Step 1.4: Feature matching is performed based on the LightGlue model. This model consists of multiple layers, each containing a self-attention module and a cross-attention module. The self-attention module updates the representation of each feature point using its own features, while the cross-attention module exchanges information between the feature point sets of two images, ensuring effective matching between feature points across frames. The network ultimately outputs an assignment matrix, representing the matching probability between feature points in two images.

[0018] Step 1.5, Keyframe Selection: This invention introduces a mask-guided keyframe selection strategy to further filter candidate keyframes. This strategy dynamically evaluates the static nature of the image by calculating the ratio between the area occupied by dynamic objects and the total image area, and prioritizes image frames with more matching feature points and a relatively low proportion of dynamic objects as keyframes. Preferably, the keyframe with the most matching feature points is selected from the top 50% of all candidate keyframes with the lowest proportion of dynamic objects.

[0019] Step 1.6, Local Optimization: When a frame is selected as a keyframe, the tracking module will perform Local Bundle Adjustment (BA) to optimize the pose of the keyframe and other keyframes related to it.

[0020] Preferably, in step 2, the loop closure detection module specifically comprises:

[0021] This module uses the DBoW3 bag-of-words model to calculate the bag-of-words vector of the current keyframe and compares it with historical keyframes for rapid identification of loop-closure candidate keyframes. After the initial candidate retrieval, this invention introduces keyframe pose uncertainty as a weighting factor to re-rank the candidate set. This addresses the problem of traditional methods, which rely solely on bag-of-words similarity scores and are susceptible to interference from repetitive textures and dynamic objects, leading to increased false positive rates. The specific weighted ranking formula is as follows:

[0022]

[0023] Wherein, S′(K c S is the weighted score. BoW (K c () represents the bag-of-words similarity score. Let be the trace of the covariance matrix of the candidate keyframe pose.

[0024] The advantage of this strategy lies in its clever integration of the output of the front-end tracking module (the pose covariance matrix obtained through local BA optimization) with the visual retrieval process of the back-end loop closure detection. The trace of this matrix is ​​a geometric metric that measures the uncertainty of the pose estimation. A smaller matrix trace value indicates a higher confidence level in the pose estimation. This strategy prioritizes keyframes that are not only visually similar but also have more stable and reliable pose estimations for subsequent geometric verification.

[0025] This cross-module information fusion significantly improves the accuracy and success rate of loop closure detection while reducing unnecessary matching and optimization overhead. It avoids time-consuming and labor-intensive geometric verification of a large number of unreliable candidate frames, thus enabling intelligent management of system resources.

[0026] For the weighted candidate keyframes, the loop closure detection module uses LightGlue for feature matching to improve the robustness of matching under changes in viewpoint and illumination. After matching, RANSAC is used for outlier removal, and geometric consistency is verified by solving the relative pose to ensure the correctness of the matched point pairs under geometric constraints, thereby reducing the possibility of false loop closures.

[0027] Preferably, in step 3, the local mapping module that integrates the octree and the implicit neural field specifically involves: this module using a dynamic sparse octree for spatial partitioning, expanding the hierarchy only within the object-occupied area, significantly reducing storage overhead. At each octree node, the system stores multiple layers of feature vectors and obtains the feature vector of any point in space through trilinear interpolation. This feature vector is input into an implicit neural network composed of two small multilayer perceptrons (MLPs) to predict the occupancy value and color value of that point.

[0028] When the view frustum of the rendered image is located at the intersection of multiple sub-maps, this invention employs an uncertainty-driven pixel-level sub-map fusion method to improve the overall map quality. Existing methods rely solely on the statistical uncertainty of pixels, and are prone to failure when faced with "afterimages" left by dynamic objects or changes in lighting. The fusion method of this invention is a strategy that combines physical geometric properties with statistical properties.

[0029] This invention maintains the expected and variance of the occupancy probability for each pixel in each sub-map to characterize the confidence and uncertainty of the location's occupancy. In the final global rendering stage, for each ray emitted from the camera, the mapping module obtains candidate pixels sorted by depth from multiple sub-maps and calculates the transmission probability of each candidate in turn. Subsequently, during fusion, the transmission probability is combined with inverse variance weights, giving higher weights to pixels with low uncertainty, geometric consistency, and visibility. The specific fusion formula is as follows:

[0030]

[0031] Among them, Z i This represents the depth value of the i-th sub-map at that pixel location. This represents the depth uncertainty of the pixel, μ. i t represents the expected probability of occupancy. i This represents the transmission probability of the pixel, and ε is a stabilizing term to prevent the denominator from being too small.

[0032] This invention combines the visibility of pixels in three-dimensional space (transmission probability) with the statistical uncertainty (inverse variance) in two-dimensional images, representing a more comprehensive fusion strategy. Transmission probability t iBy considering whether light rays are occluded when they travel to a pixel in 3D space, the new method effectively suppresses "afterimages" or "ghosting" left by moving objects. For example, when a moving human leaves a trace on a map, the pixels along its trajectory may have low uncertainty, but because their visibility (transmission probability) is reduced due to occlusion, their contribution to the final map is intelligently assigned a lower weight in the new formula, thus effectively suppressing their contribution to the final map. This makes the new method more robust in dynamic environments than methods that rely solely on uncertainty.

[0033] Beneficial effects

[0034] Compared with the prior art, the present invention has the following significant advantages:

[0035] (1) High robustness and high-precision localization. By introducing dynamic object detection and mask-based feature filtering at the tracking front end, this system can achieve stable pose estimation in complex environments with dynamic, weak texture, and varying illumination. Simultaneously, the innovative loop closure detection strategy, by combining visual appearance and geometric pose reliability, effectively reduces the false detection rate and ensures long-term high accuracy of the global pose. Experiments on the TUM RGB-D dataset show that the ATERMSE index of this invention in dynamic scenes is significantly lower than that of comparative methods.

[0036] (2) It can complete high-quality mapping. The dynamic local mapping module combines the sparsity of octrees and the dense representation of neural implicit fields to achieve efficient and high-quality scene reconstruction. In particular, the newly introduced pixel-level fusion method based on transmission probability and inverse variance weights can intelligently handle the rendering problem at the boundaries of multiple submaps, effectively reduce geometric distortion and texture artifacts in the map, and improve the visual quality of the global map.

[0037] (3) It can maintain global map consistency. By introducing a loop closure detection and global optimization mechanism based on pose uncertainty weighting, this system effectively solves the trajectory drift problem that is common in neural implicit SLAM, ensuring the global consistency and accuracy of the map. This mechanism enables the system to remain stable during long-term operation, especially in large-scale or repetitive scenarios.

[0038] (4) Improved localization and mapping efficiency. The redesigned tracking module reduces unnecessary computation from the source through frame-level pre-screening. The innovative loop closure detection weighted sorting strategy avoids geometric verification of a large number of unreliable candidates by prioritizing high-quality candidate frames, thereby significantly reducing computational overhead. These optimizations together ensure that the system maintains high real-time performance while achieving high accuracy and robustness. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the framework of the method of the present invention;

[0040] Figure 2 This is a schematic flowchart of the method of the present invention;

[0041] Figure 3 This is a schematic diagram of the trajectory successfully tracked by the method of the present invention;

[0042] Figure 4 This is a schematic diagram illustrating the performance analysis of the method of the present invention; Detailed Implementation

[0043] The following is in conjunction with the appendix Figures 1 to 4 The principles and advantages of this invention will be further illustrated by the following embodiments. All structural details mentioned in the embodiments are based on the accompanying drawings.

[0044] The following describes various exemplary embodiments of the present invention with reference to the accompanying drawings.

[0045] Example 1, as Figure 1 As shown, this embodiment describes the overall framework and workflow of a neural implicit SLAM system based on dynamic perception. The specific steps are as follows:

[0046] First, the RGB-D image stream is input to the tracking module. This module first uses a YOLOv5 network to detect dynamic objects and generate masks, then uses a SuperPoint network to extract feature points, and inputs the preserved static feature point set into a LightGlue network for matching to achieve robust inter-frame pose estimation. During tracking, the system employs a mask-guided keyframe selection strategy and performs local bundle adjustment (BA) optimization on the selected keyframes and their associated poses.

[0047] Subsequently, the loop closure detection module identifies loop closure candidate frames using the DBoW3 bag-of-words model. Once a loop closure is confirmed through geometric verification, the system performs loop closure correction and global bundle adjustment (BA) to eliminate accumulated trajectory drift.

[0048] Finally, the optimized keyframes and poses are passed to the dynamic local mapping module. This module integrates octrees and implicit neural fields for dynamic local mapping. First, based on the co-visibility relationships between keyframes, new keyframes are integrated into existing local sub-maps, or new sub-maps are initialized. The mapping module uses a method combining dynamic sparse octrees and implicit neural fields to continuously train and update the sub-maps, thereby achieving high-quality dense reconstruction. Based on the pose optimization results, the system fuses and refines the key sub-maps, ultimately generating a globally consistent and complete static environment map.

[0049] Example 2, based on Example 1, the dynamic object detection network, feature point extraction network, and feature matching network are specifically as follows:

[0050] The dynamic object detection network employs a pre-trained YOLOv5 network model to detect potential dynamic object regions in real time from the input image, and performs frame-level pre-screening based on the proportion of dynamic objects and the magnitude of image content changes. For the selected image frames, corresponding mask images are generated to mask dynamic feature points within the region during subsequent processing.

[0051] The feature point extraction network uses a pre-trained SuperPoint network model, whose structure mainly consists of three parts: a shared encoder, a feature detection decoder, and a descriptor decoder.

[0052] The system comprises several components: an encoder extracting an intermediate feature tensor from the input image, containing multiple channels to represent the image's underlying semantic information; a feature detection decoder calculating the probability of each pixel being a feature point based on the intermediate features, outputting a score tensor where each element represents the confidence level of the corresponding pixel as a feature point; and a descriptor decoder employing a sub-pixel convolution structure to reduce computational complexity and generate semi-dense feature descriptors. To generate complete descriptors, the system uses bicubic interpolation to fill in missing regions, ultimately outputting a complete descriptor tensor representing the feature descriptors corresponding to all feature points in the image.

[0053] The feature matching network employs a pre-trained LightGlue network model, whose structure consists of multiple identical layers. Each layer includes two core modules: a self-attention module and a cross-attention module. The self-attention module enhances the representational ability of feature points within a single image, improving the expression of local features; the cross-attention module enables the interaction of feature point information between two images, thereby improving the matching accuracy between feature points across frames. The model's layers progressively pass information, gradually optimizing the matching results. Each layer also includes a classifier and a corresponding confidence level to determine whether the termination condition is met, reducing redundant computation while maintaining accuracy. During the matching process, the system first normalizes the feature point sets and their corresponding descriptors from the two frames and inputs them into the LightGlue model. The model outputs a matching result representing the correspondence between feature points, presented as a probability matrix, which effectively determines the degree of matching between feature points in different images.

[0054] Preferably, the SuperPoint network model, YOLOv5 network model, and LightGlue network model mentioned above are all deployed through the Libtorch technology framework to achieve efficient model inference and real-time processing performance.

[0055] Example 3, based on Example 1, the mask-guided keyframe selection strategy is as follows:

[0056] The system analyzes the proportion of dynamic regions in the entire image across all candidate keyframes based on the mask information. Candidate keyframes with a small proportion of dynamic regions and a high number of matching feature points are prioritized. First, the proportion of dynamic objects in each candidate keyframe is calculated and sorted from smallest to largest. Then, the frame with the most matching feature points is selected from the top 50% of the ranked candidate keyframes.

[0057] Example 4, based on Example 1, the loop closure detection module is specifically as follows:

[0058] After the loop closure candidate frame search is completed, the candidate set is reordered using keyframe pose uncertainty as a weighting factor. Subsequently, the LightGlue network is used to complete feature matching and scene geometry verification. Once a loop closure is confirmed, the system will perform loop closure correction and global bundle adjustment (BA) to eliminate accumulated trajectory drift and ensure global map consistency.

[0059] Example 5, based on Example 1, the dynamic local mapping module that integrates octrees and implicit neural fields is specifically as follows:

[0060] First, based on co-view relationships, the system determines whether a new keyframe belongs to an existing key sub-map. If the new keyframe has a strong co-view relationship with a local map, it is added to that local map. If the new keyframe does not belong to any existing key sub-map, a new local map is initialized. Then, the mapping module uses a combination of dynamic sparse octrees and neural implicit fields to continuously train and update the sub-maps, achieving high-quality dense reconstruction. Based on the pose optimization results, the system merges and refines the key sub-maps, ultimately generating a globally consistent and complete static environment map.

[0061] The map storage unit uses a dynamic sparse octree for spatial partitioning, expanding the hierarchy only within the areas occupied by objects, and storing multiple layers of feature vectors on the octree nodes. Specifically, given a point in space... The system calculates its corresponding feature vector through trilinear interpolation, and then inputs this feature vector into an implicit neural network, which consists of two small multilayer perceptrons (MLPs), to predict its occupancy value o. i and color value c i Starting from the camera center o, a ray r is emitted along the normalized line-of-sight direction d. We sample N points along ray r, denoted as x. i Where i∈1,2,...,N. For each sampling point, the total depth and color value of ray r can be calculated using the following formula:

[0062]

[0063] Where, α i , Let represent the transparency and transmittance of the i-th sampling point, respectively.

[0064] We optimize the octree feature storage and decoder parameters using photometric loss and geometric loss.

[0065]

[0066] min z,θ λ p L p +L g

[0067] Finally, key submap fusion is performed. Since the scene is decomposed into multiple submaps, two main scenarios must be considered when rendering the image. The first scenario occurs when the view frustum completely intersects with a submap; in this case, we can render the image using only that submap. The second scenario occurs when the view frustum is located at the boundary of multiple submaps, making it impossible to generate a complete image from a single submap. In this case, we employ weighted fusion based on the transmission probabilities and inverse variance weights of candidate pixels from multiple submaps to obtain a more accurate image.

[0068] Example 6: The SLAM method disclosed in this invention was validated on the TUM RGB-D dataset through qualitative and quantitative analysis, specifically as follows:

[0069] The TUM RGB-D dataset is a widely used standard public dataset for evaluating the performance of visual SLAM systems. It includes various challenging scenarios such as dynamic disturbances, weakly textured regions, lighting changes, and fast motion, encompassing RGB images, depth maps, IMU data, and accurate ground trajectories. The SLAM system proposed in this invention was systematically evaluated on this dataset. The experimental platform configuration was: Intel Core i9-13900K processor, NVIDIA GeForce RTX 4080 GPU, and Ubuntu 20.04 operating system. The root mean square error (RMSE) of the absolute trajectory error (ATE) was used to evaluate the accuracy of pose estimation.

[0070] This invention uses ORB-SLAM2 as a benchmark, and also conducts comparative experiments using the following neural implicit SLAM algorithms:

[0071] iMAP: A real-time SLAM system based on neural implicit representation, which realizes implicit encoding of scene geometry through online optimization of keyframes and has the ability to reconstruct locally densely in the observation area;

[0072] NICE-SLAM: A hybrid representation method that combines hierarchical feature voxel grids with neural fields, achieving dense reconstruction of large-scale scenes through multi-resolution voxel encoding, and supporting end-to-end joint optimization of camera pose and scene geometry.

[0073] ESLAM: It uses a hybrid SDF representation method to model and update the environment, and achieves dense map construction through continuous camera tracking and point cloud fusion.

[0074] Comparative experiments in complex dynamic environments:

[0075] To verify the localization accuracy and robustness of the SLAM system of this invention in complex dynamic scenes, four typical high-dynamic sequences from the TUMRGB-D dataset were selected: fr3 / w / xyz, fr3 / w / static, fr3 / w / half, and fr3 / w / rpy. These sequences all contain varying degrees of environmental dynamic changes, including two people walking back and forth in front of a table, object occlusion, changes in lighting, and rapid changes in camera pose (such as translation and rotation), posing a significant challenge to the real-time tracking capabilities of the SLAM system. Figure 3 (b) The camera tracking trajectory of the system of the present invention in the fr3 / w / xyz sequence is shown. It can be seen that the system can stably complete the entire tracking process without obvious drift. Table 1 lists the ATERMSE indexes of each comparative algorithm in these four high dynamic range sequences. The data in the table shows that the method of the present invention achieves significantly better performance than existing methods in all test sequences:

[0076] The experiments demonstrate that the system of this invention exhibits excellent stability and accuracy in highly dynamic environments, possesses stronger real-time robustness and motion adaptability, and can effectively suppress the impact of dynamic interference on visual tracking results.

[0077] Comparative experiments under static conditions:

[0078] To further verify the accuracy performance of the SLAM system of this invention in a typical static environment, the commonly used static sequences fr1 / desk, fr2 / xyz, and fr3 / office from the TUM RGB-D dataset were selected for comparative experiments. These sequences are widely used to evaluate the camera localization accuracy of SLAM systems in static scenes and are representative. Figure 3(a) shows the tracking trajectory of the method of the present invention in the fr3 / office sequence, demonstrating the system's good stability and consistency. Table 2 lists the ATE RMSE comparison results of multiple systems, including the method of the present invention, iMAP, NICE-SLAM, ESLAM, and the traditional method ORB-SLAM2, in the above static sequence. The data in the table shows that the system proposed in this invention not only has significant advantages in dynamic environments but also maintains performance comparable to traditional mature SLAM algorithms in static environments, exhibiting good adaptability and versatility.

[0079] Table 1: Comparison of ATERMSE between the visual SLAM method of this invention and the state-of-the-art neural implicit SLAM method in high dynamic environments.

[0080] sequence name iMAP NICE-SLAM ESLAM This invention fr3 / w / xyz 0.410 0.302 0.432 0.019 fr3 / w / rpy 0.834 0.724 - 0.046 fr3 / w / static 0.102 0.092 0.075 0.016 fr3 / w / half 0.638 0.629 - 0.063

[0081] Table 2: Comparison of ATE RMSE between the visual SLAM method of this invention and the state-of-the-art SLAM method in a static environment.

[0082] sequence name iMAP NICE-SLAM ESLAM ORB-SLAM2 This invention fr1 / desk 0.049 0.027 0.025 0.016 0.018 fr2 / xyz 0.020 0.018 0.011 0.005 0.005 fr3 / office 0.058 0.030 0.024 0.010 0.010

[0083] Example 7: The performance of this invention in map reconstruction was verified on the Replica dataset, specifically as follows:

[0084] The Replica dataset possesses realistic modeling, accurate camera poses, and dense geometric ground truth, making it widely used in neural reconstruction and scene understanding research. It is suitable for quantitatively evaluating reconstruction accuracy and completeness. The method of this invention was evaluated on eight Replica datasets. The system performance analysis is as follows: Figure 4 As shown, the proposed method achieves the shortest mapping time compared to the baseline method in each iteration. Mapping time here refers to the time required to process M = 1000 pixels in a single iteration. This result demonstrates that the proposed method is more efficient in the mapping process. By using only keyframes for processing, we significantly reduce the number of frames required while achieving superior mapping performance.

[0085] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A robust visual SLAM method for complex dynamic environments, characterized in that, It consists of three parts: a tracking module, a loop closure detection module, and a dynamic local mapping module, and includes the following steps: Step 1: Receive RGB-D image data stream and perform frame-level pre-screening to generate a binary mask of dynamic objects in the image. Based on the proportion of the area occupied by the dynamic objects in the image and the difference between the image and the previous keyframe, determine whether to use the frame for subsequent processing. Step 2: If the frame passes the pre-screening, feature points are extracted from the static regions in the frame, wherein the feature point extraction utilizes a deep learning network and a binary mask of the dynamic object. Step 3: Based on the static feature points extracted in Step 2, a feature matching network is used to select keyframes and estimate poses. The selection of keyframes is based on the proportion of the area occupied by dynamic objects in the candidate keyframe image and the number of matched feature points. Step 4: Perform loop closure detection based on the acquired keyframes. The loop closure detection includes: retrieving preliminary candidate keyframes based on the bag-of-words model, and weighting and re-sorting the preliminary candidate keyframes based on their pose uncertainty and performing geometric verification based on deep learning. Step 5: Perform global optimization on the loopback confirmed by the loopback detection; Step 6: Based on the keyframes and their optimized poses, construct and update the local sub-map, wherein the sub-map adopts a neural implicit field representation; Step 7: The local sub-maps are fused to construct a global map. The fusion adopts a pixel-level sub-map fusion method based on uncertainty driving.

2. The system according to claim 1, characterized in that, The tracking module includes a YOLOv5 network model, a SuperPoint network model, and a LightGlue network model. It also includes an image frame pre-screening strategy and a candidate keyframe selection strategy.

3. The system according to claim 1, characterized in that, The loop closure detection module includes a DBoW3 bag-of-words model for retrieving loop closure candidate keyframes; the loop closure verification stage includes a feature matching calculation based on the LightGlue network.

4. The system according to claim 1, characterized in that, The dynamic local mapping module includes a neural implicit field and a dynamic sparse octree. The neural implicit field is used to predict occupancy and color values, and the dynamic sparse octree is used for spatial partitioning and storing multi-layer feature vectors.

5. The method according to claim 1, characterized in that, The deep learning network in step 2 is the SuperPoint network model, and the binary mask is generated by the YOLOv5 network model; the feature matching network for inter-frame feature matching in step 3 is the LightGlue network model.

6. The method according to claim 1, characterized in that, The weighted reordering method in step 4 follows the following formula: Wherein, S′(K c S is the weighted score. BoW (K c () represents the bag-of-words similarity score. Let be the trace of the covariance matrix of the candidate keyframe pose.

7. The method according to claim 1, characterized in that, The fusion method in step 7 involves weighted fusion of each pixel in the global map based on the transmission probability and inverse variance weights of candidate pixels from multiple sub-maps. The fusion method follows the formula below: Among them, Z i This represents the depth value of the i-th sub-map at that pixel location. This represents the depth uncertainty of the pixel, μ. i t represents the expected probability of occupancy. i This represents the transmission probability of the pixel, and ε is a stabilizing term to prevent the denominator from being too small.

Images