Large-scale monocular vision slam-gs method and system based on depth prior and subgraph management

Abstract

This application discloses a large-scale monocular visual SLAM-GS method based on depth prior and subgraph management, comprising: acquiring keyframes of a monocular video stream; acquiring an original depth prior map for the keyframes; aligning the original depth prior map with the current local map to obtain an aligned depth prior map; dynamically dividing the global map into multiple subgraphs and maintaining a bounded set of active subgraphs; when creating a new subgraph, initializing the map elements of the new subgraph using keyframes in the overlapping area with the old subgraph and their aligned depth prior maps; constructing and optimizing a 3D Gaussian map based on the aligned depth prior map within the set of active subgraphs; maintaining a global pose map with subgraphs as nodes, and correcting the global cumulative error by optimizing the relative poses between subgraphs. This invention achieves integrated fusion of monocular visual SLAM and 3D Gaussian modeling in large-scale, long-term scenes.

Description

Technical Field

[0001] This application relates to a large-scale monocular vision SLAM-GS method and system based on depth prior and subgraph management, belonging to the field of computer vision technology. Background Technology

[0002] Simultaneous Localization and Mapping (SLAM) is a core technology for achieving environmental perception and autonomous localization in fields such as robotics, AR / VR, and autonomous driving. Monocular vision SLAM is widely used due to its advantages such as lightweight equipment and low cost. However, traditional monocular SLAM suffers from problems such as scale ambiguity, slow initialization due to reliance on multi-view constraints for depth estimation, and poor robustness in weakly textured scenes. With the breakthrough of 3D Gaussian Splatting (3DGS) technology in efficient 3D reconstruction and real-time rendering, its integration with monocular SLAM has become an important direction for improving the accuracy and efficiency of dense mapping. However, existing monocular SLAM-GS methods still face bottlenecks in large-scale scenes, such as high global map memory consumption, large long-term cumulative error, inconsistency between depth prior and map scale, and lack of effective geometric guidance for Gaussian map initialization. It is difficult to balance mapping efficiency, localization accuracy, and system real-time performance in large-scale scenes. Therefore, it is urgent to introduce depth prior guidance and subgraph management mechanisms to achieve efficient, robust, and globally consistent monocular vision SLAM-GS localization and mapping in large-scale scenes. Summary of the Invention

[0003] According to one aspect of this application, a large-scale monocular vision SLAM-GS method based on depth prior and subgraph management is provided, which realizes the integrated fusion of monocular vision SLAM and 3D Gaussian modeling in large-scale, long-term scenes.

[0004] Large-scale monocular vision SLAM-GS methods based on depth priors and subgraph management include: Acquire keyframes from a monocular video stream; Obtain the original depth prior map for the keyframe; The original depth prior map is scale-aligned with the current local map to obtain the aligned depth prior map. The global map is dynamically divided into multiple subgraphs, and a set of active subgraphs with bounded size is maintained. Specifically, when creating a new subgraph, the map elements of the new subgraph are initialized using keyframes of the overlapping area with the old subgraph and their aligned depth prior map. Within the set of active subgraphs, a 3D Gaussian map is constructed and optimized based on the aligned depth prior map; Maintain a global pose graph with subgraphs as nodes, and correct the global cumulative error by optimizing the relative poses between subgraphs.

[0005] Furthermore, obtaining the original depth prior map includes: Call the pre-trained monocular depth estimation model based on the Transformer architecture to generate the original depth map; The scale alignment includes: Solve for the similarity transformation Sim3 parameters (s, R, t) to minimize the reprojection error between the transformed point cloud generated from the original depth prior map and the 3D points in the current local map that have at least 10 matching relationships, thus obtaining the aligned depth prior map.

[0006] Furthermore, the dynamic subgraph partitioning is triggered based on any of the following conditions: The number of keyframes in the currently active subgraph has reached the preset upper limit K; Alternatively, the Euclidean distance between the current camera optical center and the optical center of the anchor keyframe of the current active subgraph exceeds a preset threshold. .

[0007] Furthermore, the active subgraph set includes the currently optimized subgraph, the preceding subgraph, and the succeeding subgraph of the currently optimized subgraph; Maintaining the set of active subgraphs of the specified size includes: A subgraph state machine is established, which includes an active state, a resident state, and an archived state. The active state indicates that the subgraph is currently resident in the GPU memory and participating in optimization. The resident state indicates that the subgraph has been unloaded from the GPU memory, and its Gaussian parameters are compressed into half-precision floating-point numbers and stored in the host memory. The archived state indicates that the subgraph has been stored in an external storage device. When a subgraph is switched to an inactive state, it is unloaded from GPU memory and compressed for storage. When the camera predicts that it will revisit an archived subgraph region based on the global pose map, the subgraph is preloaded back into GPU memory and converted to an active state.

[0008] Furthermore, the map elements for initializing the new subgraph include: Select N keyframes from the tail of the old subgraph and the head of the new subgraph as an overlapping window; Using the aligned depth prior map of keyframes within the overlapping window and its optimized camera pose Tcw; A 3D point cloud is generated by backprojection and used as the initial seed for 3D Gaussian elements in a new subgraph. The backprojection involves converting pixel coordinates... and the corresponding depth value Convert to 3D points in the camera coordinate system, and then perform inverse camera pose transformation. Transform to the world coordinate system.

[0009] Furthermore, the construction and optimization of the 3D Gaussian map includes: The 3D Gaussian elements instantiated based on the aligned depth prior map include center position, covariance matrix, spherical harmonic coefficients, color, and opacity. The image is rendered using a differentiable rasterizer with the goal of minimizing the photometric error between the rendered image and the real keyframe image. Sliding window optimization is performed every time a preset number of keyframes are added, and the pose of the keyframes and the parameters of the 3D Gaussian elements within the window are adjusted together. The photometric error is a combination of L1 loss and the structural similarity index SSIM.

[0010] Furthermore, maintaining the global attitude graph includes: When the number of shared keyframes between two subgraphs reaches a preset value or visual loop closure is detected by the bag-of-words model, a keyframe based on relative transformation is added between the two subgraphs. The edge constraints are defined, and pose graph optimization is performed periodically to update the global coordinates of all 3D Gaussian elements within each subgraph.

[0011] Furthermore, it also includes: After subgraph merging or loop closure correction, redundant 3D Gaussian elements in the global map are removed. The elimination includes: Construct a FAISS approximate nearest neighbor search index for the center points of all 3D Gaussian elements in the global map; For the 3D Gaussian elements to be integrated Query its most recent existing 3D Gaussian elements ; If the center distance is less than the preset physical threshold =0.1 meters, then it is considered redundant, and will be and Merged into new 3D Gaussian elements Its parameter is a weighted average of the two based on opacity or the number of observations.

[0012] According to another aspect of this application, a large-scale monocular vision SLAM-GS system based on depth prior and subgraph management is also provided, comprising: The front-end tracking module is used to acquire keyframes from the monocular video stream and perform initial pose estimation; The depth prior acquisition module is used to acquire the original depth prior map for the key frame; The depth alignment module is used to scale-align the original depth prior map with the current local map and output the aligned depth prior map. The subgraph management module is used to dynamically divide the global map into multiple subgraphs, maintain a bounded set of active subgraphs, and implement streaming scheduling of subgraphs between GPU memory, host memory, and external storage. The subgraph initialization module is used to initialize the map elements of the new subgraph when it is created, using the keyframes of the overlapping area with the old subgraph and its aligned depth prior map. The local mapping and optimization module is used to construct and optimize a 3D Gaussian map based on the aligned depth prior map within the set of active subgraphs. The global optimization module is used to maintain a global pose graph with subgraphs as nodes, and corrects the global cumulative error by optimizing the relative poses between subgraphs.

[0013] Furthermore, it also includes a redundancy removal module, which is used to perform nearest neighbor search through the FAISS index after subgraph merging or loop closure correction to identify and merge redundant 3D Gaussian elements with a spatial distance of less than 0.1 meters.

[0014] The beneficial effects that this application can produce include: The large-scale monocular vision SLAM-GS method and system provided in this application, based on depth prior and subgraph management, effectively solves the inherent scale ambiguity problem of monocular vision SLAM by introducing depth prior and aligning it with the local map, significantly improving the accuracy of 3D reconstruction and the robustness of the system. The adoption of a global map dynamic subgraph partitioning and bounded active subgraph set management strategy effectively controls computational load and memory consumption in large-scale scenes, ensuring system real-time performance and scalability. New subgraphs are initialized using keyframes in overlapping regions and aligned depth priors, greatly improving the efficiency and consistency of subgraph construction. The construction and optimization of 3D Gaussian maps based on active subgraphs balances the fineness of scene representation with optimization efficiency. Simultaneously, through global pose graph optimization with subgraphs as nodes, the relative poses between subgraphs can be continuously corrected, suppressing global cumulative errors and achieving integrated fusion of monocular vision SLAM and 3D Gaussian modeling in large-scale, long-term scenes. Attached Figure Description

[0015] Figure 1 This is a diagram showing the overall architecture and data flow of a large-scale monocular vision SLAM-GS method and system based on depth prior and subgraph management in one embodiment of this application. Detailed Implementation

[0016] The present application is described in detail below with reference to the embodiments, but the present application is not limited to these embodiments.

[0017] See Figure 1 A large-scale monocular vision SLAM-GS method based on depth prior and subgraph management includes the following steps: Acquire keyframes from a monocular video stream; During system initialization, the weights of the pre-trained Dust3R monocular depth estimation model based on the Transformer architecture are loaded, the first sub-image S0 is initialized, its anchor point is set as the first keyframe, and it is set to active state. Parameters are set as follows: maximum number of keyframes K=100, overlap window size N=12, Euclidean distance threshold 50, physical threshold 0.1 meters. In real-time operation, for the t-th frame, sparse optical flow is used to track the previous keyframe; if the number of tracking points is less than 50 or the time since the last keyframe exceeds 1 second, the current frame is set as the new keyframe.

[0018] Obtain the original depth prior map for the keyframe; For each image identified as a keyframe, the pre-trained monocular depth estimation model Dust3R based on the Transformer architecture is invoked to generate the original depth map as the original depth prior map.

[0019] The original depth prior map is scaled with the current local map to obtain the aligned depth prior map. Retrieve a set of points P from the active subgraph that has at least 15 2D-3D matching points with the current keyframe. Use the RANSAC framework to solve for the similarity transformation Sim3 parameters (s, R, t) to minimize the reprojection error between the transformed point cloud generated from the original depth prior map and the 3D points in the current local map that have at least 10 matching relationships, thus obtaining the aligned depth prior map.

[0020] The global map is dynamically divided into multiple subgraphs, and a bounded set of active subgraphs is maintained. Subgraph partitioning is triggered when any of the following conditions are met: the number of keyframes in the current active subgraph reaches the preset limit K=100 frames; or the Euclidean distance between the current camera optical center and the anchor keyframe optical center of the current active subgraph exceeds the preset threshold of 50.

[0021] The active subgraph set consists of the subgraph currently being optimized, its predecessor subgraph, and its successor subgraph. Maintaining the set of active subgraphs of the specified size includes: A subgraph state machine is established, which includes an active state, a resident state, and an archived state. The active state indicates that the subgraph is currently resident in the GPU memory and participating in optimization. The resident state indicates that the subgraph has been unloaded from the GPU memory, and its Gaussian parameters are compressed into half-precision floating-point numbers and stored in the host memory. The archived state indicates that the subgraph has been stored in an external storage device. When a subgraph is switched to an inactive state, it is unloaded from GPU memory and compressed for storage. When the camera predicts that it will revisit an archived subgraph region based on the global pose map, the subgraph is preloaded back into GPU memory and converted to an active state.

[0022] When creating a new subgraph, the map elements of the new subgraph are initialized using keyframes of the overlapping area with the old subgraph and their aligned depth prior map. N=12 keyframes from the tail of the old sub-image and the head of the new sub-image are selected as the overlapping window; the depth prior map after alignment of the keyframes within the overlapping window and its optimized camera pose are used. ; Generate a 3D point cloud through back projection, and use this 3D point cloud as the initialization seed for 3D Gaussian elements in the new subgraph.

[0023] Within the active subgraph set, a 3D Gaussian map is constructed and optimized based on the aligned depth prior map; 3D Gaussian elements are instantiated based on the aligned depth prior map. These 3D Gaussian elements include center position, covariance matrix, spherical harmonic coefficients, color, and opacity. The image is rendered using a differentiable rasterizer with the goal of minimizing the photometric error between the rendered image and the real keyframe image. Sliding window optimization is performed every 5 new keyframes to jointly adjust the pose of the keyframes and the parameters of the Gaussian elements within the window. The photometric error is calculated using a combination of L1 loss and the structural similarity index SSIM.

[0024] Maintain a global pose graph with subgraphs as nodes, and correct the global cumulative error by optimizing the relative poses between subgraphs; When two subgraphs share at least 3 co-visual keyframes or visual loop closure is detected by bag-of-words model, add edge constraints based on relative transformation between the two subgraphs and periodically perform pose graph optimization to update the global coordinates of all Gaussian elements in each subgraph.

[0025] After subgraph merging or loop closure correction, redundant 3D Gaussian elements in the global map are removed. The elimination process includes: constructing a FAISS approximate nearest neighbor search index for the center points of all 3D Gaussians in the global map; for the Gaussian element to be integrated, querying its nearest k=3 existing Gaussian elements; if the center distance is less than the preset physical threshold of 0.1 meters, it is determined to be redundant, and the redundant Gaussian elements are merged into a new Gaussian, whose parameter is a weighted average of the two based on opacity or the number of observations.

[0026] See Figure 1Input & Front-End are the input and front-end; RGB Input is the image input; Key Frame Selection (RAFT) is the keyframe selection (based on the RAFT algorithm); Selected KFs are the filtered keyframes; DROID Front-End (Tracking) is the front-end (pose tracking); VGGT .Grid-Based scale Align is grid-based scale alignment; Submap Creation & 3DGS are submap creation and 3D Gaussian sputtering mapping; SubmapCreation Strategy is the submap creation strategy; KF Count>= K is the number of keyframes ≥ K; Dist(Current,Last)>Thresh is the distance between the current frame and the previous frame > threshold; Trigger New Submap is to trigger a new submap; SubmapInitialization (Overlap N=12) is submap initialization (overlapping window N=12); 3DGS Mapping is 3D Gaussian sputtering mapping; L_photo + L_depth are color loss and depth loss, Rendered Image I represents the rendered image I; Keyframe I represents the keyframe I; Depth Loss represents the depth loss; Color Loss represents the color loss; Merge & Global Optimization represents fusion and global optimization; Submap Merge represents submap fusion; NN Search (FAISS) represents nearest neighbor search (based on the FAISS library); Duplicate Pruning represents redundancy / duplication pruning; Global Map (Consistent & Pruned) represents the global map (consistent and pruned).

[0027] In this application, for each selected keyframe, a pre-trained monocular depth estimation model based on the Transformer architecture, such as Dust3R, is invoked to obtain the original predicted depth map. Design a lightweight depth-scale alignment module that, based on the current SLAM optimization state, specifically, identifies at least 10 matching 3D Gaussian center points in the currently active subgraph that correspond to the current keyframe. and the 2D projection of these points in the current frame. Solve for a similarity transformation Sim3. By minimizing the reprojection error or directly aligning the 3D point cloud, calculate a scale factor s, rotation R, and translation t such that the transformed predicted depth point cloud... The local map point P of the SLAM system is aligned in the metric space. Finally, this transformation is applied to obtain the aligned depth prior map. This serves as a strong constraint for subsequent geometric initialization.

[0028] In bounded memory subgraph management, the global map is divided into multiple temporally or spatially contiguous subgraphs. Two specific trigger conditions are set, and a new subgraph is created when either condition is met: a) the number of keyframes in the currently active subgraph reaches a preset upper limit K, for example, K=100 frames; b) the Euclidean distance between the current camera optical center and the anchor keyframe optical center of the currently active subgraph exceeds a threshold. ,For example The system maintains a set of active subgraphs, typically containing only one currently optimized subgraph, along with its predecessor and successor subgraphs, for smooth transitions; new subgraphs are created. Instead of starting from zero, an overlapping window is selected, consisting of the N most recent keyframes, preferably N=12, which belong to the old subgraph. The tail of the graph and the head of the new subgraph; using the aligned depth prior of these keyframes. and its optimized camera pose A set of 3D point clouds is generated through back projection. ,in, This is the back projection function. This set of 3D point clouds... Directly used as a new subgraph The initialization seed for 3D Gaussian sprayed elements inherits aligned, scale-consistent geometric information in terms of scale and position, ensuring continuity at subgraph boundaries. The system maintains a subgraph state machine, including active, resident, and archived states. Inactive subgraphs, i.e., those not immediately optimized or rendered, are unloaded from GPU memory: their Gaussian parameters are compressed and transferred to host (CPU) memory or SSD. When camera motion determines, based on the global pose map, that it is about to re-enter a region of an archived subgraph, the subgraph is preloaded back into GPU memory and converted to an active or resident state, enabling on-demand streaming access. Within active subgraphs, an aligned depth prior map is used. Initialized point cloud Instantiate 3D Gaussian elements, each containing a center position, covariance matrix, spherical harmonic coefficients, color, and opacity. Render a new perspective image using a differentiable rasterizer and perform a sliding window optimization at a fixed frequency. By minimizing the photometric error between the rendered image and the real keyframe image, jointly optimize the camera pose of all keyframes within the window and the parameters of all Gaussian elements within the sub-image.

[0029] In maintaining global consistency, a subgraph is established. This is the global pose graph of a node, and the node attribute is the Sim3 transformation of this subgraph to the world coordinate system, denoted as . An edge is added between two subgraphs when they share at least three co-view keyframes, or when visual loop closure is detected via a bag-of-words model. The constraint value of the edge is the relative transformation calculated based on the co-view geometry. Regularly run pose graph optimization in the background to minimize transformation errors of all edges. After optimization, synchronously update the global coordinates of all Gaussian elements within each subgraph. Perform duplicate Gaussian elimination based on efficient indexing: after merging the subgraph into the global map or performing loop closure correction, perform redundancy removal. First, determine the center point of all Gaussians in the global map. Build a FAISS (Facebook AI Similarity Search) index that supports near nearest neighbor queries for large-scale point clouds in milliseconds. For each 3D Gaussian element to be integrated... (Assuming its center is q), query its k nearest existing 3D Gaussian elements using FAISS. .if With the nearest neighbor The center distance is less than a physically meaningful threshold. Then determine This is redundant. At this point, and Merging, new 3D Gaussian elements The parameter is a weighted average of the two, and the weights can be determined based on their opacity or the number of observations. This process can effectively control the infinite expansion of the map.

[0030] Through the above steps, this method can effectively solve the scale drift problem in large-scale monocular visual SLAM, improve the accuracy and completeness of map construction, and achieve efficient memory usage and large scene modeling capabilities through subgraph management mechanism.

[0031] Table 1 shows a comparison of GPU memory usage of this invention and other methods on long sequences, including the runtime, peak GPU memory usage, and frame count of the NPUFly sequence. Runtime is reported in seconds, and memory usage in GB; lower values ​​are better. Here, Method represents the method, Frames represents the number of frames, and Time... Mem represents the time consumed (unit: seconds / frame or total time, the smaller the value, the better). The values ​​represent memory usage (unit: GB, lower is better). Speed ​​comparison: Hi-SLAM2 is the fastest among all sequences, followed by this application, while MonoGS is the slowest, taking several times longer than the other two. Memory comparison: This application has the lowest memory usage among all sequences, significantly outperforming Hi-SLAM2 and MonoGS, demonstrating better memory efficiency. It's worth noting that for those seeking extreme speed, Hi-SLAM2 is the optimal choice; for those seeking a balance between memory efficiency and performance, this application performs better, leading in memory usage across the board, with speed approaching Hi-SLAM2 and far superior to MonoGS. MonoGS is at a disadvantage in both time and memory, making it unsuitable for large sequence scenarios.

[0032] Table 1

[0033] Table 2 shows a comparison of the reconstruction results of the submap-based method and the baseline method in this application. Higher PSNR / SSIM and lower LPIPS are preferred. Here, Algorithm refers to the algorithm, Metric to the evaluation metric, Average to the mean value, PSNR to Peak Signal-to-Noise Ratio, SSIM to Structural Similarity Index Measure, LPIPS to Learned Perceptual ImagePatch Similarity, Photo-SLAM to Photo-SLAM algorithm, MonoGS to MonoGS algorithm, Hi-SLAM2 to Hi-SLAM2 algorithm, and Ours (submap) to the submap method of this application. It is worth noting that this application achieves the best average values ​​for PSNR, SSIM, and LPIPS, resulting in the best reconstruction quality among the four methods. This application also demonstrates stable performance across all sequences, without the significant performance drop seen in Hi-SLAM2 on 300-map sequences. Hi-SLAM2 performs well in some sequences, but its robustness is insufficient; MonoGS has moderate overall performance, lagging behind this application and Hi-SLAM2; Photo-SLAM is the worst in all metrics, with the worst reconstruction quality. This application achieves higher PSNR / SSIM and lower LPIPS, and performs more evenly across different sequences, demonstrating the advantages of the subgraph strategy in 3DGS-SLAM.

[0034] Table 2

[0035] Example 2 A large-scale monocular vision SLAM-GS system based on depth prior and subgraph management includes: a front-end tracking module, a depth prior acquisition module, a depth alignment module, a subgraph management module, a subgraph initialization module, a local mapping and optimization module, and a global optimization module.

[0036] The front-end tracking module acquires keyframes from the monocular video stream and performs initial pose estimation. This module uses the sparse optical flow method to track the current frame and the previous keyframe. When the number of tracking points is less than 50 or the time since the last keyframe exceeds 1 second, the system sets the current frame as the new keyframe. The front-end tracking module calculates the initial pose of the keyframe through tracking, providing a basis for subsequent processing.

[0037] The depth prior acquisition module obtains the original depth prior map for keyframes. This module loads the weights of the pre-trained Dust3R depth estimation model, processes each image labeled as a keyframe, and outputs the original depth prior map. It does not rely on the constructed map and can independently provide an initial depth estimate for each keyframe, effectively solving the problem of scale uncertainty in monocular SLAM. The depth alignment module scales the original depth prior map with the current local map, outputting an aligned depth prior map. Specifically, this module retrieves a set of points from the active submap that have at least 15 2D-3D matches with the current keyframe. Then, it uses the RANSAC framework to solve the Sim3 transformation, including the scale factor s, rotation matrix R, and translation vector t, aligning the original depth prior map with the retrieved 3D point sets. Applying this transformation yields the aligned depth prior map, ensuring that the depth information maintains a consistent scale with the current map. The subgraph management module dynamically divides the global map into multiple subgraphs, maintains a bounded set of active subgraphs, and implements streaming scheduling of subgraphs between GPU memory, host memory, and external storage. This module sets parameters K=100 and N=12, where K represents the maximum number of keyframes each subgraph can contain, and N represents the upper limit of the number of active subgraphs the system can maintain simultaneously. The subgraph management module dynamically creates new subgraphs based on the spatial distribution and temporal relationships of keyframes. When the number of keyframes in a subgraph reaches K, the system creates a new subgraph. Simultaneously, this module maintains a list of active subgraphs. When the number of active subgraphs exceeds N, the least active subgraph is unloaded from GPU memory to host memory or external storage based on the most recently used principle to ensure efficient system memory usage. When a new subgraph is created, the subgraph initialization module uses keyframes from the overlapping area of ​​the old subgraph and their aligned depth priors to initialize the map elements of the new subgraph. The system initializes the first subgraph S0, sets its anchor point to the first keyframe, and sets it to active status. When a new subgraph needs to be created, this module extracts depth information and feature points from the keyframes overlapping with the old subgraph, transforms this information into the coordinate system of the new subgraph, and uses it as the initial map elements of the new subgraph, ensuring continuity and consistency between subgraphs. The local mapping and optimization module constructs and optimizes a 3D Gaussian map based on an aligned depth prior map within the set of active submaps. This module receives the aligned depth prior map, converts the depth information into a 3D Gaussian point cloud, and performs local optimization within the current active submap. The optimization process includes density adjustment, position optimization, and appearance parameter optimization of the Gaussian point cloud, improving the map's accuracy and integrity by minimizing reprojection errors and depth consistency errors. The global optimization module maintains a global pose graph with subgraphs as nodes, and corrects the global cumulative error by optimizing the relative poses between subgraphs. This module treats each subgraph as a rigid body node, and overlapping areas between subgraphs form constraint edges. When the system detects a loop closure or requires global optimization, this module reduces the cumulative error of the entire map by optimizing the relative pose relationships between subgraphs, ensuring global consistency of the map in large-scale scenarios.

[0038] The system also includes a redundancy removal module, which, after subgraph merging or loop closure correction, uses the FAISS index to perform nearest neighbor search, identifying and merging redundant 3D Gaussian elements with a spatial distance of less than 0.1 meters. After subgraph merging or loop closure correction, multiple 3D Gaussian elements representing the same entity may exist on the map. The redundancy removal module first constructs a FAISS index to index the spatial locations of all 3D Gaussian elements, then performs a nearest neighbor search to find pairs of Gaussian elements with a spatial distance of less than 0.1 meters. For each pair of redundant elements, the system selects to retain the higher-quality one based on indicators such as the number of observations and uncertainty, or merges the two into a new Gaussian element, thereby reducing map redundancy and improving map quality and system efficiency.

[0039] This system effectively solves the problems of scale uncertainty and computational resource limitations in large-scale scenes of monocular visual SLAM by using depth prior information and efficient subgraph management strategies, and achieves high-precision and high-efficiency large-scale environmental 3D reconstruction and localization functions.

[0040] The above description is merely a few embodiments of this application and is not intended to limit this application in any way. Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any changes or modifications made by those skilled in the art without departing from the scope of the technical solution of this application using the disclosed technical content are equivalent to equivalent implementation cases and fall within the scope of the technical solution.

Claims

1. A large-scale monocular visual SLAM-GS method based on depth prior and subgraph management, characterized in that, include: Acquire keyframes from a monocular video stream; Obtain the original depth prior map for the keyframe; The original depth prior map is scale-aligned with the current local map to obtain the aligned depth prior map. The global map is dynamically divided into multiple sub-maps, and a set of active sub-maps with bounded size is maintained. When creating a new sub-map, the map elements of the new sub-map are initialized using keyframes of the overlapping area with the old sub-map and their aligned depth prior map. Within the set of active subgraphs, a 3D Gaussian map is constructed and optimized based on the aligned depth prior map; Maintain a global pose graph with subgraphs as nodes, and correct the global cumulative error by optimizing the relative poses between subgraphs.

2. The large-scale monocular vision SLAM-GS method based on depth prior and subgraph management according to claim 1, characterized in that, The process of obtaining the original depth prior map includes: Call the pre-trained monocular depth estimation model based on the Transformer architecture to generate the original depth map; The scale alignment includes: Solve for the similarity transformation Sim3 parameters (s, R, t) to minimize the reprojection error between the transformed point cloud generated from the original depth prior map and the 3D points in the current local map that have at least 10 matching relationships, thus obtaining the aligned depth prior map.

3. The large-scale monocular vision SLAM-GS method based on depth prior and subgraph management according to claim 1, characterized in that, The dynamic subgraph partitioning is triggered based on any of the following conditions: The number of keyframes in the currently active subgraph has reached the preset upper limit K; Alternatively, the Euclidean distance between the current camera optical center and the optical center of the anchor keyframe of the current active subgraph exceeds a preset threshold. .

4. The large-scale monocular vision SLAM-GS method based on depth prior and subgraph management according to claim 1, characterized in that, The active subgraph set consists of the subgraph currently being optimized, its predecessor subgraph, and its successor subgraph. Maintaining the set of active subgraphs of the specified size includes: A subgraph state machine is established, which includes an active state, a resident state, and an archived state. The active state indicates that the subgraph is currently resident in the GPU memory and participating in optimization. The resident state indicates that the subgraph has been unloaded from the GPU memory, and its Gaussian parameters are compressed into half-precision floating-point numbers and stored in the host memory. The archived state indicates that the subgraph has been stored in an external storage device. When a subgraph is switched to an inactive state, it is unloaded from GPU memory and compressed for storage. When the camera predicts that it will revisit an archived subgraph region based on the global pose map, the subgraph is preloaded back into GPU memory and converted to an active state.

5. The large-scale monocular vision SLAM-GS method based on depth prior and subgraph management according to claim 1, characterized in that, The map elements for initializing the new subgraph include: Select N keyframes from the tail of the old subgraph and the head of the new subgraph as an overlapping window; Using the aligned depth prior map of keyframes within the overlapping window and its optimized camera pose Tcw; A 3D point cloud is generated by backprojection and used as the initial seed for 3D Gaussian elements in a new subgraph. The backprojection involves converting pixel coordinates... and the corresponding depth value Convert to 3D points in the camera coordinate system, and then perform inverse camera pose transformation. Transform to the world coordinate system.

6. The large-scale monocular vision SLAM-GS method based on depth prior and subgraph management according to claim 1, characterized in that, The construction and optimization of the 3D Gaussian map includes: The 3D Gaussian elements instantiated based on the aligned depth prior map include center position, covariance matrix, spherical harmonic coefficients, color, and opacity. The image is rendered using a differentiable rasterizer with the goal of minimizing the photometric error between the rendered image and the real keyframe image. Sliding window optimization is performed every time a preset number of keyframes are added, and the pose of the keyframes and the parameters of the 3D Gaussian elements within the window are adjusted together. The photometric error is a combination of L1 loss and the structural similarity index SSIM.

7. The large-scale monocular vision SLAM-GS method based on depth prior and subgraph management according to claim 1, characterized in that, The maintenance of the global attitude graph includes: When the number of shared keyframes between two subgraphs reaches a preset value or visual loop closure is detected by the bag-of-words model, a keyframe based on relative transformation is added between the two subgraphs. The edge constraints are defined, and pose graph optimization is performed periodically to update the global coordinates of all 3D Gaussian elements within each subgraph.

8. The large-scale monocular vision SLAM-GS method based on depth prior and subgraph management according to claim 1, characterized in that, Also includes: After subgraph merging or loop closure correction, redundant 3D Gaussian elements in the global map are removed. The elimination includes: Construct a FAISS approximate nearest neighbor search index for the center points of all 3D Gaussian elements in the global map; For the 3D Gaussian elements to be integrated Query its most recent existing 3D Gaussian elements ; If the center distance is less than the preset physical threshold =0.1 meters, then it is considered redundant, and will be and Merged into new 3D Gaussian elements Its parameter is a weighted average of the two based on opacity or the number of observations.

9. A large-scale monocular vision SLAM-GS system based on depth prior and subgraph management, characterized in that, include: The front-end tracking module is used to acquire keyframes from the monocular video stream and perform initial pose estimation; The depth prior acquisition module is used to acquire the original depth prior map for the key frame; The depth alignment module is used to scale-align the original depth prior map with the current local map and output the aligned depth prior map. The subgraph management module is used to dynamically divide the global map into multiple subgraphs, maintain a bounded set of active subgraphs, and implement streaming scheduling of subgraphs between GPU memory, host memory, and external storage. The subgraph initialization module is used to initialize the map elements of the new subgraph when it is created, using the keyframes of the overlapping area with the old subgraph and its aligned depth prior map. The local mapping and optimization module is used to construct and optimize a 3D Gaussian map based on the aligned depth prior map within the set of active subgraphs. The global optimization module is used to maintain a global pose graph with subgraphs as nodes, and corrects the global cumulative error by optimizing the relative poses between subgraphs.

10. The large-scale monocular vision SLAM-GS system based on depth prior and subgraph management according to claim 9, characterized in that, It also includes a redundancy removal module, which is used to perform nearest neighbor search through the FAISS index after subgraph merging or loop closure correction to identify and merge redundant 3D Gaussian elements with a spatial distance of less than 0.1 meters.

Images