A Large-Scale Outdoor Monocular SLAM Method Based on Hybrid Association and Multi-Scale Gaussian Optimization

By introducing a pre-trained point-map model and a multi-scale optimization strategy, the tracking stability and scale drift problems of monocular SLAM in complex outdoor environments were solved, achieving high-precision 3D scene reconstruction and localization.

CN123118942APending Publication Date: 2026-11-17WUHAN UNIV OF SCI & TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611340148.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-01
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing monocular SLAM methods based on 3DGS suffer from poor tracking stability, severe scale drift, and susceptibility of 3D maps to noise in complex outdoor environments.

Method used

We adopt 3D Gaussian splashing as the scene representation method, introduce a pre-trained Point-map model to generate geometric prior information, combine sparse geometric constraints and dense photometric constraints to construct a hybrid correlation tracking mechanism, and optimize the weights of Gaussian units through multi-scale optimization and local reliability weighting strategies.

Benefits of technology

It improves the stability and positioning accuracy of camera pose estimation in complex outdoor environments, reduces scale drift, enhances 3D reconstruction quality and map consistency, and reduces the impact of noise on map optimization.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118942A_ABST
    Figure CN123118942A_ABST
Patent Text Reader

Abstract

This invention discloses a large-scale outdoor monocular SLAM method based on hybrid correlation and multi-scale Gaussian optimization, belonging to the fields of computer vision, robot autonomous localization, and 3D scene reconstruction. The method utilizes a monocular camera to acquire continuous RGB image sequences, predicts the 3D geometric information of the scene through a pre-trained Point-map model, and initializes a 3D Gaussian scene model. It constructs a hybrid correlation pose tracking model that integrates ORB feature geometric constraints, Point-map geometric priors, and 3D Gaussian photometric consistency constraints to achieve camera pose estimation. A keyframe management mechanism is established, and a multi-scale consistency optimization model and a local reliability weighted optimization model are combined to jointly optimize the 3D Gaussian map. Finally, the camera motion trajectory and the optimized 3D Gaussian map are output, and new perspective image rendering and 3D scene reconstruction are completed. This invention can improve the localization accuracy, tracking stability, map scale consistency, and 3D scene reconstruction quality of monocular SLAM systems in complex outdoor environments.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of computer vision, Simultaneous Localization and Mapping (SLAM), and 3D scene reconstruction. Specifically, it relates to a large-scale outdoor monocular SLAM method based on hybrid correlation tracking, multi-scale Gaussian optimization, and a local reliability weighting strategy. This method uses 3D Gaussian Splatting (3DGS) as the scene representation model, introduces pre-trained point-map geometric prior information, and achieves high-precision camera localization and high-fidelity 3D scene reconstruction in complex outdoor environments by fusing sparse geometric constraints and dense photometric constraints. It can be widely applied in fields such as autonomous driving, mobile robots, UAV mapping, digital twins, and augmented reality. Background Technology

[0002] Simultaneous Localization and Mapping (SLAM) is a fundamental technology in fields such as robot navigation, autonomous driving, and augmented reality. Its main task is to simultaneously perform camera pose estimation and environmental map construction using sensor information in unknown environments. Due to the advantages of monocular vision systems, such as low hardware cost, lightweight equipment, and flexible deployment, monocular vision SLAM has become a research hotspot both domestically and internationally in recent years.

[0003] Currently, traditional monocular SLAM mainly includes feature-based methods and direct-method methods. Feature-based methods, represented by the ORB-SLAM series, build sparse maps by extracting image feature points, achieving high localization accuracy, but their map representation capabilities are limited, making it difficult to achieve high-quality 3D scene reconstruction. Direct-method SLAM completes pose estimation by minimizing image photometric errors, obtaining relatively dense environmental information, but it is sensitive to changes in illumination, rapid movement, and low-texture areas, and is prone to tracking failures in complex outdoor environments.

[0004] In recent years, the development of neural implicit representation and 3D Gaussian Splatting (3DGS) technologies has provided new solutions for high-quality scene reconstruction. 3DGS utilizes a large number of learnable 3D Gaussian primitives to represent scene geometry and texture, offering advantages over NeRF in terms of faster rendering speed, higher training efficiency, and better real-time performance, thus gradually being applied to visual SLAM systems. However, existing 3DGS-based monocular SLAM methods still have the following shortcomings: First, pose estimation mainly relies on image photometric consistency, which can easily lead to local optima when the camera undergoes rapid movement, large field of view rotation, or changes in illumination, resulting in decreased tracking stability; second, monocular vision inherently suffers from scale blur, which can easily cause scale drift during long-distance mapping, affecting the mapping process. Figure 1Third, in complex outdoor environments, there are a large number of noise observation and depth prediction errors. During the optimization of Gaussian parameters, errors are easily accumulated, causing geometric distortion and a decrease in rendering quality in the reconstructed model.

[0005] Therefore, a novel monocular SLAM method is needed that can fully integrate geometric prior information, take into account both sparse geometric constraints and dense photometric constraints, and combine multi-scale optimization and local reliability weighting mechanisms to improve positioning accuracy, scale consistency and 3D reconstruction quality in complex outdoor environments. Summary of the Invention

[0006] To address the problems of poor tracking stability, severe scale drift, and susceptibility of 3D Gaussian maps to noise in existing monocular SLAM systems in large-scale outdoor environments, this invention proposes a large-scale outdoor monocular SLAM method based on hybrid correlation and multi-scale Gaussian optimization.

[0007] This invention employs 3D Gaussian splashing as a unified scene representation method, introduces a pre-trained Point-map model to generate geometric prior information, and establishes a hybrid correlation tracking mechanism that integrates sparse geometric constraints and dense photometric constraints to improve the stability of camera pose estimation in complex environments. Simultaneously, by constructing a multi-scale pyramid optimization model, geometric priors and Gaussian maps are jointly optimized at different spatial scales, effectively reducing monocular SLAM scale drift. Furthermore, an adaptive weight model is constructed based on local depth consistency and geometric reliability to perform local reliability weighted optimization of Gaussian primitives, reducing the impact of noisy observations on map updates and improving the accuracy of 3D scene reconstruction and the rendering quality of new perspectives.

[0008] The specific content of the invention is as follows:

[0009] 1. A large-scale outdoor monocular SLAM method based on hybrid correlation and multi-scale Gaussian optimization, comprising the following steps:

[0010] Step 1) Construct an initial 3D Gaussian scene model. Continuously acquire outdoor environment RGB image sequences using a monocular camera, and perform camera calibration, distortion correction, and scale unification on the input images. Then, input the pre-processed images into a pre-trained Point-map model to predict the corresponding 3D point map geometric information, obtaining the scene's geometric prior.

[0011] The 3D Gaussian scene is initialized based on the aforementioned geometric priors, and the scene's geometric structure is represented by multiple 3D Gaussian primitives. Each Gaussian primitive is represented as follows:

[0012]

[0013] in, Indicates the first The spatial center coordinates of a Gaussian element This represents the three-dimensional covariance matrix, used to describe the spatial scale and orientation of Gaussian elements. Indicates color parameters, This represents the transparency parameter. All Gaussian elements together constitute a 3D Gaussian map of the scene, providing an explicit scene representation for subsequent camera tracking and map optimization.

[0014] Step 2) Establish a hybrid associated pose tracking model. For the current input image, first extract ORB feature points and perform feature matching with reference keyframes. Combine the geometric priors provided by Point-map to recover the corresponding three-dimensional spatial points and establish the geometric correspondence between two-dimensional image points and three-dimensional spatial points.

[0015] Meanwhile, the current Gaussian map is used to generate a predicted image from the current viewpoint using a differentiable Gaussian renderer, and the photometric consistency error between the predicted image and the real image is calculated.

[0016] To fully integrate geometric and photometric information, this invention constructs a joint pose optimization objective function:

[0017]

[0018] in, Indicates the current camera pose; Indicates geometric reprojection error; Indicates photometric consistency error; and These represent the weight coefficients of the corresponding constraint terms.

[0019] The geometric reprojection error is expressed as:

[0020]

[0021] In the formula, Indicates the first A three-dimensional spatial point; Represents the corresponding image feature points; This represents the camera projection function.

[0022] The objective function is optimized using the Lie algebra Gauss-Newton iterative method to obtain the camera pose in the current frame, thereby improving the stability and positioning accuracy of camera tracking in complex outdoor environments.

[0023] Step 3) Establish a keyframe management mechanism based on hybrid co-viewability. Based on the ORB feature co-viewability relationship, Gaussian co-viewability relationship and camera motion amplitude between the current frame and existing keyframes, jointly determine whether the current frame should be added to the map as a new keyframe.

[0024] The Gaussian co-view rate is defined as:

[0025]

[0026] in, and These represent keyframes. and keyframes The observable set of Gaussian elements.

[0027] When the current frame meets the motion distance threshold, ORB co-view rate threshold, and Gaussian co-view rate threshold conditions, the current frame is inserted into the keyframe set; when the number of keyframes in the sliding window exceeds the preset upper limit, redundant keyframes are deleted based on the comprehensive contribution of keyframes to reduce computational complexity and maintain local map optimization efficiency.

[0028] Step 4) Construct a multi-scale consistency optimization model. In order to reduce the scale drift generated by the monocular SLAM system during long-distance motion, this invention constructs a multi-layer image pyramid and performs Gaussian differentiable rendering and photometric consistency optimization at each scale.

[0029] Let the image pyramid have a total of For layers, the overall multi-scale photometric uniformity loss is defined as:

[0030]

[0031] in, Indicates the first Optical consistency error of layer images; This represents the corresponding scale weight, with coarse scales assigned a larger weight to ensure the stability of the global geometric structure.

[0032] Simultaneously, point-map geometric priors are introduced to construct scale consistency constraints:

[0033]

[0034] in, This represents the 3D points predicted by the Point-map. Indicates the corresponding Gaussian center; This represents the local scale factor.

[0035] By jointly optimizing multi-scale photometric consistency constraints and scale consistency constraints, a hierarchical optimization process from coarse to fine is achieved, thereby improving the overall scale consistency of the map.

[0036] Step 5) Establish a local reliability weighted map optimization model. Considering that the pre-trained geometric prior has local prediction errors in complex outdoor environments, this invention establishes a local reliability evaluation mechanism based on depth consistency and photometric consistency, and adopts different optimization weights for different regions.

[0037] The local reliability weight of the Gaussian element is defined as:

[0038]

[0039] in, This represents the error between the Point-map predicted depth and the Gaussian rendered depth. This represents the photometric residual in the corresponding region; and This indicates the adjustment parameter.

[0040] The final map optimization objective function is expressed as:

[0041]

[0042] The location, covariance, color, and transparency parameters of Gaussian elements are iteratively updated according to the objective function, so that reliable regions can contribute more to the map optimization process, while noisy regions are automatically reduced in optimization weight, thereby improving the geometric accuracy of the map and the quality of texture reconstruction.

[0043] Step 6) Repeat steps 1 to 5 until the current sequence processing ends or the system reaches the convergence condition, and output the final optimized camera motion trajectory and 3D Gaussian map.

[0044] Using an optimized 3D Gaussian map, a differentiable Gaussian renderer is used to render new perspective images from any viewpoint, and outputs 3D scene reconstruction results with high geometric consistency and high texture fidelity.

[0045] The present invention has the following advantages and beneficial effects:

[0046] (1) This invention constructs a hybrid associated pose tracking model that integrates ORB feature geometric constraints, Point-map geometric priors and Gaussian photometric consistency constraints, thereby achieving joint optimization of geometric and photometric information. This effectively improves the stability and accuracy of camera pose estimation in complex outdoor environments and reduces tracking drift caused by rapid movement, lighting changes and low-texture scenes.

[0047] (2) By establishing a multi-scale pyramid consistency optimization model, this invention jointly optimizes the photometric information and geometric prior information of the image at different scales, realizing a hierarchical optimization process from coarse to fine, effectively reducing the scale accumulation error generated by the monocular SLAM system during long-distance movement, and improving the overall scale consistency and geometric accuracy of the map.

[0048] (3) By introducing a local reliability weighted optimization strategy, the present invention adaptively adjusts the weight of each Gaussian element in map optimization based on depth consistency and photometric consistency, thereby reducing the impact of noisy observations and unreliable geometric priors on map optimization and improving the optimization stability and scene reconstruction quality of the three-dimensional Gaussian map.

[0049] (4) The present invention uses three-dimensional Gaussian splashing as a scene representation method, and combines hybrid correlation tracking, multi-scale consistency optimization and local reliability weighted optimization to achieve coordinated optimization of camera positioning and map construction, thereby improving the geometric expression capability of three-dimensional scenes and the rendering quality of new perspective images.

[0050] (5) This invention does not rely on high-precision lidar or depth camera. It can complete high-precision positioning and 3D scene reconstruction using only monocular RGB images and pre-trained geometric priors. It has the advantages of high computational efficiency, wide applicability and high engineering application value. It can be widely used in fields such as autonomous driving, mobile robot autonomous navigation, UAV environmental perception, digital twins and augmented reality. Attached Figure Description

[0051] Figure 1 This is a complete logical flowchart of a large-scale outdoor monocular SLAM method based on hybrid correlation and multi-scale Gaussian optimization according to the present invention. Detailed Implementation

[0052] The advantages and objectives of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the description herein is only for explaining the present invention and is not intended to limit the present invention.

[0053] This invention provides a large-scale outdoor monocular SLAM method based on hybrid correlation and multi-scale Gaussian optimization, the specific implementation process of which is shown in the appendix. Figure 1 As shown, it includes the following steps:

[0054] Step 1) After the system starts, it continuously acquires RGB image sequences of the outdoor environment using a monocular camera, and reads the camera's intrinsic parameters to complete image distortion correction and scale normalization processing, so that all input images have a unified imaging model. Subsequently, the preprocessed images are input into a pre-trained Point-map network to predict the 3D point map geometric information corresponding to the current image and obtain the 3D coordinates of each pixel.

[0055] A 3D Gaussian map is initialized based on the 3D coordinates predicted by the Point-map. Each spatial point corresponds to a 3D Gaussian element, which is expressed as follows:

[0056]

[0057] in, Initialize the 3D coordinates predicted by the Point-map; Initialize the Gaussian scale and orientation based on the distribution of points in the neighborhood space; The value is assigned by the corresponding pixel color; Initialize to preset transparency parameters. After initializing all Gaussian elements, an initial 3D Gaussian map of the scene is obtained, providing a scene representation for subsequent pose tracking and map optimization.

[0058] Step 2) After receiving the current input image, the system first extracts ORB feature points and corresponding descriptors, and establishes feature matching relationships with reference keyframes; it then establishes the correspondence between two-dimensional image points and three-dimensional spatial points by combining the three-dimensional points predicted by Point-map, and calculates the initial value of the current camera pose using the PnP algorithm.

[0059] After obtaining the initial pose, differentiable rendering is performed using the current 3D Gaussian map to generate a predicted image from the current viewpoint, and the photometric consistency error between the predicted and ground truth images is calculated. To simultaneously utilize geometric and photometric constraints, a joint optimization objective function is constructed:

[0060]

[0061] in, This represents the geometric reprojection error. This indicates the photometric consistency error. and These represent the weights of the two types of constraints, respectively.

[0062] Geometric reprojection error is calculated using the following formula:

[0063]

[0064] The system uses Lie algebras to parameterize the camera pose and employs the Gauss-Newton algorithm to iteratively optimize the objective function. When the error between two consecutive iterations is less than a preset threshold, the system outputs the optimal pose for the current frame, which is then used as the initial value for tracking in the next frame.

[0065] Step 3) After completing the localization of the current frame, the system determines whether the keyframe needs to be updated. If the viewpoint change between the current frame and the most recent keyframe is significant, or if many unmodeled areas appear in the local map, the current frame is added to the keyframe set. Simultaneously, the Gaussian co-view rate between the current keyframe and existing keyframes is calculated:

[0066]

[0067] in, and These represent keyframes. and keyframes The observable set of Gaussian elements.

[0068] When the common view rate is low, it means that the current view contains more new scene information, and the system will use the new observations to supplement the Gaussian map; when the number of keyframes in the sliding window exceeds the set limit, the keyframe with the lowest contribution will be deleted to maintain the efficiency of local map optimization.

[0069] Step 4) After the keyframe update, optimize the local 3D Gaussian map. First, construct a multi-layer image pyramid, perform Gaussian differentiable rendering at each scale, and calculate the photometric consistency error at the corresponding scale:

[0070]

[0071] in, Indicates the number of layers in the image pyramid. Indicates the first The photometric error corresponding to the layer image. This indicates the corresponding scale weight.

[0072] The system first optimizes the overall geometry using low-resolution images, and then gradually restores local details using high-resolution images. Simultaneously, it uses the 3D points predicted by the Point-map as a scale reference, participating in map optimization along with photometric consistency, ensuring the map maintains overall scale consistency throughout the optimization process.

[0073] Step 5) After completing multi-scale optimization, calculate the reliability weight of each Gaussian primitive based on the error between the Point-map predicted depth and the current Gaussian map rendering depth, as well as the photometric residual of the corresponding region:

[0074]

[0075] in, This indicates depth consistency error. This represents the photometric residual.

[0076] Construct a map optimization objective function based on the calculated reliability weights:

[0077]

[0078] Based on the aforementioned objective function, the system jointly updates the position, covariance matrix, color parameters, and transparency parameters of Gaussian elements. For areas with high reliability, their optimization weights are increased; for areas with large errors, the parameter update magnitude is reduced, thereby mitigating the impact of abnormal observations on the map optimization results.

[0079] Step 6) Repeat steps 2) through 5) until all image sequences have been processed or the system meets the convergence condition. Finally, the optimized camera motion trajectory and 3D Gaussian map are output, and the optimized 3D Gaussian map is used to perform image differentiability rendering from any viewpoint to obtain the final 3D scene reconstruction result.

Claims

1. A large-scale outdoor monocular SLAM method based on hybrid correlation and multi-scale Gaussian optimization, comprising the following steps: Step 1) Construct an initial 3D Gaussian scene model; continuously acquire outdoor environment RGB image sequences using a monocular camera, perform camera calibration, distortion correction, and scale unification processing on the input images; input the pre-processed images into a pre-trained Point-map model to predict the corresponding 3D point map geometric information and obtain the scene geometric prior; initialize the 3D Gaussian scene based on the geometric prior, using multiple 3D Gaussian primitives to represent the scene geometric structure, each Gaussian primitive being represented as:

2. wherein represents the spatial center coordinate of the th Gaussian cell; represents a three-dimensional covariance matrix; represents a color parameter; represents a transparency parameter; all three-dimensional Gaussian cells together form a three-dimensional Gaussian map; Step 2) Establish a hybrid associated pose tracking model; extract ORB feature points from the current image and establish feature matching relationships with reference keyframes; combine the geometric priors predicted by the Point-map model to recover the corresponding 3D spatial points, and establish the geometric correspondence between 2D image points and 3D spatial points; use the current 3D Gaussian map to generate a predicted image under the current viewpoint through a differentiable Gaussian renderer, and calculate the photometric consistency error between the predicted image and the real image; establish a joint pose optimization objective function:

3. wherein, denotes the current camera pose, denotes the geometric reprojection error, denotes the photometric consistency error, and denote the weight coefficients of the geometric constraint term and the photometric constraint term, respectively; The geometric reprojection error is expressed as:

4. Among them, Indicates the first A three-dimensional point, Represents the corresponding image feature points. The camera projection function is represented; the joint pose optimization objective function is optimized using the Lie algebra Gauss-Newton iterative method to obtain the camera pose of the current frame; Step 3) Establish a keyframe management mechanism; determine whether to insert a keyframe based on the ORB feature co-view relationship, Gaussian co-view relationship, and camera motion amplitude between the current frame and existing keyframes; whereby the Gaussian co-view rate is expressed as:

5. wherein, and respectively represent the key frame and the key frame observable Gaussian cell set; when the preset key frame insertion condition is met, the current frame is added to the key frame set; when the number of key frames exceeds the preset sliding window size, redundant key frames are deleted according to the key frame contribution degree; Step 4) constructing a multi-scale consistency optimization model; establishing a multi-layer image pyramid, respectively performing Gaussian differentiable rendering at each scale and calculating the photometric consistency error of the corresponding scale; assuming that the image pyramid contains layers, then the multi-scale photometric consistency loss is represented as:

6. wherein, represents the first photometric consistency error corresponding to the layer image, represents the scale weight; meanwhile, the three-dimensional points predicted by the Point-map model are introduced as geometric priors to establish scale consistency constraints:

7. Among them, This represents the 3D points predicted by the Point-map model. Indicates the center position of the corresponding Gaussian element. Represents the local scale factor; jointly optimizes multi-scale photometric consistency loss and scale consistency constraints; Step 5) Establish a local reliability weighted map optimization model; based on the error between the depth predicted by the Point-map model and the current Gaussian map rendering depth, and the corresponding luminosity residuals of the region, calculate the local reliability weight of each Gaussian element, which is expressed as:

8. Among them, This represents the error between the predicted depth and the rendered depth. This represents the photometric residual in the corresponding region. and Indicate the adjustment parameters; establish the map optimization objective function:

9. Update the position, covariance matrix, color parameters, and transparency parameters of Gaussian elements based on the map optimization objective function; Step 6) Repeat steps 2) to 5) until all image sequence processing is completed, output the camera motion trajectory and the optimized 3D Gaussian map, and render the new perspective image based on the optimized 3D Gaussian map.