Large-scale scene geometric reconstruction method of Gaussian splashing based on load balancing

Abstract

The invention discloses a load balancing Gaussian sputtering method for large-scale scene geometric reconstruction. The method comprises the following steps: step 1, acquiring a multi-view RGB image sequence of a large-scale scene, and processing the multi-view RGB image sequence by using a COLMAP tool to obtain camera parameters and an initial sparse point cloud; 2, based on a pixel-level load balancing strategy, effective Gaussian primitives are screened, spatial distribution of the effective Gaussian primitives is regulated and controlled through a loss function, and redundant primitives are reduced; 3, introducing global consistent geometric prior, fusing monocular depth and sparse point cloud depth to generate a reference depth map, and constructing depth consistency and smooth loss; 4, dividing illumination areas through a multi-area exposure compensation model, adaptively correcting brightness, and constructing image luminosity loss; and 5, performing global optimization in combination with multi-dimensional loss. And 6, rendering a depth map for the trained model, generating a TSDF field by using a TSDF fusion algorithm, and extracting a scene grid to obtain a final three-dimensional geometric model. According to the method, large-scale scene high-precision reconstruction can be realized on a single GPU, memory consumption is remarkably reduced, and geometric reconstruction quality and illumination robustness are improved.

Description

Technical Field

[0001] This invention belongs to the field of 3D reconstruction and computer vision, specifically involving a 3D Gaussian Splatting (3DGS) large-scale scene geometric reconstruction method that integrates pixel-level load balancing, global geometric prior and multi-region exposure compensation. It is applicable to high-precision scene modeling in fields such as urban planning, autonomous driving, and virtual reality. Background Technology

[0002] The core objective of large-scale scene 3D reconstruction is to construct 3D models with accurate geometric structures from image data to achieve high-quality visualization and physical simulation. In recent years, 3D Gaussian Splatting has become the mainstream method in this field due to its high computational efficiency and realistic rendering—it represents the scene using a discrete set of 3D Gaussian primitives and achieves rendering through efficient rasterization. However, existing 3DGS methods face two major challenges in large-scale scene reconstruction:

[0003] Memory bottleneck: The number of 3D Gaussian primitives continues to increase during training, leading to a surge in GPU memory consumption. Even high-end hardware struggles to support global optimization across all scenarios. To address this issue, existing methods often employ partition-based multi-GPU strategies (such as CityGaussianV2 and VastGaussian). However, these methods require maintaining redundant block boundary information, which not only fails to reduce overall memory consumption but also results in geometric discontinuities at block boundaries due to a lack of cross-block consistency constraints, thus reducing reconstruction quality.

[0004] Insufficient robustness in geometry and lighting: On the one hand, existing methods rely on image supervision to generate Gaussian distributions, which can easily lead to excessive concentration of Gaussian distributions in geometric edges and texture-rich areas, while sparse texture areas are not adequately represented, causing problems such as floating object artifacts and incomplete surface reconstruction; on the other hand, changes in lighting in large-scale outdoor scenes (such as fluctuations in sunlight intensity and glass reflections) can lead to inconsistent image exposure, further affecting the stability of reconstruction and rendering. Summary of the Invention

[0005] This invention proposes a "large-scale scene geometry reconstruction method based on load balancing Gaussian splashing". Through three core modules, pixel-level load balancing, globally consistent geometric prior and multi-region exposure compensation, the reconstruction quality and robustness are improved while reducing memory consumption.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A large-scale scene geometry reconstruction method based on load-balanced Gaussian Splatting includes the following steps:

[0008] Step 1: Acquire multi-view RGB images of large-scale scenes, obtain camera intrinsic and extrinsic parameters and sparse point clouds through COLMAP, and initialize a 3D Gaussian primitive set after downsampling the images;

[0009] Step 2: Select effective high-performance cells using a pixel-level load balancing strategy, design load balancing loss and incorporate it into the total loss to achieve uniform distribution of high-performance cells and memory optimization.

[0010] Step 3: Construct a globally consistent geometric prior based on monocular depth estimation and COLMAP sparse point cloud, and improve the consistency of geometric reconstruction through scale calibration, depth loss and smoothing constraints;

[0011] Step 4: Employ a multi-region exposure compensation model to divide the image into overexposed, underexposed, and normal regions and apply learnable compensation coefficients to correct uneven image brightness.

[0012] Step 5: Construct the overall objective function by fusing load balancing loss, geometric constraint loss, and image loss, and optimize Gaussian elements and model parameters through backpropagation;

[0013] Step 6: Render a depth map for the trained model, generate a TSDF field using the TSDF fusion algorithm, and extract the scene mesh to obtain the final 3D geometric model.

[0014] Beneficial effects of the present invention

[0015] Compared with the prior art, the present invention has the following significant advantages:

[0016] (1) High memory efficiency: By reducing redundant high-level elements through a pixel-level load balancing strategy, the GPU memory consumption is only 13.5GB (single GPU), which is much lower than CityGaussianV2, and supports single GPU training in large-scale scenarios.

[0017] (2) Excellent geometric reconstruction accuracy: Global geometric prior constraints improve cross-view consistency and effectively suppress planktonic artifacts;

[0018] (3) Strong robustness to illumination: The multi-area exposure compensation model is adapted to complex outdoor illumination.

[0019] (4) Fast training speed: The training time is only 3 hours, balancing efficiency and accuracy. Attached Figure Description

[0020] Figure 1 : Overall framework diagram of the method of this invention;

[0021] Figure 2 Pixel-level load balancing performance comparison chart;

[0022] Figure 3Comparison of reconstructed meshes in different scenarios. Detailed Implementation

[0023] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments described are only for explaining the present invention and are not intended to limit the present invention.

[0024] Step 1: Data Preprocessing and Initial Gaussian Initialization

[0025] Input data: Multi-view RGB images of large-scale scenes, with camera intrinsic and extrinsic parameters and sparse point clouds obtained through COLMAP.

[0026] Image downsampling: Downsamples the long side of an image to a suitable pixel size, balancing accuracy and computational efficiency.

[0027] Gaussian initialization: Generate an initial set of 3D Gaussian primitives based on sparse point clouds. Each Gaussian primitive consists of a center, a covariance matrix (determined by the scale matrix and rotation matrix), opacity, and spherical harmonic coefficients (used to calculate view-dependent colors).

[0028] Step 2: Pixel-level load balancing strategy

[0029] By dynamically adjusting the distribution of Gaussian primitives, reducing redundant primitives, and supplementing sparse regions, memory optimization and geometric accuracy improvement are achieved, specifically including:

[0030] 2.1 Gaussian Filter

[0031] Based on the opacity threshold, Gaussian primitives that significantly contribute to the pixel count are filtered out, and invalid primitives are removed:

[0032]

[0033] T i β1 and β2 are preset thresholds for cumulative opacity.

[0034] 2.2 Load Balancing Loss Design

[0035] Calculate the number of Gaussian pixels required for each pixel, and achieve a Gaussian uniform distribution by minimizing the standard deviation loss:

[0036]

[0037] Where N = H × W (H and W are the image height and width).

[0038] Step 3: Globally Consistent Geometric Prior Constraints

[0039] Integrating monocular depth and sparse point cloud scale information improves the consistency of geometric reconstruction, specifically including:

[0040] 3.1 Depth Map Acquisition and Calibration

[0041] Initial dense depth: Obtain monocular dense depth map D using models such as DepthAnything-V2. d =F(I).

[0042] Sparse depth projection: Sparse depth based on COLMAP sparse point cloud computing.

[0043] Scale calibration: Optimization of D using the least squares method d The scale and offset are used to obtain the calibrated depth D. f :

[0044] 3.2 Depth Loss and Smoothing Constraints

[0045] Depth Loss: Using the calibration depth Df as supervision, constrain the Gaussian rendering depth Dr:

[0046] Smoothing Constraints: Using the Canny edge detector to identify depth discontinuities, smoothing constraints are applied to continuous regions to avoid floating-point errors.

[0047] Step 4: Multi-area exposure compensation

[0048] To address the issue of uneven lighting in outdoor scenes, image brightness is dynamically adjusted in different areas, specifically including:

[0049] 4.1 Regional Division

[0050] The image is converted to grayscale (Igray). The top 1% of pixels by brightness are classified as overexposed areas, the bottom 1% as underexposed areas, and the rest as normal areas. 4.2 Exposure Compensation

[0051] Learnable compensation coefficients are applied to different areas to correct brightness:

[0052]

[0053] Among them, a b ,b b (overexposure), a d ,b d (Underexposure), a, b (global) are learnable compensation coefficients, c (p) is pixel color, and Ie is the compensated image.

[0054] 4.3 Image Loss Calculation

[0055] Combining L1 loss and SSIM loss, the decision to use the compensated image is made based on the SSIM values ​​of the rendered image Ir and the ground truth Igt.

[0056] Step 5: Model Training and Geometric Reconstruction

[0057] Total loss function: The load balancing loss, geometric constraint loss, and image loss are fused, and the Gaussian meta-parameters are optimized through backpropagation: L total =L load +L depth +L smooth +L image

[0058] Mesh extraction: After training, a depth map is rendered for each viewpoint, and a TSDF field is generated through the TSDF fusion algorithm to extract the scene mesh.

[0059] Step 6: Performance Evaluation

[0060] The following metrics were used to evaluate performance:

[0061] Geometric precision: Accuracy (Acc), Completeness (Comp), Precision, Recall, F1-score;

[0062] Rendering quality: PSNR, SSIM, LPIPS;

[0063] Efficiency: Training time (h), GPU memory consumption (GB)

[0064] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A load-balanced Gaussian sputtering method for large-scale scene geometry reconstruction, characterized in that, Includes the following steps: Step 1: Obtain multi-view RGB image sequences of large-scale scenes, and use COLMAP tool to process them to obtain camera intrinsic parameters, extrinsic parameters and initial sparse point cloud; Step 2: Optimize the Gaussian primitive distribution based on the pixel-level load balancing strategy, select effective Gaussian primitives, and adjust the Gaussian spatial distribution through the loss function; Step 3: Introduce globally consistent geometric prior constraints, generate a fused depth map, and construct depth consistency and smoothing loss; Step 4: Correct the illumination difference through a multi-region exposure compensation model, divide the illumination area and adaptively correct the brightness to construct the image luminance loss; Step 5: Perform global optimization by combining load balancing loss, depth consistency loss, depth smoothing loss, and image photometric loss. Step 6: Render a depth map for the trained model, generate a TSDF field using the TSDF fusion algorithm, and extract the scene mesh to obtain the final 3D geometric model.

2. The method according to claim 1, wherein the pixel-level load balancing strategy in step 2 further comprises: Step 2-1: Set a threshold based on the opacity and cumulative opacity of Gaussian primitives to filter out Gaussian primitives that significantly contribute to pixel rendering and filter out redundant primitives. Step 2-2: Calculate the number of Gaussian primitives required for each pixel, design a load balancing loss function, minimize the loss to make the Gaussian primitives evenly distributed in space, reduce redundant primitives in overly dense areas, and supplement primitives in sparse areas.

3. The method according to claim 1, wherein the globally consistent geometric prior constraints in step 3 specifically include: Step 3-1: Use a monocular depth estimation model to obtain an initial dense depth map, and project the sparse point cloud generated by COLMAP to obtain a sparse depth map. Step 3-2: Optimize the scale and offset parameters using the least squares method to correct the initial dense depth map into a fused depth map with the same scale as the sparse depth map; Step 3-3: Construct a depth consistency loss using the fused depth map as supervision, and introduce a depth smoothing loss based on edge detection to constrain the depth continuity of adjacent pixels.

4. The method according to claim 1, wherein the multi-region exposure compensation model in step 4 comprises: Step 4-1: Convert the input image to grayscale and divide it into overexposed, underexposed, and normal areas according to brightness; Step 4-2: Set learnable exposure compensation coefficients for different areas to adaptively correct pixel colors; Step 4-3: Select the corrected or original rendered image based on the structural similarity (SSIM) between the rendered image and the real image, and calculate the image photometric loss weighted by L1 loss and SSIM loss.

5. According to the method of claim 1, the load balancing loss in step 2 is optimized by adjusting the opacity of Gaussian pixels: when the number of Gaussian pixels exceeds the mean, the opacity is reduced to remove redundant pixels; when the number of Gaussian pixels is lower than the mean, the opacity is increased to supplement pixels, so that the Gaussian pixels are evenly distributed in space, thereby reducing GPU memory consumption.

6. According to the method of claim 1, the depth smoothing loss in step 3 identifies depth discontinuity regions using the Canny edge detector, and applies smoothing constraints only to the depths of adjacent pixels in non-edge regions to avoid blurring of edge details.

7. According to the method described in claim 1, the global optimization in step 5 synchronously updates the Gaussian meta-parameters (including Gaussian center, covariance matrix, and opacity), camera pose parameters, and exposure compensation coefficients through backpropagation, thereby achieving joint optimization of scene geometry, camera pose, and illumination parameters.

8. The method according to claim 1, the output results include a high-fidelity three-dimensional mesh geometric model, an optimized depth map and a corrected appearance texture image, which can be directly used for engineering applications such as urban planning visualization, autonomous driving scene modeling and virtual reality scene construction.

9. The method according to claim 1, wherein the input image in step 1 is an unprocessed original RGB image without illumination correction, texture enhancement or other preprocessing, to avoid interference from preprocessing artifacts on Gaussian distribution optimization and geometric reconstruction.

Images