A sparse-view 3D reconstruction method based on a 3DGS dual-branch initialization model

By using a dual-branch model generation and fusion method, the problem of insufficient Gaussian field initialization under sparse perspective was solved, and a more stable 3D reconstruction effect was achieved.

CN123118925APending Publication Date: 2026-11-17GUILIN UNIV OF ELECTRONIC TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611535219.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-30
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Under sparse perspective conditions, existing methods struggle to generate a sufficiently covered and stable initial Gaussian field, and the coarse initialization of Gaussian parameters leads to background blurring and local geometric instability.

Method used

A dual-branch depth calibration and geometric confidence-gated residual prediction method is adopted. Geometric anchor points and basic Gaussian parameters are generated by DCBP, and geometric confidence and parameter residuals are predicted by GCRP. Combined with cross-branch gating fusion, a stable initial Gaussian field is generated.

Benefits of technology

It improves the stability and accuracy of 3D reconstruction from a sparse perspective, reduces floating artifacts and background blurring in the initial Gaussian field, and enhances the reliability of geometric relationships.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118925A_ABST
    Figure CN123118925A_ABST
Patent Text Reader

Abstract

This invention discloses a sparse-view 3D reconstruction method based on a dual-branch initialization model of 3DGS. The model includes a Depth Calibration Backprojection (DCBP) branch and a Geometric Confidence Gated Residual Prediction (GCRP) branch. DCBP performs scale and offset calibration on monocular depth, generating initial point clouds and basic Gaussian parameters through multi-view backprojection, filtering, and fusion. GCRP uses the initial 3D points as query anchors, aggregates multi-view image features, calculates geometric confidence, and predicts scale and opacity residuals. The geometric confidence is used to control the cross-branch fusion of residuals and basic parameters to generate an initial Gaussian field. After differentiable rendering and image reconstruction loss optimization, the RGB image and depth map of the target view are output. This invention can reduce the impact of point cloud noise and occlusion errors on parameter initialization, reduce floating artifacts, and improve reconstruction stability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of computer vision, 3D scene reconstruction, and neural rendering, and particularly to a 3D Gaussian reconstruction method for sparse viewpoint input scenes. More specifically, this invention relates to a two-branch Gaussian initialization model consisting of a depth calibration backprojection branch and a geometric confidence-gated residual prediction branch, used to generate initial point clouds, basic Gaussian parameters, and their confidence-gated residual corrections. Background Technology

[0002] 3D Gaussian field representation methods model scenes using a set of Gaussian ellipsoids with position, scale, opacity, and color attributes, demonstrating high efficiency in novel viewpoint synthesis and sparse viewpoint reconstruction tasks. Existing methods typically rely on structure-of-motion (SfM) methods or sparse reconstruction results as the initial source of Gaussian centers, and initialize the remaining Gaussian parameters using nearest neighbor distances, fixed initial opacity values, or simple heuristic rules.

[0003] Under sparse view conditions, especially when the number of training views is small, generating an initial Gaussian field based solely on sparse or random point clouds can easily lead to problems such as insufficient coverage of the initial Gaussian center and slow convergence speed in subsequent optimization. At the same time, the initialization of parameters such as Gaussian scale and opacity is relatively coarse, which can easily lead to phenomena such as blurred background regions and local geometric instability.

[0004] Monocular depth sensing can provide some geometric priors, but existing methods typically treat depth backprojection, point cloud construction, and Gaussian parameter prediction as independent processes, failing to establish a control relationship between initial geometric reliability and the intensity of Gaussian parameter correction. Therefore, a two-branch Gaussian initialization method is still needed: a depth calibration backprojection branch generates geometric anchor points and basic Gaussian parameters, a geometric confidence-gated residual prediction branch estimates geometric reliability and predicts parameter residuals using multi-view features, and finally, a stable initial Gaussian field is generated through cross-branch gating fusion. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a sparse-view 3D Gaussian reconstruction method based on dual-branch depth calibration and geometric confidence-gated residual prediction. This method generates geometric anchor points and fundamental Gaussian parameters using DCBP, predicts geometric confidence and parameter residuals using GCRP, and generates the initial Gaussian field using cross-branch gating fusion.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] S1. Acquire sparse multi-view images of the same scene, camera intrinsic and extrinsic parameters, and sparse reconstructed point clouds; perform monocular depth estimation on the sparse multi-view images to obtain monocular depth maps corresponding to each input viewpoint.

[0008] S2. Using DCBP, based on the geometric correspondence between the 3D points in the sparsely reconstructed point cloud and the corresponding pixels in each monocular depth map, determine the scale correction parameters and offset correction parameters to obtain the aligned depth map; based on the camera intrinsic and extrinsic parameters, backproject the aligned depth map to the unified world coordinate system, and perform multi-view consistency screening, fusion, downsampling, and outlier filtering on the candidate 3D points generated from different viewpoints to obtain the initial point cloud.

[0009] S3. Using DCBP, with the 3D points in the initial point cloud as the Gaussian center, determine the basic scale, basic rotation, basic opacity, and basic color or spherical harmonic coefficients based on neighborhood distance, point density, visibility, and color information to form the basic Gaussian parameters.

[0010] S4. Extract multi-scale image features from each input viewpoint using GCRP, and use the initial 3D points generated by DCBP as feature query anchor points to project the initial 3D points onto the image feature maps of each input viewpoint.

[0011] S4-1. Sample image features at the corresponding projection position, and aggregate the image features sampled from the same 3D point at different viewpoints according to visibility, depth consistency and viewpoint weight to obtain point-level multi-view fusion features.

[0012] S5. Using GCRP, calculate the geometric confidence score based on at least two of the following: multi-view depth consistency, number of effective visible views, cross-view feature consistency, and local geometric stability. The geometric confidence score is used to characterize the reliability of the initial point as a scene surface point.

[0013] S5-1. The point-level multi-view fusion features are concatenated with the 3D point coordinates, color, neighborhood distance statistics, local geometric features and geometric confidence, and then input into the residual prediction module in GCRP to obtain the corresponding scale residual and opacity residual.

[0014] S6. Map the geometric confidence to residual gating weights, and according to the residual gating weights, perform cross-branch gating fusion of the parameter residuals output by GCRP and the basic Gaussian parameters output by DCBP to obtain the initialized three-dimensional Gaussian field.

[0015] S7. Perform differentiable projection and rasterization rendering on the initialized 3D Gaussian field, compare the rendered image with the real image, and calculate the image reconstruction loss including L1 loss and SSIM loss.

[0016] S7-1. Backpropagate the image reconstruction loss to update the position, scale, rotation, opacity, color, or spherical harmonic coefficients of the Gaussian ellipsoid; and prune or densify the Gaussian ellipsoid based on the gradient magnitude, opacity contribution, geometric confidence, and rendering error.

[0017] S7-2. Input the target camera pose into the optimized 3D Gaussian field, and output the scene RGB image and depth map from the corresponding viewpoint.

[0018] Beneficial effects

[0019] This invention performs scale and offset calibration on monocular depth using DCBP, and completes multi-view backprojection, filtering and fusion. It can generate initial point clouds and basic Gaussian parameters with more complete coverage and more stable geometric relationships under sparse view conditions, reducing the dependence on SfM point cloud density and fixed initial parameter values.

[0020] This invention uses GCRP with 3D points generated by DCBP as feature query anchors, aggregates multi-view image features, and calculates geometric confidence by combining depth consistency, visibility, feature consistency, and local geometric stability. The geometric confidence is used to control the correction intensity of parameter residuals on the basic Gaussian parameters, so that the two branches form a closed-loop coupling between geometric anchors, reliability judgment, and parameter correction, thereby reducing floating artifacts, background blurring, and local geometric instability, and improving the stability and accuracy of sparse view 3D reconstruction. Attached Figure Description

[0021] Figure 1 This is a flowchart of the sparse perspective 3D reconstruction method based on the 3DGS dual-branch initialization model of the present invention.

[0022] Figure 2 This is a structural diagram of the dual-branch Gaussian initialization model of the present invention, which includes DCBP Branch and GCRPBranch. Detailed Implementation

[0023] The present invention will be further described below with reference to the accompanying drawings and embodiments. The described embodiments are some, not all, of the embodiments of the present invention. The detailed description of the embodiments of the present invention below with reference to the accompanying drawings is not intended to limit the scope of protection of the present invention, but only to illustrate selected embodiments of the present invention.

[0024] like Figure 1 As shown, this invention first uses a two-branch Gaussian initialization model to obtain the initial Gaussian field, the structure of which is as follows: Figure 2As shown, the model includes a Depth Calibration Backprojection Branch (DCBP Branch) and a Geometric Confidence Gated Residual Prediction Branch (GCRP Branch). DCBP performs scale and offset calibration on the monocular depth and generates an initial point cloud and basic Gaussian parameters through multi-view backprojection, filtering, and fusion. GCRP uses these initial 3D points as feature query anchors, aggregates multi-view image features, calculates geometric confidence, and predicts Gaussian parameter residuals. The geometric confidence is used to control the cross-branch fusion of parameter residuals and basic Gaussian parameters, thereby obtaining a complete initial Gaussian field model.

[0025] Subsequently, using a 3D Gaussian splash as the main framework, the initialized 3D Gaussian ellipsoid is projected, depth-sorted, and rasterized from different viewpoints. Adaptive adjustments such as densification, cloning, splitting, and pruning are then performed on the Gaussian ellipsoid based on gradient information. During training, the image reconstruction loss between the rendered image and the real image is used for supervision, and parameters such as Gaussian center, scale, rotation, opacity, and spherical harmonics are updated through backpropagation. Finally, the RGB images and depth maps of the scene from different viewpoints are output.

[0026] Specifically, the method includes the following steps:

[0027] S1. Obtain sparse multi-view images, camera parameters, and sparse reconstructed point clouds, and generate an aligned depth map.

[0028] Input sparse multi-view images of the same scene acquired from different perspectives, and obtain camera intrinsic and extrinsic parameters and sparse reconstructed point clouds generated by the SfM method:

[0029]

[0030] Where N represents the number of input viewpoints. Each image I n Corresponding camera intrinsic parameter matrix K n and camera external parameters T n The camera extrinsic parameters are used to describe the transformation relationship between the camera coordinate system and the unified world coordinate system.

[0031] The images from each input viewpoint are input into the Depth Anything V2 monocular depth estimation network in DCBP to obtain the corresponding depth maps:

[0032]

[0033] in, D represents a monocular depth estimation network. n(u,v) represents the predicted depth value corresponding to pixel (u,v) in the nth view. DCBP projects the sparsely reconstructed point cloud onto the nth view, establishes a matching relationship between the camera depth of the effective projection points and the predicted depth of the corresponding pixels, fits scale correction parameters and offset correction parameters, and performs linear correction on the predicted depth to obtain an aligned depth map consistent with a unified world scale.

[0034] S2. Back-project the aligned depth map using DCBP to generate candidate 3D points.

[0035] For the effective pixel p=(u,v) in the nth view image, it is back-projected onto the camera coordinate system based on the camera intrinsic parameters and the corresponding depth value:

[0036]

[0037] Based on the camera's extrinsic parameters, the 3D points in the camera coordinate system are transformed to a unified world coordinate system. Let the transformation from the camera to the world coordinate system be T. w←n =[R n ,t n ],but:

[0038]

[0039] Simultaneously, the RGB color corresponding to pixel p is assigned to the generated 3D point, thus obtaining candidate 3D points with color attributes:

[0040]

[0041] Among them, C n (u,v) represents the color information of the corresponding pixel.

[0042] S2-1. Candidate 3D points are screened, fused, and downsampled using DCBP.

[0043] Candidate 3D points generated from different perspectives are uniformly transformed to the same world coordinate system, and the candidate points are filtered through multi-view geometric consistency. For candidate 3D point P... i Project it onto other input viewpoints and compare the deviation between its projected depth and the predicted depth of the corresponding viewpoint:

[0044]

[0045] Where, π n (·) represents the projection function of a 3D point in the nth viewpoint, z i,nThis represents the depth of a 3D point in the camera coordinate system at that viewpoint. When the depth deviation meets a preset threshold, the point is considered to have geometric consistency in the corresponding viewpoint; when the number of views with geometric consistency is less than the preset threshold, the point is removed as a low-confidence point. Furthermore, duplicate points that are spatially adjacent are fused. For point sets Ω belonging to the same local neighborhood or the same voxel... i Its fusion position can be represented as:

[0046]

[0047] Where, ω j The fusion weights for the corresponding candidate points are defined by ε, a constant set to prevent the denominator from being zero. Subsequently, voxel downsampling, farthest-point sampling, or random downsampling are employed to reduce the number of point clouds. Noise is then removed through depth outlier filtering, radius outlier filtering, and boundary point filtering to obtain the initial point cloud set.

[0048]

[0049] Where M is the number of initial points after filtering and downsampling.

[0050] S3. Construct basic Gaussian parameters based on the initial point cloud using DCBP.

[0051] Treat each 3D point in the initial point cloud as the center of a 3D Gaussian ellipsoid:

[0052]

[0053] According to point P i The nearest neighbor points are at the structural base scale. Let N be the distance to the structural base scale. k (P i Let P be a point. i The fundamental scale of the k-nearest neighbor set can be expressed as:

[0054]

[0055] Where, β s This is the scaling factor. The base scale can be set to an isotropic scale or an anisotropic scale in three directions constructed based on the local point cloud covariance. The base rotation parameter can be initialized to a unit rotation or determined based on the eigenvectors obtained from principal component analysis of the local point cloud. The base opacity can be set to a preset value or determined based on point density and multi-view visibility.

[0056]

[0057] Among them, a i 0Let σ(·) be the initial log-probability parameter for opacity. The base color or zeroth-order spherical harmonic coefficients are obtained by weighted fusion of the colors of the corresponding pixels at that point under multiple visible viewpoints, thus forming the base Gaussian parameters.

[0058]

[0059] S4. Extract multi-scale image features through GCRP and perform projection-based multi-view aggregation.

[0060] S4-1. Input the images from each sparse viewpoint into the backbone network and feature pyramid network (FPN) in GCRP to extract multi-scale image features. The backbone network outputs multi-scale features at different stages:

[0061]

[0062] Shallow features have high spatial resolution and are used to preserve texture, contour, and edge details; deep features have strong semantic expressive power and are used to describe object structure and overall scene information. FPN fuses features at different scales through a top-down path and lateral connections. For example:

[0063]

[0064]

[0065] Where, φ l (·) represents the channel mapping operation, and Up(·) represents the upsampling operation. This yields the multi-scale image features corresponding to each viewpoint:

[0066]

[0067] This multi-scale feature contains both local detail information and high-level semantic information, providing a feature foundation for subsequent point-level Gaussian parameter residual prediction.

[0068] S4-2. Using the initial 3D points generated by DCBP as feature query anchor points, perform multi-view feature aggregation based on projection relationships.

[0069] For a 3D point P in the initial point cloud i Project the image onto the image plane of the nth viewpoint based on the camera parameters of each viewpoint:

[0070]

[0071] Among them, T n←w This represents the transformation from the world coordinate system to the nth camera coordinate system. Image features are sampled at the projection positions of the feature maps at each scale using bilinear interpolation.

[0072]

[0073] Where, p i,n l This represents the corresponding position of the projected coordinates in the feature map of layer l. The validity mask m is determined based on the point's visibility, depth consistency, viewing angle, and image boundary conditions. i,n Weighted aggregation of features sampled from the same 3D point across multiple viewpoints:

[0074]

[0075] Where, ω i,n This represents the nth viewpoint on point P. i Feature aggregation weights. Aggregated features at different scales are concatenated or weighted and fused to obtain multi-view features:

[0076]

[0077] This multi-view feature can fuse texture, contour and semantic information from different perspectives at a unified 3D point location, reducing parameter prediction errors caused by occlusion or missing information from a single perspective.

[0078] S5. Estimate the geometric confidence of the initial point using GCRP.

[0079] Calculate the geometric confidence score for each initial point based on at least two of the following: multi-view depth consistency, visibility, feature consistency, and local geometric stability. The depth consistency confidence score can be expressed as:

[0080]

[0081] Among them, V i Point P i The effective set of visible viewpoints, δ d This is the depth error control parameter. Visibility confidence can be expressed as:

[0082]

[0083] Feature consistency confidence can be determined based on the differences between features sampled from different perspectives and aggregated features:

[0084]

[0085] The confidence level for local geometric stability can be determined based on the covariance characteristics of the point's neighborhood, the plane fitting residuals, or the normal consistency. The confidence levels are then normalized and weighted for fusion.

[0086]

[0087] Among them, c i∈[0,1] represents point P i The final geometric confidence level, λ d , λ v , λ f , λ g This indicates the weight of each confidence component.

[0088] S5-1. Predict Gaussian parameter residuals using the residual prediction module in GCRP.

[0089] By concatenating multi-view features, point coordinates, color, neighborhood distance statistics, local geometric features, and geometric confidence, a point-level comprehensive feature is constructed.

[0090]

[0091] Where, d i nbr G represents the neighborhood distance or point density feature. i This represents the local geometric stability characteristics. The synthesized features are then input into the MLP Gaussian parameter residual predictor:

[0092]

[0093] The Gaussian parameter residuals output by the MLP include at least:

[0094]

[0095] Where, Δs i For the scale residual, Δa i This is the opacity residual. In an optional implementation, the parametric residual further includes the rotation residual Δr. i Color residual ΔC i or spherical harmonic coefficient residual ΔSH i The MLP does not directly regenerate the Gaussian center, but instead predicts local parameter corrections based on the existing point cloud geometry and fundamental Gaussian parameters, thereby reducing the instability caused by directly predicting the complete Gaussian parameters. Subsequently, residual gating weights are generated based on the geometric confidence obtained in S5.

[0096]

[0097] For initial points with high geometric confidence, the residual gating weights are increased, enabling the MLP to finely correct the underlying Gaussian parameters using reliable multi-view features. For initial points with low geometric confidence, the residual gating weights are decreased, preserving more of the underlying parameters and avoiding anomalous scales or opacities caused by depth errors, occluded boundaries, and floating points. Scale parameters are preferably fused using residuals in logarithmic space.

[0098]

[0099]

[0100] The opacity parameter is preferably fused in logarithmic probability space:

[0101]

[0102]

[0103] When predicting rotational residuals or spherical harmonic coefficient residuals, they can be expressed as follows:

[0104]

[0105]

[0106] S6. Perform geometric confidence-gated fusion of the basic Gaussian parameters output by DCBP and the parameter residuals output by GCRP to obtain the initialization parameters of each three-dimensional Gaussian ellipsoid, and combine them with the initial point cloud coordinates to form a complete initial Gaussian field:

[0107]

[0108] S7. Project, rasterize, render, and optimize the parameters of the initial 3D Gaussian field generated by the dual-branch model.

[0109] The three-dimensional Gaussian ellipsoid obtained in S6 is projected onto the two-dimensional image plane of the target viewpoint. For the i-th three-dimensional Gaussian ellipsoid, its three-dimensional covariance matrix can be expressed as:

[0110]

[0111] Through camera projection transformation, the 3D covariance is mapped to the 2D screen space covariance, and the Gaussian ellipsoids are sorted according to the depth of the Gaussian center in the target camera coordinate system. Alpha blending is performed in order from near to far, and the rendered color of pixel p can be represented as:

[0112]

[0113] in:

[0114]

[0115] This represents the cumulative transmittance up to the i-th Gaussian. A similar method can be used to calculate the rendered depth map from the target's perspective.

[0116] S7-1. Calculate the image reconstruction loss and optimize the Gaussian field parameters.

[0117] The rendered RGB image nCorresponding real image I n The comparison is performed, and the image reconstruction loss is calculated. The L1 loss can be expressed as:

[0118]

[0119] It can also be combined with structural similarity loss:

[0120]

[0121] Gradient backpropagation is performed based on the total loss to update parameters such as Gaussian center, scale, rotation, opacity, and spherical harmonics. During optimization, the Gaussian ellipsoid is adaptively adjusted based on its positional gradient, scale, opacity, visibility count, and rendering contribution. For Gaussian ellipsoids with large gradients and small scales, cloning can be used to increase the number of local points; for Gaussian ellipsoids with large gradients and large scales, splitting can be used to generate multiple smaller Gaussian ellipsoids to improve the representation of complex and detailed regions. Gaussian ellipsoids with opacity below a preset threshold, long-term invisibility, small projection contribution, or low geometric confidence that cannot be corrected through optimization are pruned to reduce redundant points and floating artifacts.

[0122] S7-2, Output scene rendering results from any viewpoint.

[0123] After optimization, any target camera pose is input into a 3D Gaussian field. Through 3D Gaussian projection and rasterization rendering, the scene RGB image and depth map at the corresponding viewpoint are output to describe the texture appearance and 3D structure information of the scene.

[0124] Those skilled in the art will understand that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Without departing from the spirit and principles of the present invention, modifications, substitutions, or combinations can be made to the network structure, feature aggregation method, geometric confidence calculation method, parameter residual type, and fusion method in the above embodiments, and all equivalent modifications and substitutions should be included within the protection scope of the present invention.

Claims

1. A sparse-view 3D reconstruction method based on a dual-branch initialization model using 3DGS, characterized in that, The method employs a two-branch Gaussian initialization model, which includes a depth calibration back projection branch (DCBP branch) and a geometric confidence gated residual prediction branch (GCRP branch). The method includes the following steps: S1. Acquire sparse multi-view images of the same scene, camera intrinsic and extrinsic parameters, and sparse reconstructed point cloud; perform monocular depth estimation on the sparse multi-view images to obtain monocular depth maps corresponding to each input view. S2. Using DCBP, based on the geometric correspondence between the 3D points in the sparsely reconstructed point cloud and the corresponding pixels in each monocular depth map, determine the scale correction parameters and offset correction parameters of each monocular depth map to obtain the aligned depth map; back-project the aligned depth map to the unified world coordinate system according to the camera intrinsic and extrinsic parameters, and perform multi-view consistency screening, duplicate point fusion, downsampling, and outlier point filtering on the candidate 3D points generated from different viewpoints to obtain the initial point cloud; S3. Using DCBP, the three-dimensional points in the initial point cloud are used as the Gaussian center. The basic scale is determined based on the neighborhood distance or local point density. The basic opacity is determined based on the point density or visibility. The basic color or spherical harmonic coefficient is determined based on the color information of the visible viewpoint, thus forming the basic Gaussian parameters. S4. Extract multi-scale image features from each input viewpoint using GCRP, and use the three-dimensional points in the initial point cloud generated by DCBP as feature query anchor points to project the three-dimensional points onto the image feature maps of each input viewpoint. S4-1. Sample image features at the corresponding projection position, and aggregate the image features sampled from the same 3D point at different viewpoints according to visibility, depth consistency and viewpoint weight to obtain the point-level multi-view fusion feature corresponding to the 3D point. S5. Using GCRP, calculate the geometric confidence of each initial point based on at least two of the following: multi-view depth consistency, number of effective visible views, cross-view feature consistency, and local geometric stability. The geometric confidence is used to characterize the reliability of the initial point as a scene surface point. S5-1. The point-level multi-view fusion features are concatenated with the 3D point coordinates, color, neighborhood distance statistics, local geometric features and geometric confidence, and input into the residual prediction module in GCRP to obtain the scale residual and opacity residual of the corresponding Gaussian ellipsoid. S6. Map the geometric confidence to residual gating weights, and according to the residual gating weights, gating and fusing the scale residuals and opacity residuals output by GCRP with the basic Gaussian parameters output by DCBP to obtain an initialized three-dimensional Gaussian field; wherein, the higher the geometric confidence, the stronger the correction effect of the prediction residuals on the basic Gaussian parameters, and the lower the geometric confidence, the higher the degree of preservation of the basic Gaussian parameters. S7. Perform differentiable projection and rasterization rendering on the initialized three-dimensional Gaussian field, compare the rendered image with the real image, and calculate the image reconstruction loss including L1 loss and SSIM loss. S7-1. Backpropagate the image reconstruction loss to update the position, scale, rotation, opacity, color, or spherical harmonic coefficients of the Gaussian ellipsoid; and prune or densify the Gaussian ellipsoid based on the gradient magnitude, opacity contribution, geometric confidence, and rendering error. S7-2. Input the target camera pose into the optimized 3D Gaussian field, and output the scene RGB image and depth map from the corresponding viewpoint.

2. The method according to claim 1, characterized in that, The DCBP uses a monocular depth estimation network to obtain monocular depth maps for each input viewpoint, and fits scale correction parameters and offset correction parameters based on the correspondence between the projection depth of 3D points in the sparse reconstructed point cloud at the corresponding viewpoint and the monocular depth value; the monocular depth estimation network uses Depth Anything V2.

3. The method according to claim 1, characterized in that, The DCBP backprojects the effective pixels in the aligned depth map to the three-dimensional space, transforms the candidate three-dimensional points generated from different perspectives to a unified world coordinate system, and obtains the initial point cloud through multi-view depth consistency screening, duplicate point fusion, downsampling, and outlier filtering; using the three-dimensional points in the initial point cloud as Gaussian centers, it constructs basic Gaussian parameters based on neighborhood distance, point density, visibility, and color information.

4. The method according to claim 1, characterized in that, The GCRP uses the FPN multi-scale feature extraction structure to obtain multi-scale image features from each input viewpoint. Using the initial 3D points generated by DCBP as feature query anchor points, the initial 3D points are projected onto the image feature maps of each viewpoint according to the camera intrinsic and extrinsic parameters. Image features at corresponding positions are sampled by bilinear interpolation and weighted aggregation is performed based on visibility, depth consistency and viewpoint weight to obtain point-level multi-view fusion features.

5. The method according to claim 1, characterized in that, The GCRP calculates geometric confidence based on at least two of the following: multi-view depth consistency, number of effective visible views, cross-view feature consistency, and local geometric stability. The point-level multi-view fusion features are then concatenated with 3D point coordinates, color, neighborhood distance statistics, local geometric features, and geometric confidence, and input into the MLP residual prediction module to obtain the corresponding scale residual and opacity residual.

6. The method according to claim 1, characterized in that, The cross-branch gated residual fusion in S6 includes: mapping the geometric confidence obtained by GCRP to residual gate weights, and using the residual gate weights to control the correction strength of the parameter residuals output by GCRP to the basic Gaussian parameters output by DCBP. For the i-th 3D Gaussian ellipsoid, the fundamental scale parameter output by DCBP is: The base opacity log-odds parameter of the DCBP output is: The scale residuals output by GCRP are The opacity residual output by GCRP is The geometric confidence score is mapped to the gate weights using a gate function. Then the scale and opacity parameters after fusion can be expressed as: in, Used to control the intensity of the effect of the prediction residual. When A larger value indicates higher geometric reliability of the Gaussian ellipsoid, leading to more use of residual correction from the parameter prediction network output; when... When the value is small, it indicates that the Gaussian ellipsoid may be located in an occlusion boundary, a depth error region, or a weak observation region, reducing the impact of the prediction residual on the final parameters and thus avoiding the amplification of erroneous parameters. This geometric confidence fusion mechanism can fully utilize the expressive power of the parameter prediction network, improve the robustness of the initialization stage to low confidence regions, and reduce floating Gaussian, background blurring and local geometric instability.