A method for sparse view 3D reconstruction

Abstract

The application provides a sparse view three-dimensional reconstruction method, comprising the following steps: S1, acquiring an original image set and a plurality of groups of primary original image pairs; S2, estimating a local three-dimensional point cloud map and a confidence map of each group of primary original image pairs based on a trained dense stereo model; S3, constructing a connected graph based on the confidence map set; S4, generating an optimized subgraph, and acquiring a high-level original image pair set; S5, acquiring each group of paired poses and scaling factors, and generating a global three-dimensional point cloud map; S6, generating camera internal and external parameters; and S7, taking the global three-dimensional point cloud map as an initial scene of a 3D Gaussian sputtering model, and optimizing 3D Gaussian attribute parameters to realize reconstruction of a target scene. Through the combination of the 3D Gaussian sputtering and the end-to-end dense stereo model, the application can efficiently reconstruct a high-quality 3D scene from sparse and non-pose images, and realize reconstruction of a target scene.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, specifically relating to a method for 3D reconstruction of sparse views. Background Technology

[0002] Significant progress has been made in 3D reconstruction technology, but it typically requires densely acquired multi-view data and accurate initial camera poses; however, this requirement limits its widespread application. Traditional 3D reconstruction methods decompose the task through modular processes, involving complex multi-stage mappings; for example, dense reconstruction based on 3D Gaussian sputtering relies on SfM preprocessing, whose process includes keypoint extraction, matching, scene graph construction, and incremental reconstruction. However, both SfM and 3D Gaussian sputtering (3DGS) require dense multi-view images to support feature matching and Gaussian point densification, but in sparse view scenes, accumulated errors often occur due to insufficient feature matching. Neural Radiation Field (NeRF) models the scene through MLP, but its training and inference speeds are limited, and it also relies excessively on pre-computed camera poses and dense view data.

[0003] Therefore, there is an urgent need for a method to achieve high-quality 3D reconstruction under sparse views, avoiding the dependence on dense viewpoints and precise camera poses in traditional methods. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method for three-dimensional reconstruction of sparse views, the method comprising:

[0005] S1 acquires multiple original images of different views within the target scene to obtain an original image set; and combines all the original images in the original image set in pairs to obtain multiple sets of primary original image pairs;

[0006] S2 acquires the trained dense stereo model and sequentially inputs each pair of primary original images into the trained dense stereo model to estimate two local 3D point cloud maps and two confidence maps for each pair of primary original images, thus obtaining the local 3D point cloud map set and the confidence map set.

[0007] S3 is based on a confidence graph set. It assigns the center point of each original image to a node, constructs an edge between the two corresponding nodes of each pair of primary original images, and converts the average confidence of each pair of primary original images into the weight value of each edge to construct a connected graph.

[0008] S4 uses the Kruskal algorithm to generate an optimized subgraph of the connected graph. Based on the optimized subgraph, it selects all primary original image pairs corresponding to the edge set with the smallest weight value to obtain a high-level original image pair set.

[0009] S5 sequentially acquires each pair of poses and scaling factors corresponding to each pair of high-level raw images, and based on each pair of poses and scaling factors, sequentially transforms each corresponding local 3D point cloud atlas to the global coordinate system to generate a global 3D point cloud map.

[0010] S6 calculates the average focal length based on the correspondence between the corresponding local 3D point cloud map and pixel coordinates of each set of high-level raw images, and generates camera intrinsic and extrinsic parameters based on the average focal length.

[0011] The S7 uses the global 3D point cloud map as the explicit initial scene for the 3D Gaussian sputtering model, and optimizes the 3D Gaussian attribute parameters based on the camera's intrinsic and extrinsic parameters to reconstruct the target scene.

[0012] Specifically, the dense stereo model in step S2 includes two identical sub-models; each sub-model includes a ViT encoder, a transformer decoder with cross-attention mechanism, and a regression head.

[0013] Specifically, the method for obtaining the trained dense 3D model in step S2 includes:

[0014] Multiple sets of primary sample image pairs are obtained, and each set of primary sample image pairs is enhanced to obtain multiple sets of advanced sample image pairs.

[0015] Two point cloud images and two corresponding sets of effective pixels are generated for each pair of high-level sample images using a multi-view reconstruction method.

[0016] The initial parameters of the dense stereo model are initialized using the weights of the CroCo pre-trained model. Then, each pair of high-level sample images is input into the dense stereo model in turn to estimate the two point cloud prediction maps and the confidence prediction map of each pair of high-level sample images.

[0017] Construct a first loss function to calculate the spatial distance between the predicted point cloud images and the actual point cloud images, and generate a distance loss value; construct a second loss function to calculate the confidence-weighted regression loss value of all valid pixels in each set of valid pixels based on the confidence prediction image.

[0018] The model is iteratively updated based on the distance loss value and the confidence-weighted regression loss value until the model converges, generating a trained dense stereo model.

[0019] Specifically, step S6 further includes:

[0020] S61 calculates the focal length of each camera and the average focal length by minimizing the error through the Weiszfeld algorithm based on the correspondence between the corresponding local 3D point cloud map and pixel coordinates of each set of high-level raw images.

[0021] S62 generates the global relative pose by solving the singular value decomposition method based on two point cloud prediction maps and confidence prediction maps for each high-level sample image pair.

[0022] S63 generates absolute pose based on global relative pose and average focal length, and generates camera intrinsic and extrinsic parameters.

[0023] Specifically, the 3D Gaussian sputtering model in step S7 includes a Gaussian point cloud distribution conversion unit, a differentiable Gaussian rasterization unit, and a decoupled appearance modeling unit.

[0024] Specifically, step S7 further includes:

[0025] S71 uses the global 3D point cloud map as the explicit initial scene for 3D Gaussian sputtering, and uses the Gaussian point cloud map distribution conversion unit to convert the global 3D point cloud map into a 3D Gaussian point cloud map distribution, and then into a 3D Gaussian image.

[0026] Based on the camera's intrinsic and extrinsic parameters, the S72 uses a differentiable Gaussian rasterization unit to project and render 3D Gaussian image blocks onto a 2D image plane to obtain a predicted image.

[0027] S73 uses a decoupled appearance modeling unit to generate a transformation map based on the predicted image, and then optimizes the appearance of the predicted image according to the transformation map to obtain a primary rendered image.

[0028] S74 constructs a third loss function, and iteratively updates the primary rendered image based on the third loss function, optimizing the 3D Gaussian attribute parameters until the loss function tends to stabilize, thus obtaining the advanced rendered image;

[0029] The S75 uses geometric information from advanced rendered images to reconstruct dense scenes, thereby achieving the reconstruction of the target scene.

[0030] Specifically, the 3D Gaussian attribute parameters mentioned in step S74 include the center point, covariance matrix, color information, and opacity.

[0031] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0032] (1) This invention combines sparse view images into image pairs and uses dense stereo models to directly predict pixel-level 3D point cloud maps, retaining more detailed information. It can still achieve the three-dimensional reconstruction task when only 2 to 5 sparse images are required for input.

[0033] (2) This invention reduces computational complexity by constructing a connected graph and using the Kruskal algorithm to filter out low-confidence pairs. It also converts independent relative poses into globally consistent poses by adjusting the point cloud map, transformation matrix, and scaling factor of each image pair. This aligns all point cloud maps with the point cloud map in the world coordinate system, achieving global consistency.

[0034] (3) By combining 3D Gaussian sputtering and end-to-end dense stereo model, this invention can efficiently reconstruct high-quality 3D scenes from sparse and poseless images, thereby achieving the reconstruction of the target scene. Attached Figure Description

[0035] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a three-dimensional reconstruction method for sparse views in Embodiment 1 of the present invention;

[0037] Figure 2 This is a schematic diagram of generating a local 3D point cloud atlas and a confidence atlas based on a dense 3D model in Embodiment 1 of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0039] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0040] Please see Figure 1 , Figure 1 This invention provides a method for three-dimensional reconstruction of sparse views according to Embodiment 1; the method includes:

[0041] S1 acquires multiple original images of different views within the target scene to obtain an original image set; and combines all the original images in the original image set in pairs to obtain multiple sets of primary original image pairs;

[0042] S2 acquires the trained dense stereo model and sequentially inputs each pair of primary original images into the trained dense stereo model. It then estimates two local 3D point cloud maps and two confidence maps for each pair of primary original images, thus obtaining the local 3D point cloud map set and the confidence map set.

[0043] In this embodiment of the invention, the dense stereo model includes two identical sub-models; wherein each sub-model includes a ViT encoder, a transformer decoder with a cross-attention mechanism, and a regression head.

[0044] Please see Figure 2 , Figure 2 This is a schematic diagram illustrating the generation of local 3D point cloud atlases and confidence atlases based on dense 3D models in Embodiment 1 of the present invention; each set of primary original images is paired. In the dense stereo model after training, the input is first encoded in a Siamese manner by a ViT encoder with the same weights, producing two tokens represented as F. 1 and F 2 .

[0045] F 1 =Encoder(I 1 )

[0046] F 2 =Encoder(I 2 )

[0047] Joint inference is performed between the two in the decoder. The transformer decoder with cross-attention is a general Transformer network equipped with cross-attention functionality. Therefore, each decoder block sequentially performs self-attention (each token of one primary raw image attends to tokens of the same primary raw image), then performs cross-attention (each token of one primary raw image attends to all other tokens of another primary raw image), and finally provides the tokens to the MLP. During decoder propagation, information is continuously shared between the two sub-models, with each decoder block attending to tokens from the other sub-model.

[0048]

[0049] Where i = 1, ..., B, for a decoder with B blocks, and using encoder symbols and initialization. G represents the i-th block in the submodel v∈{1,2}. 1 and G 2 It is the input symbol, where G 2 Symbols from another sub-model.

[0050]

[0051] In each sub-model, a separate regression head receives the decoder symbol set and outputs a local 3D point cloud map and a confidence map; ultimately, two local 3D point cloud maps X are obtained. 1,1 and X 2,1 Two confidence plots C 1,1 and C 2,1 .

[0052] In this embodiment of the invention, the specific method for obtaining the trained dense 3D model in step S2 includes:

[0053] Multiple sets of primary sample image pairs are obtained, and each set of primary sample image pairs is enhanced to obtain multiple sets of advanced sample image pairs.

[0054] In this embodiment of the invention, the dense stereo model is trained using a mixture of eight datasets: Habitat, ARKitScenes, MegaDepth, StaticScenes3D, BlendedMVS, ScanNet++, CO3Dv2, and Waymo. These datasets represent different scene types: indoor, outdoor, synthetic, real-world, object-centered, etc. Multiple pairs of primary sample images are extracted from each dataset. Standard data augmentation techniques, namely random color jitter and random centering cropping, are employed; the centered random cropping helps generate more focal points. The cropping is centered so that the principal point is always located at the center of the training pair. During training, each pair of high-level sample images, along with its inverse pair, is also provided to the model to aid generalization.

[0055] Two point cloud images Y for each high-level sample image pair z∈{1,2} are generated using a multi-view reconstruction method. 1,1 and Y 2,1 and the corresponding two sets of effective pixel sets D 1,1 ,

[0056] The initial parameters of the dense stereo model are initialized using the weights of the CroCo pre-trained model. Then, each pair of high-level sample images is input into the dense stereo model in turn to estimate the two point cloud prediction maps and the confidence prediction map for each pair of high-level sample images.

[0057] Cross-view completion (CroCo) is a recently proposed pre-trained paradigm that has proven to perform well on a variety of downstream 3D vision tasks.

[0058] In this embodiment of the invention, given camera intrinsic parameters Two point cloud images, real image Y1,1 and Y 2,1 You can directly get the actual depth map Obtained, such as Y 1,1 =K -1 [1H 1,2 ,2H 1,2 H 1,2 Y is expressed in the camera coordinate system. n ,m Y represents the point cloud image from camera n. n Expressed in the coordinate system of camera m:

[0059] Y n,m =Q m Q n g(Y n )

[0060] in G:(x,y,z)→(x,y,z,1) is the world to camera pose for images n and m, and g:(x,y,z)→(x,y,z,1) is the homogeneous mapping.

[0061] Two sets of effective pixel sets D 1,1 , Ground reality is defined on a pixel level. The effective pixel set i∈D for high-level sample images r∈{1,2}. r The regression loss is simply defined as the Euclidean distance:

[0062]

[0063] To address the scale ambiguity between predictions and ground reality, a scale factor z = norm(Y) is used. 1,1 ,Y 2,1 )and The normalized predicted point cloud and the actual ground view point cloud represent the average distance of all valid pixels to the origin, respectively:

[0064]

[0065] Construct a first loss function to calculate the spatial distance between the predicted point cloud images and the actual point cloud images, generating a distance loss value. Construct a second loss function to calculate the confidence-weighted regression loss value of all valid pixels in each set of valid pixels, based on the confidence prediction image. Iterate and update the model according to the distance loss value and the confidence-weighted regression loss value until the model converges, generating a trained dense stereo model.

[0066] In this embodiment of the invention, joint learning predicts a score for each pixel, which represents the network's confidence in that particular pixel. The final training target is the set of valid pixels i∈D. r Confidence-weighted regression loss value:

[0067]

[0068] in, Let be the confidence score of pixel i, and α be a hyperparameter controlling the regularization term. To ensure strictly positive confidence, define...

[0069] S3 is based on a confidence graph set. It assigns the center point of each original image to a node, constructs an edge between the two corresponding nodes of each pair of primary original images, and converts the average confidence of each pair of primary original images into the weight value of each edge to construct a connected graph.

[0070] S4 uses the Kruskal algorithm to generate an optimized subgraph of the connected graph. Based on the optimized subgraph, it selects all primary original image pairs corresponding to the edge set with the smallest weight value to obtain the advanced original image pair set.

[0071] Given all primary raw image pairs {I 1 ,I 2 ,...,I n First, construct a connected graph T(A,B), where N images form vertex A, and each edge b = (n,m) represents image I. n and I m Shared visual content. Optimized subgraphs of the pass graph are generated using the Kruskal algorithm, and their overlap is measured based on the average confidence of the pairs. Low-confidence pairs are then filtered out to obtain a high-level set of original image pairs.

[0072] S5 sequentially acquires each pair of poses and scaling factors corresponding to each pair of high-level raw images, and based on each pair of poses and scaling factors, sequentially transforms each corresponding local 3D point cloud atlas to the global coordinate system to generate a global 3D point cloud map.

[0073] In this embodiment of the invention, a connection graph T(A,B) is used to recover the globally aligned point cloud. For all cameras n = 1, ..., N, first predict the pairwise point cloud X for each pair of high-level original images e = (n, m) ∈ ε. n,m ,X m,n and its associated confidence plot C n,m C m,n .

[0074] Define X m,e :=X m，n and X n,e :=X n,n The goal is to rotate all pairwise predictions into a common coordinate system, introducing pairwise poses for each pair e∈ε. and scaling σ e>0. Then, the following optimization problem is defined:

[0075]

[0076] If e = (n, m), then r ∈ e is taken as r ∈ {n, m}. For a given pair of e, the same rigid transformation γ e The point cloud map X should be aligned. n,e and X m,e With global coordinate point cloud map X n and X m Because X n,e and X m,e By definition, coordinate systems are represented in the same form. To avoid... The trivial solution, forcibly setting Πeσ e =1.

[0077] In this embodiment of the invention, step S6 further includes:

[0078] S61 calculates the focal length of each camera and the average focal length by minimizing the error through the Weiszfeld algorithm based on the correspondence between the corresponding local 3D point cloud map and pixel coordinates of each set of high-level raw images.

[0079] Nearest neighbor (NN) search is performed in the 3D point cloud graph space to establish the correspondence between pixels of two images in the high-level original image pair. Image I is preserved. 1 and I 2 The correspondence between them M 1,2 That is:

[0080]

[0081] Point cloud map X 1，1 in I 1 The coordinates are represented in a coordinate system. Therefore, the camera intrinsic parameters are estimated through a simple optimization problem. Assuming the principal point is approximately located at the center and the pixels are squares, it is only necessary to estimate the focal length of each camera.

[0082]

[0083] Where i' = iW / 2 and j' = jW / 2. The optimal algorithm is found through several iterations based on the Weiszfeld algorithm. Regarding the focal length of the second camera The simplest choice is to (I) 2 ,I 1 ) make inferences and use the above formula, where X 2,2 Replace X 1,1 .

[0084] S62 generates the global relative pose by solving the singular value decomposition method based on two point cloud prediction maps and confidence prediction maps for each high-level sample image pair.

[0085] S63 generates absolute pose based on global relative pose and average focal length, and generates camera intrinsic and extrinsic parameters.

[0086] In this embodiment of the invention, the global relative pose is generated by singular value decomposition, and the point cloud map is aligned and compared using Procrustes. (or equivalently, ), to obtain the relative attitude γ * =[R * |t * ]:

[0087]

[0088] Let I μ Indicates a query image, I β This represents a reference image from which a 2D-3D correspondence can be obtained. First, starting from X... μ,μ Estimate I μ Internal parameters. Obtain I μ and I β The 2D correspondence between them, which in turn generates I β The 2D-3D correspondence is determined, and then the PnP-RANSAC algorithm is run to obtain the absolute pose.

[0089] The S7 uses the global 3D point cloud map as the explicit initial scene for the 3D Gaussian sputtering model, and optimizes the 3D Gaussian attribute parameters based on the camera's intrinsic and extrinsic parameters to reconstruct the target scene.

[0090] In this embodiment of the invention, the 3D Gaussian sputtering model in step S7 includes a Gaussian point cloud distribution conversion unit, a differentiable Gaussian rasterization unit, and a decoupled appearance modeling unit.

[0091] In this embodiment of the invention, step S7 further includes:

[0092] The S71 uses the global 3D point cloud map as the explicit initial scene for 3D Gaussian sputtering, and uses the Gaussian point cloud map distribution conversion unit to convert the global 3D point cloud map into a 3D Gaussian point cloud map distribution, and then into a 3D Gaussian image.

[0093] In this embodiment of the invention, each global 3D point cloud map is sequentially mapped as an explicit initial scene to the 3D Gaussian point cloud distribution space as a 3D Gaussian sputtering, generating multiple sets of 3D Gaussian point cloud distributions Φ. i The parameters of a 3D Gaussian point cloud distribution include the center point coordinates μ, the covariance matrix ∑, the color value Y, and the opacity A. The 3D Gaussian point cloud Φi (x) is represented in three-dimensional space by the three-dimensional covariance matrix ∑ and the coordinates of the center point τ:

[0094]

[0095] Where x is the coordinate of a point in three-dimensional space; the three-dimensional covariance matrix ∑=OSS T R T , where O is the rotation matrix and S is the scaling matrix.

[0096] Based on the camera's intrinsic and extrinsic parameters, the S72 uses a differentiable Gaussian rasterization unit to project and render 3D Gaussian image blocks onto a 2D image plane to obtain a predicted image.

[0097] In this embodiment of the invention, each 3D Gaussian image is projected onto the 2D image plane corresponding to each camera pose, resulting in 2D Gaussian point clouds corresponding to different camera poses; specifically, for a camera pose... The projection transformation matrix from world coordinates to camera coordinates is φ, which projects the primary 3D Gaussian point cloud distribution corresponding to the primary 3D Gaussian image onto the camera pose. Calculate the coordinates of the corresponding 2D image plane in the projected camera coordinate system. and covariance matrix for:

[0098]

[0099] Where J is the Jacobian matrix of the projection transformation, used to convert a 3D vector into a 2D vector, thereby achieving a projection transformation from the world coordinate system to the camera coordinate system. The purpose of this transformation is to transform a 3D Gaussian image in the world coordinate system to a 2D image plane space in the camera coordinate system.

[0100] Next, the 2D Gaussian point cloud is subjected to differentiable Gaussian rasterization. In the differentiable rasterization process, the 2D image plane is divided into multiple pixel regions. Each pixel region is responsible for processing the 2D Gaussian point cloud projected onto its range. Spatial hashing or grid indexing is used to quickly determine the subset of 2D Gaussian point cloud corresponding to each pixel region. Within each pixel region, the Gaussian points are sorted from farthest to nearest according to their depth values ​​in the camera coordinate system to ensure the correct blending order in subsequent steps. For each pixel, the color values ​​and opacities of all 2D Gaussian point clouds covering that pixel are blended using alpha in depth order to obtain a primary prediction image of size σW×βH. The feature value f of each pixel in the primary prediction image w,h Calculated using AlphaBlending, the formula is as follows:

[0101]

[0102] Where, η i =A i Φ(x), U w,h For this pixel, it is a subset of the 2D Gaussian point cloud; F i For U w,h The eigenvalue of the i-th 2D Gaussian point cloud; η i For U w,h The opacity of the i-th 2D Gaussian point cloud; x is the coordinate of the point in 3D space; φ i For U w,h The coordinates of the center point of the i-th 2D Gaussian point cloud.

[0103] S73 uses a decoupled appearance modeling unit to generate a transformation map based on the predicted image, and then optimizes the appearance of the predicted image according to the transformation map to obtain a primary rendered image.

[0104] In this embodiment of the invention, appearance optimization of the predicted image is performed by decoupling the appearance modeling unit. The aim is to eliminate appearance inconsistencies (such as brightness differences and color casts) caused by changes in lighting and camera exposure, thereby generating a visually consistent rendered image. First, each predicted image is sequentially... Downsampling to a low resolution yields a downsampled image; the size of the downsampled image is [value], and an appearance embedding of length t is introduced. Then embed the appearance The image is stitched pixel-by-pixel to the 3-channel downsampled image to obtain the embedded image ω with 3+t channels. i ;

[0105]

[0106] Then, sequentially, each embedded image ω i Inputting into a convolutional neural network (CNN), the CNN network progressively processes ω i Perform upsampling to generate and Multiple primary transformation maps ρ with the same resolution i Finally, ρ is used sequentially. i For each corresponding predicted image After performing appearance optimization, multiple primary rendered images were obtained.

[0107] S74 constructs a third loss function, and iteratively updates the primary rendered image based on the third loss function, optimizing the 3D Gaussian attribute parameters until the loss function tends to stabilize, thus obtaining the advanced rendered image.

[0108] In this embodiment of the invention, the third loss function in step S7 is:

[0109] L c =(1-θ)L c +θL D-SSIM

[0110] Among them, L c L is the difference loss function, used to calculate the absolute difference in pixel values ​​between the predicted image and the original image; D-SSIM θ is the D-SSIM loss function, used to calculate the structural similarity difference between the predicted image and the original image; θ is the weight of the D-SSIM loss function. c The calculation formula is:

[0111]

[0112] Where n is the total number of predicted images, For the i-th predicted image, This corresponds to the original image. Ensure that the predicted image and the original image are aligned in color and brightness.

[0113] L D-SSIM The calculation formula is:

[0114]

[0115] The Structural Similarity Index (SSIM) measures the consistency between the predicted image and the real image in terms of texture, contrast, and structure, thus avoiding the loss of geometric details.

[0116] The S75 uses geometric information from advanced rendered images to reconstruct dense scenes, thereby achieving the reconstruction of the target scene.

[0117] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for three-dimensional reconstruction of sparse views, characterized in that, The method includes: S1 acquires multiple original images of different views within the target scene to obtain an original image set; and combines all the original images in the original image set in pairs to obtain multiple sets of primary original image pairs; S2 acquires the trained dense stereo model and sequentially inputs each pair of primary original images into the trained dense stereo model to estimate two local 3D point cloud maps and two confidence maps for each pair of primary original images, thus obtaining the local 3D point cloud map set and the confidence map set. S3 is based on a confidence graph set. It assigns the center point of each original image to a node, constructs an edge between the two corresponding nodes of each pair of primary original images, and converts the average confidence of each pair of primary original images into the weight value of each edge to construct a connected graph. S4 uses the Kruskal algorithm to generate an optimized subgraph of the connected graph. Based on the optimized subgraph, it selects all primary original image pairs corresponding to the edge set with the smallest weight value to obtain a high-level original image pair set. S5 sequentially acquires each pair of poses and scaling factors corresponding to each pair of high-level raw images, and based on each pair of poses and scaling factors, sequentially transforms each corresponding local 3D point cloud atlas to the global coordinate system to generate a global 3D point cloud map. S6 calculates the average focal length based on the correspondence between the corresponding local 3D point cloud map and pixel coordinates of each set of high-level raw images, and generates camera intrinsic and extrinsic parameters based on the average focal length. The S7 uses the global 3D point cloud map as the explicit initial scene for the 3D Gaussian sputtering model, and optimizes the 3D Gaussian attribute parameters based on the camera's intrinsic and extrinsic parameters to reconstruct the target scene.

2. The method according to claim 1, characterized in that, The dense stereo model described in step S2 includes two identical sub-models; each sub-model includes a ViT encoder, a transformer decoder with cross-attention mechanism, and a regression head.

3. The method according to claim 1, characterized in that, The specific method for obtaining the trained dense 3D model in step S2 includes: Multiple sets of primary sample image pairs are obtained, and each set of primary sample image pairs is enhanced to obtain multiple sets of advanced sample image pairs. Two point cloud images and two corresponding sets of effective pixels are generated for each pair of high-level sample images using a multi-view reconstruction method. The initial parameters of the dense stereo model are initialized using the weights of the CroCo pre-trained model. Then, each pair of high-level sample images is input into the dense stereo model in turn to estimate the two point cloud prediction maps and the confidence prediction map of each pair of high-level sample images. Construct a first loss function to calculate the spatial distance between the predicted point cloud images and the actual point cloud images, and generate a distance loss value; construct a second loss function to calculate the confidence-weighted regression loss value of all valid pixels in each set of valid pixels based on the confidence prediction image. The model is iteratively updated based on the distance loss value and the confidence-weighted regression loss value until the model converges, generating a trained dense stereo model.

4. The method according to claim 1, characterized in that, Step S6 further includes: S61 calculates the focal length of each camera and the average focal length by minimizing the error through the Weiszfeld algorithm based on the correspondence between the corresponding local 3D point cloud map and pixel coordinates of each set of high-level raw images. S62 generates the global relative pose by solving the singular value decomposition method based on two point cloud prediction maps and confidence prediction maps for each high-level sample image pair. S63 generates absolute pose based on global relative pose and average focal length, and generates camera intrinsic and extrinsic parameters.

5. The method according to claim 1, characterized in that, The 3D Gaussian sputtering model described in step S7 includes a Gaussian point cloud distribution conversion unit, a differentiable Gaussian rasterization unit, and a decoupled appearance modeling unit.

6. The method according to claim 5, characterized in that, Step S7 further includes: S71 uses the global 3D point cloud map as the explicit initial scene for 3D Gaussian sputtering, and uses the Gaussian point cloud map distribution conversion unit to convert the global 3D point cloud map into a 3D Gaussian point cloud map distribution, and then into a 3D Gaussian image. Based on the camera's intrinsic and extrinsic parameters, the S72 uses a differentiable Gaussian rasterization unit to project and render 3D Gaussian image blocks onto a 2D image plane to obtain a predicted image. S73 uses a decoupled appearance modeling unit to generate a transformation map based on the predicted image, and then optimizes the appearance of the predicted image according to the transformation map to obtain a primary rendered image. S74 constructs a third loss function, and iteratively updates the primary rendered image based on the third loss function, optimizing the 3D Gaussian attribute parameters until the loss function tends to stabilize, thus obtaining the advanced rendered image; The S75 uses geometric information from advanced rendered images to reconstruct dense scenes, thereby achieving the reconstruction of the target scene.

7. The method according to claim 5, characterized in that, The 3D Gaussian attribute parameters mentioned in step S74 include the center point, covariance matrix, color information, and opacity.

Images