Sparse view 3d reconstruction method based on patch-wise geometric alignment and texture decoupling

Abstract

This invention discloses a sparse-view 3D reconstruction method based on block-based geometric alignment and texture decoupling, belonging to the field of computer vision and 3D reconstruction technology. The method includes: acquiring RGB images and prior depth maps from a sparse viewpoint, and initializing a 3D Gaussian sputtering model; eliminating monocular depth local scale drift through block-based local alignment and constructing a second-order normal loss to constrain geometric consistency; achieving adaptive smoothing regularization based on texture-structure decoupling weights guided by normal gradients; generating artifact masks through depth residual analysis, applying soft penalties and performing hard pruning on anomalous Gaussian primitives; jointly optimizing color reprojection loss and geometric, smoothing, and penalty losses, iteratively updating Gaussian parameters, and achieving high-fidelity rendering of the new viewpoint. This invention employs the aforementioned sparse-view 3D reconstruction method based on block-based geometric alignment and texture decoupling, solving the problems of geometric instability, texture-structure coupling interference, and floating artifacts under sparse viewpoints.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and 3D reconstruction technology, and in particular to a sparse viewpoint 3D reconstruction method based on block geometry alignment and texture decoupling. Background Technology

[0002] With the development of 3D representation technology, 3D Gaussian Splatting has become an important means of scene reconstruction due to its superior rendering speed and quality. However, under sparse viewpoint conditions, this technology faces serious geometric instability problems due to the lack of sufficient multi-view observation constraints. Although existing methods attempt to introduce monocular depth priors to constrain macroscopic structures, the following technical bottlenecks still exist in achieving high-fidelity novel viewpoint synthesis: The first limitation is the non-rigid scale ambiguity of monocular depth: while monocular depth prediction models can recover relative depth, they lack absolute scale constraints. Existing global affine alignment methods assume that the entire image shares a uniform scale mapping, but in complex scenes, depth deviations often exhibit non-linear distortions with spatial distribution. This rigid correction cannot correct local non-rigid distortions, resulting in a lack of smoothness at the microscopic level in the reconstructed surface.

[0003] The second issue is the lack of higher-order geometric property constraints: existing optimization frameworks are essentially fitting a scalar depth field, while rendering quality is highly dependent on the continuity of surface normals. Tiny noise in the depth values ​​is amplified during the differentiation process, causing high-frequency oscillations in the normal field, resulting in non-physical brightness fluctuations and wave effects in the rendering results when the lighting changes.

[0004] The third issue is the coupling interference between texture and geometry: traditional edge-aware smoothing strategies typically rely on color gradients to identify geometric edges. However, when high-frequency textures exist on the object's surface, the algorithm may mistakenly identify pure texture regions as geometric edges and stop smoothing, causing the Gaussian point cloud to produce false geometric undulations in order to fit the color.

[0005] The fourth issue is the floating artifact problem under sparse perspectives: In regions with weak geometric constraints, 3DGS easily generates a large number of geometrically unreasonable Gaussian ellipses. Although these primitives can be preserved by fitting the color error of the training perspective, they appear as floating, hazy artifacts under new perspectives. Existing methods mostly rely on heuristic rules to suppress artifacts and lack a systematic removal mechanism based on geometric consistency. Summary of the Invention

[0006] The purpose of this invention is to provide a sparse perspective 3D reconstruction method based on block geometry alignment and texture decoupling, which overcomes the problems of geometric instability, texture and structure coupling interference, and floating artifacts in existing sparse perspective 3D reconstruction methods, and realizes high-quality and highly realistic new perspective synthesis and geometric reconstruction under sparse input conditions.

[0007] To achieve the above objectives, this invention provides a sparse-view 3D reconstruction method based on block geometry alignment and texture decoupling, comprising the following steps: S1. Obtain RGB images and corresponding camera intrinsic parameters from sparse viewpoints. Use a pre-trained monocular depth network to infer from the RGB images to obtain an initial prior depth map with macroscopic scene structure. Simultaneously, a 3D Gaussian sputtering model is initialized, and the corresponding rendered image and initial rendered depth map are obtained through differentiable rendering. ; S2. Based on the pixel space corresponding to the RGB image obtained in step S1, define the image domain. Image domain Divided into Non-overlapping local blocks ( (representing the index of a local block), for any pixel within each local block. Initial prior depth map Compared with the initial rendered depth map Local normalization is performed (to eliminate local scale drift and nonlinear distortion); the 3D point cloud is obtained by backprojection from the locally normalized depth map, and the tangent vector is calculated to derive the a priori pseudo-normal field reflecting the local curvature. With the rendering pseudo-normal field reflecting the surface orientation A block-based second-order normal loss is constructed based on the cosine similarity of the pseudo-normal phase field. And in conjunction with global first-order consistency loss Constituting the final geometric loss ; S3. Introduce the gradient of the prior normal graph. As a measure of geometrical changes, a texture-structure decoupling weight function is constructed. ;Utilizing texture-structure decoupling weighting functions based on geometric edge thresholds It intelligently distinguishes texture details from real geometric edges in color gradients; it constructs an adaptive depth smoothing loss function by combining second-order depth constraints and first-order normal constraints. This achieves enhanced smoothness in flat textured areas while preserving structure at geometric edges; S4. Initial prior depth map Compared with the initial rendered depth map Global standardization is performed separately to obtain a fully standardized rendering depth map. Standardized prior depth map with full map Calculate the normalized rendering depth map of the entire image. Standardized prior depth map with full map The absolute difference is used to generate a depth residual map. Set residual threshold and far-plane cutoff threshold Identify geometrically anomalous regions and generate a binarized artifact mask. Apply a soft opacity penalty loss to the Gaussian elements projected onto the mask region. During the densification iteration process, hard pruning is performed on primitives that meet the opacity threshold condition and whose depth residual is consistently higher than the set residual threshold. S5. Construct a structure including color reprojection loss and final geometric loss. Smoothing loss function and Opacity Soft Penalty Loss Total loss function The Gaussian parameters are updated iteratively through multi-constraint collaborative optimization, and the trained Gaussian primitives are finally used for image rendering and geometry generation from any new perspective.

[0008] Preferably, the pre-trained monocular depth network in step S1 is Depth Anything V2.

[0009] Preferably, step S2 specifically includes: S21. Local normalization processing: This involves normalizing the image domain... Divided into Non-overlapping local blocks For each local block any pixel in For the initial prior depth map Compared with the initial rendered depth map Perform local standardization; the local standardization formula is: ; in, Represents pixels in the initial prior depth map or initial rendered depth map after local normalization. Depth value at that location, This represents the pixels in the initial prior depth map or initial rendered depth map before local normalization. The original depth value at that location, Indicates the index of a local block. This represents the average depth within the local block. For depth standard deviation, To prevent numerical instability of small constants; S22. Pseudo-normal field construction: Based on the depth map after local normalization, a 3D point cloud is obtained by backprojection using camera intrinsic parameters. The tangent vectors in the horizontal and vertical directions are approximated using the central difference method. and The unit pseudo-normal vector is obtained by normalizing the cross product of the tangent vectors. : ; in, The L2 norm is used to derive the a priori pseudonormal field reflecting the local curvature. With the rendering pseudo-normal field reflecting the surface orientation ; S23. Construction of Block-based Second-Order Normal Loss: Based on the cosine similarity of the pseudo-normal field, construct the block-based second-order normal loss. : ; in, This represents the unit rendering pseudo-normal vector at pixel x. This represents the unit prior pseudo-normal vector at pixel x. Represents the L1 norm; S24. Final geometric loss construction: combining global first-order consistency loss. Obtain the final geometric loss : ; in, and These are the weighting coefficients.

[0010] Preferably, in step S21, the number of local blocks The downsampling factor of the original image is taken in the range of 16 to 64.

[0011] Preferably, in step S21, the average depth and depth standard deviation The calculation formulas are as follows: ; ; in, Represents a local block The total number of pixels within.

[0012] Preferably, the global first-order consistency loss in step S24 Calculate according to the following formula: ; in, This represents a normalized rendering depth map of the entire image. This represents the standardized prior depth map of the entire map.

[0013] Preferably, step S3 specifically includes: S31. Texture-structure decoupling weight construction: Introducing the gradient of the prior normal graph. As a measure of geometrical changes, a texture-structure decoupling weight function is constructed. : ; in, For the input image in pixels The color, For indicator functions, Geometric edge threshold, and To control the influence of color and geometric information on the decoupling weights, This represents an exponential function with base e. Represents pixels The gradient of the unit prior pseudo-normal vector at the given location; S32. Construction of Adaptive Depth Smoothing Loss: Based on the texture-structure decoupling weight function, and combined with second-order depth constraints and first-order normal constraints, an adaptive depth smoothing loss function is constructed. : ; in, The Laplacian operator is used to constrain abrupt changes in curvature. and These are the depth and normal smoothing weights, respectively. Represents pixels The gradient of the pseudo-normal vector at the unit rendering point. Represents pixels in the initial rendered depth map after local normalization. The depth value at that location.

[0014] Preferably, step S4 specifically includes: S41. Initial prior depth map Compared with the initial rendered depth map Global standardization is performed separately to obtain a fully standardized rendering depth map. Standardized prior depth map with full map Calculate the normalized rendering depth map of the entire image. Standardized prior depth map with full map The absolute difference is used to generate a depth residual map. : ; S42. Set residual threshold and far-plane cutoff threshold Identify geometrically anomalous regions and generate a binarized artifact mask. : ; in, Represents pixels in the initial rendered depth map before local normalization. The original depth value at that location, Used to identify geometrically anomalous regions with significant depth residuals. Used to exclude invalid, extremely distant background regions; S43. Apply a soft opacity penalty to the Gaussian elements projected within the mask area using the following formula. : ； in, Accumulated opacity per pixel This represents the total number of pixels within the mask area; S44. During the densification iteration, hard pruning is performed on primitives that meet the opacity threshold condition and whose depth residual is consistently higher than the set residual threshold.

[0015] Preferably, in step S42, the residual threshold Set to 3 times the global standard deviation: Far-plane truncation threshold Set as the far clipping plane in the camera parameters.

[0016] Preferably, step S5 specifically includes: S51, Combined with color reprojection loss Final geometric loss Smoothing loss function and Opacity Soft Penalty Loss Construct the total loss function : ; in, , , and Color reprojection loss Final geometric loss Smoothing loss function and Opacity Soft Penalty Loss Weight parameters; S52, By minimizing the total loss function Multi-constraint collaborative optimization iteratively updates Gaussian parameters; during training, every... In the next densification iteration, hard pruning is performed on Gaussian elements that are continuously in the artifact mask region and whose opacity is below a preset threshold until the model converges. S53. Input the trained Gaussian primitives into the differentiable rendering pipeline to achieve high-fidelity image rendering and geometry generation from any new perspective.

[0017] Therefore, the sparse perspective 3D reconstruction method based on block geometry alignment and texture decoupling described above has the following beneficial effects: (1) Solved the problem of scale inconsistency in monocular depth prior: This invention transforms complex global spatial nonlinear distortion into local linear alignment through block depth decomposition and standardization strategy, effectively eliminating local scale drift of monocular depth in complex scenes.

[0018] (2) Improved geometric consistency and realism of the reconstructed surface: This invention improves the constraint from a single depth value to a second-order normal dimension, explicitly constrains the surface curvature and orientation, and effectively suppresses the surface wave effect and non-physical brightness fluctuations that are common under sparse view.

[0019] (3) Effective decoupling of texture and structure is achieved: This invention uses prior normal gradient to guide smooth weights, which can accurately identify and protect real geometric edges, while suppressing pseudo geometric undulations in flat areas with rich textures, significantly enhancing the physical rationality of the reconstruction results.

[0020] (4) Significantly reduced floating artifacts under new perspectives: This invention automatically locates geometrically abnormal regions through depth residuals, and combines soft penalty and hard pruning mechanisms to effectively remove outlier Gaussian units generated in order to fit color errors, thereby improving the cleanliness of the rendering results.

[0021] (5) It takes into account both the stability of the overall structure and the precision of local details: Through the collaborative work of first-order global position constraints and multi-level local geometric guidance, this invention can significantly optimize the flatness and edge sharpness of the micro surface while ensuring the accuracy of the macro layout of the scene.

[0022] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0023] Figure 1 This is a flowchart of a method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall process of an embodiment of the present invention; Figure 3 This is a depth comparison image of 3DGS, DNGS, and the method of the present invention from three perspectives in some scenes on the LLFF dataset according to an embodiment of the present invention; Figure 4 This is a comparison of the rendering of 3DGS, DNGS, and the method of this invention from three perspectives in some scenes on the LLFF dataset according to embodiments of the present invention; Figure 5 This is a depth comparison image of 3DGS, DNGS and the method of the present invention in six perspectives in some scenes on the LLFF dataset according to the embodiments of the present invention; Figure 6 This is a comparison of the rendering of 3DGS, DNGS, and the method of this invention from six perspectives in some scenes on the LLFF dataset according to embodiments of the present invention. Detailed Implementation

[0024] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0025] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0026] Example like Figures 1-2 As shown, this embodiment provides a sparse viewpoint 3D reconstruction method based on block geometry alignment and texture decoupling, including the following steps: S1. Obtain RGB images and corresponding camera intrinsic parameters from sparse viewpoints. Use the Depth Anything V2 model to infer the RGB images and obtain an initial prior depth map with macroscopic scene structure. Simultaneously, a 3D Gaussian sputtering model is initialized, and the corresponding rendered image and initial rendered depth map are obtained through differentiable rendering. ; S2. Based on the pixel space corresponding to the RGB image obtained in step S1, define the image domain. Image domain Divided into Non-overlapping local blocks ( (representing the index of a local block), for any pixel within each local block. Initial prior depth map Compared with the initial rendered depth map Local normalization is performed (to eliminate local scale drift and nonlinear distortion); the 3D point cloud is obtained by backprojection from the locally normalized depth map, and the tangent vector is calculated to derive the a priori pseudo-normal field reflecting the local curvature. With the rendering pseudo-normal field reflecting the surface orientation A block-based second-order normal loss is constructed based on the cosine similarity of the pseudo-normal phase field. And in conjunction with global first-order consistency loss Constituting the final geometric loss Specifically: S21. Local normalization processing: This involves normalizing the image domain... Divided into Non-overlapping local blocks (Number of local blocks) Based on the downsampling factor of the original image (ranging from 16 to 64), for each local block any pixel in For the initial prior depth map Compared with the initial rendered depth map Perform local standardization; the local standardization formula is: ; in, Represents pixels in the initial prior depth map or initial rendered depth map after local normalization. Depth value at that location, This represents the pixels in the initial prior depth map or initial rendered depth map before local normalization. The original depth value at that location, Indicates the index of a local block. This represents the average depth within the local block. For depth standard deviation, To prevent numerical instability caused by small constants, the depth mean and depth standard deviation The calculation formulas are as follows: ; ; in, Represents a local block The total number of pixels within.

[0027] S22. Pseudo-normal field construction: Based on the depth map after local normalization, a 3D point cloud is obtained by backprojection using camera intrinsic parameters. The tangent vectors in the horizontal and vertical directions are approximated using the central difference method. and The unit pseudo-normal vector is obtained by normalizing the cross product of the tangent vectors. : ; in, The L2 norm is used to derive the a priori pseudonormal field reflecting the local curvature. With the rendering pseudo-normal field reflecting the surface orientation ; S23. Construction of Block-based Second-Order Normal Loss: Based on the cosine similarity of the pseudo-normal field, construct the block-based second-order normal loss. : ; in, Indicated in pixels The unit renders the pseudo normal vector at that location. Indicated in pixels The unit prior pseudo-normal vector at that location, Represents the L1 norm; S24. Final geometric loss construction: combining global first-order consistency loss. Obtain the final geometric loss : ; in, and Weighting coefficients (in this embodiment) 0.05, 0.1), global first-order consistency loss Calculate according to the following formula: ; in, This represents a normalized rendering depth map of the entire image. This represents the standardized prior depth map of the entire map.

[0028] S3. Introduce the gradient of the prior normal graph. As a measure of geometrical changes, a texture-structure decoupling weight function is constructed. ;Utilizing texture-structure decoupling weighting functions based on geometric edge thresholds It intelligently distinguishes texture details from real geometric edges in color gradients; it constructs an adaptive depth smoothing loss function by combining second-order depth constraints and first-order normal constraints. To achieve enhanced smoothness in flat texture areas while preserving structure at geometric edges, the following specific measures are taken: S31. Texture-structure decoupling weight construction: Introducing the gradient of the prior normal graph. As a measure of geometrical changes, a texture-structure decoupling weight function is constructed. : ; in, For the input image in pixels The color, For indicator functions, Geometric edge threshold, and To control the influence of color information and geometric information on the decoupling weights (in this embodiment) 10; 0.05; Set to 0.2. This represents an exponential function with base e. Represents pixels The gradient of the unit prior pseudo-normal vector at the given location; S32. Construction of Adaptive Depth Smoothing Loss: Based on the texture-structure decoupling weight function, and combined with second-order depth constraints and first-order normal constraints, an adaptive depth smoothing loss function is constructed. : ; in, The Laplacian operator is used to constrain abrupt changes in curvature. and These are the depth and normal smoothing weight coefficients, respectively (in this embodiment) ; ), Represents pixels The gradient of the pseudo-normal vector at the unit rendering point. Represents pixels in the initial rendered depth map after local normalization. The depth value at that location.

[0029] S4. Initial prior depth map Compared with the initial rendered depth map Global standardization is performed separately to obtain a fully standardized rendering depth map. Standardized prior depth map with full map Calculate the normalized rendering depth map of the entire image. Standardized prior depth map with full map The absolute difference is used to generate a depth residual map. ; Set residual threshold and far-plane cutoff threshold Identify geometrically anomalous regions (residual threshold) Set to 3 times the global standard deviation: Far-plane truncation threshold This embodiment sets the far clipping plane as the camera parameter. ), and generate a binary artifact mask. The expression is: ; in, Represents pixels in the initial rendered depth map before local normalization. The original depth value at that location, Used to identify geometrically anomalous regions with significant depth residuals. Used to exclude invalid, extremely distant background regions; Apply a soft opacity penalty loss to the Gaussian elements projected within the mask area using the following formula. : ; in, Accumulated opacity per pixel This represents the total number of pixels within the mask area; During the densification iteration, hard pruning is performed on primitives that meet the opacity threshold condition and whose depth residual is continuously higher than the set residual threshold. In this embodiment, the opacity threshold is set to 0.05. During the training process, a check is performed every 100 iterations. In this embodiment, if the set residual threshold is exceeded for three consecutive cycles (such as the 100th, 200th, and 300th checks), hard pruning is performed. S5. Construct a structure including color reprojection loss and final geometric loss. Smoothing loss function and Opacity Soft Penalty Loss Total loss function The Gaussian parameters are iteratively updated through multi-constraint collaborative optimization, and the trained Gaussian primitives are ultimately used for image rendering and geometry generation from arbitrary new perspectives. Specifically: S51, Combined with color reprojection loss Final geometric loss Smoothing loss function and Opacity Soft Penalty Loss Construct the total loss function : ; in, , , and Color reprojection loss Final geometric loss Smoothing loss function and Opacity Soft Penalty Loss Weight parameters; S52, By minimizing the total loss function Multi-constraint collaborative optimization iteratively updates Gaussian parameters; during training, every... In the next densification iteration, hard pruning is performed on Gaussian elements that are continuously in the artifact mask region and whose opacity is below a preset threshold until the model converges. S53. Input the trained Gaussian primitives into the differentiable rendering pipeline to achieve high-fidelity image rendering and geometry generation from any new perspective.

[0030] This embodiment evaluates the performance of the present invention and current mainstream methods based on NeRF and 3DGS and their improvements on multiple datasets. The experimental design aims to verify the ability of the present method to overcome scale distortion and surface distortion by using a block geometry guidance strategy under extremely sparse view input conditions. The experiments cover complex indoor scenes and fine object structures. As shown in Tables 1 and 2, the comparative experimental results of the present method and existing sparse view reconstruction techniques on the LLFF, MipNeRF and Tanks and Temples datasets under three and six views are presented.

[0031] Table 1. Comparison results of different datasets from three perspectives

[0032] Table 2 Comparison results of different datasets from six perspectives

[0033] Data shows that even in extreme cases with only three viewpoints, our method maintained its leading performance across various scenarios. For example, in the LLFF dataset, the PSNR reached 21.01 dB, exceeding the benchmark MipNeRF by more than 5 dB and outperforming the DNGS model, which is an improvement on 3DGS. This is mainly because everyday scenes like LLFF contain many uneven objects, and global alignment cannot handle the local stretching deformation of monocular depth. Our method corrects these local distortions through block alignment. In the texture-dense MipNeRF dataset, our method achieved a minimum LPIPS of 0.169 across six viewpoints, significantly better than MipNeRF's 0.320. This is because traditional methods cannot distinguish between color transitions and geometric edges, easily creating artificial pits and bumps on flat tile or table textures. Our method introduces normal gradient constraints, preserving edges only at true geometric transitions, resulting in smoother and more realistic rendered images.

[0034] Even with six viewpoints, our method remains robust in large scenes like Tanks and Temples, significantly outperforming the baseline model in both PSNR and SSIM. Furthermore, experimental data shows that the SSIM score drops noticeably in large scenes with sparse viewpoints. This is because Gaussian point clouds blindly fit color errors. Our method's depth residual mechanism identifies and prunes these geometrically illogical and isolated points based on the residual map, resulting in a cleaner final image and greatly reducing hover artifacts, such as depth artifacts. Figure 3 and depth Figure 5As shown in the diagram. The ablation experiments (Tables 3 and 4) also reveal that removing the block guide leads to inaccurate depth alignment, causing a direct drop in performance metrics; removing texture structure decoupling blurs the edges; and removing artifact suppression makes the image appear dirty. This demonstrates that all three modules are indispensable in addressing the three core issues of scale, detail, and artifacts.

[0035] Furthermore, to qualitatively evaluate the effect of block geometry-guided 3D Gaussian splashing under a sparse viewpoint, this method performs visualization rendering on the results. Figure 4 and Figure 6 It can be observed that this method, by correcting local non-rigid distortions, makes the main subject of the image closer to the real object, and the display of edge details is more refined. Due to the introduction of texture-structure decoupling and the artifact suppression mechanism, it can be clearly seen that in the rendered image of the small tower, the artifacts around it and the floating objects in the center of the image are significantly reduced.

[0036] Tables 3 and 4 present ablation comparison experiments on three benchmark datasets under three and six viewpoints, respectively, demonstrating the specific contributions of each module to the sparse viewpoint reconstruction performance: Table 3. Ablation comparison experimental results from three perspectives on different datasets.

[0037] Table 4. Ablation comparison experimental results from six perspectives on different datasets.

[0038] Experimental results show that removing any core module leads to a significant performance drop. For example, in the LLFF scenario, without block geometry guidance, the PSNR drops from 21.01dB to 20.53dB, reflecting that correcting local distortions in the monocular depth prior is crucial for stabilizing the underlying geometry when there are very few viewpoints. On the MipNeRF dataset, which is rich in texture details, removing texture structure decoupling worsens the perceptual loss LPIPS from 0.190 to 0.209, indicating that color gradients alone cannot distinguish between texture and structure, and normal gradients must be introduced to protect true geometric edges. In the Tanks and Temples scenario, removing the artifact suppression mechanism reduces the PSNR by about 0.40dB, confirming that this module is very effective in identifying and removing outliers.

[0039] This performance dependency remains robust even when the number of viewpoints increases to six. The PSNR drops by 0.58 dB after removing block guidance on the LLFF dataset, indicating that even with an increased number of viewpoints, the problem of local scale drift still requires block alignment to resolve. On MipNeRF, the complete scheme achieves an optimal LPIPS of 0.169, which degrades to 0.178 once the decoupling mechanism is removed, further demonstrating the value of second-order geometric constraints for high-fidelity rendering. Similarly, in the large scenes of Tanks and Temples, removing artifact suppression causes the PSNR to drop from 21.86 dB to 21.62 dB. In summary, the three core modules proposed in this embodiment have significant synergistic effects in improving geometric consistency, enhancing structural decoupling, and eliminating levitation artifacts. The complete invention maintains optimal performance in PSNR, SSIM, and LPIPS, verifying the systematic nature and robustness of this method in high-fidelity reconstruction with sparse viewpoints.

[0040] Therefore, this invention employs the aforementioned sparse-view 3D reconstruction method based on block geometry alignment and texture decoupling. By guiding with block geometry and decoupling texture and structure, it significantly improves the 3D reconstruction quality under sparse viewpoints. It effectively solves the local scale drift problem of monocular depth prior, enhances the geometric consistency and smoothness of the reconstructed surface, and suppresses surface wavy effects and non-physical brightness fluctuations. Simultaneously, through a depth residual-guided artifact suppression mechanism, it greatly reduces floating artifacts under new viewpoints, achieving precise decoupling of texture and geometric structure.

[0041] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A sparse-view 3D reconstruction method based on block geometry alignment and texture decoupling, characterized in that, Includes the following steps: S1. Obtain RGB images and corresponding camera intrinsic parameters from sparse viewpoints. Use a pre-trained monocular depth network to infer from the RGB images to obtain an initial prior depth map with macroscopic scene structure. Simultaneously, a 3D Gaussian sputtering model is initialized, and the corresponding rendered image and initial rendered depth map are obtained through differentiable rendering. ; S2. Based on the pixel space corresponding to the RGB image obtained in step S1, define the image domain. Image domain Divided into Each local block is a non-overlapping local block, and for any pixel within each local block... Initial prior depth map Compared with the initial rendered depth map Local normalization is performed; the 3D point cloud is obtained by backprojection from the locally normalized depth map, and the tangent vector is calculated. The a priori pseudo-normal field reflecting the local curvature is then derived. With the rendering pseudo-normal field reflecting the surface orientation ; Constructing a block-based second-order normal loss based on the cosine similarity of the pseudo-normal phase field. And in conjunction with global first-order consistency loss Constituting the final geometric loss ; S3. Introduce the gradient of the prior normal graph. As a measure of geometrical changes, a texture-structure decoupling weight function is constructed. ; Using texture-structure decoupling weight function based on geometric edge threshold It intelligently distinguishes texture details from real geometric edges in color gradients; it constructs an adaptive depth smoothing loss function by combining second-order depth constraints and first-order normal constraints. ; S4. Initial prior depth map Compared with the initial rendered depth map Global standardization is performed separately to obtain a fully standardized rendering depth map. Standardized prior depth map with full map Calculate the normalized rendering depth map of the entire image. Standardized prior depth map with full map The absolute difference is used to generate a depth residual map. Set residual threshold and far-plane cutoff threshold And generate a binary artifact mask. Apply a soft opacity penalty loss to the Gaussian elements projected onto the mask region. During the densification iteration process, hard pruning is performed on primitives that meet the opacity threshold condition and whose depth residual is consistently higher than the set residual threshold. S5. Construct a structure including color reprojection loss and final geometric loss. Smoothing loss function and Opacity Soft Penalty Loss Total loss function The Gaussian parameters are updated iteratively through multi-constraint collaborative optimization, and the trained Gaussian primitives are finally used for image rendering and geometry generation from any new perspective.

2. The sparse viewpoint 3D reconstruction method based on block geometry alignment and texture decoupling according to claim 1, characterized in that, The pre-trained monocular deep network in step S1 is Depth Anything V2.

3. The sparse viewpoint 3D reconstruction method based on block geometry alignment and texture decoupling according to claim 1, characterized in that, Step S2 specifically includes: S21. Local normalization processing: This involves normalizing the image domain... Divided into Non-overlapping local blocks For each local block any pixel in For the initial prior depth map Compared with the initial rendered depth map Perform local standardization; the local standardization formula is: ； in, Represents pixels in the initial prior depth map or initial rendered depth map after local normalization. Depth value at that location, This represents the pixels in the initial prior depth map or initial rendered depth map before local normalization. The original depth value at that location, Indicates the index of a local block. This represents the average depth within a local block. For depth standard deviation, To prevent numerical instability of small constants; S22. Pseudo-normal field construction: Based on the depth map after local normalization, a 3D point cloud is obtained by backprojection using camera intrinsic parameters. The tangent vectors in the horizontal and vertical directions are approximated using the central difference method. and The unit pseudo-normal vector is obtained by normalizing the cross product of the tangent vectors. : ； in, The L2 norm is used to derive the a priori pseudonormal field reflecting the local curvature. With the rendering pseudo-normal field reflecting the surface orientation ; S23. Construction of Block-based Second-Order Normal Loss: Based on the cosine similarity of the pseudo-normal field, construct the block-based second-order normal loss. : ； in, Indicated in pixels The unit renders the pseudo normal vector at that location. Indicated in pixels The unit prior pseudo-normal vector at that location, Represents the L1 norm; S24. Final geometric loss construction: combining global first-order consistency loss. Obtain the final geometric loss : ； in, and These are the weighting coefficients.

4. The sparse viewpoint 3D reconstruction method based on block geometry alignment and texture decoupling according to claim 3, characterized in that, In step S21, the number of local blocks The downsampling factor of the original image is taken in the range of 16 to 64.

5. The sparse viewpoint 3D reconstruction method based on block geometry alignment and texture decoupling according to claim 3, characterized in that, In step S21, the average depth and depth standard deviation The calculation formulas are as follows: ； ； in, Represents a local block The total number of pixels within.

6. The sparse viewpoint 3D reconstruction method based on block geometry alignment and texture decoupling according to claim 3, characterized in that, Global first-order consistency loss in step S24 Calculate according to the following formula: ； in, This represents a normalized rendering depth map of the entire image. This represents the standardized prior depth map of the entire map.

7. The sparse viewpoint 3D reconstruction method based on block geometry alignment and texture decoupling according to claim 1, characterized in that, Step S3 specifically includes: S31. Texture-structure decoupling weight construction: Introducing the gradient of the prior normal graph. As a measure of geometrical changes, a texture-structure decoupling weight function is constructed. : ； in, For the input image in pixels The color, For indicator functions, Geometric edge threshold, and To control the influence of color and geometric information on the decoupling weights, This represents an exponential function with base e. Represents pixels The gradient of the unit prior pseudo-normal vector at the given location; S32. Construction of Adaptive Depth Smoothing Loss: Based on the texture-structure decoupling weight function, and combined with second-order depth constraints and first-order normal constraints, an adaptive depth smoothing loss function is constructed. : ； in, The Laplacian operator is used to constrain abrupt changes in curvature. and These are the depth and normal smoothing weights, respectively. Represents pixels The gradient of the pseudo-normal vector at the unit rendering point. Represents pixels in the initial rendered depth map after local normalization. The depth value at that location.

8. The sparse viewpoint 3D reconstruction method based on block geometry alignment and texture decoupling according to claim 1, characterized in that, Step S4 specifically includes: S41. Initial prior depth map Compared with the initial rendered depth map Global standardization is performed separately to obtain a fully standardized rendering depth map. Standardized prior depth map with full map Calculate the normalized rendering depth map of the entire image. Standardized prior depth map with full map The absolute difference is used to generate a depth residual map. : ； S42. Set residual threshold and far-plane cutoff threshold Identify geometrically anomalous regions and generate a binarized artifact mask. : ； in, Represents pixels in the initial rendered depth map before local normalization. The original depth value at that location; S43. Apply a soft opacity penalty to the Gaussian elements projected within the mask area using the following formula. : ； in, Accumulated opacity per pixel This represents the total number of pixels within the mask area; S44. During the densification iteration, hard pruning is performed on primitives that meet the opacity threshold condition and whose depth residual is consistently higher than the set residual threshold.

9. The sparse viewpoint 3D reconstruction method based on block geometry alignment and texture decoupling according to claim 8, characterized in that, In step S42, the residual threshold Set to 3 times the global standard deviation: Far-plane truncation threshold Set as the far clipping plane in the camera parameters.

10. The sparse viewpoint 3D reconstruction method based on block geometry alignment and texture decoupling according to claim 1, characterized in that, Step S5 specifically includes: S51, Combined with color reprojection loss Final geometric loss Smoothing loss function and Opacity Soft Penalty Loss Construct the total loss function : ； in, , , and Color reprojection loss Final geometric loss Smoothing loss function and Opacity Soft Penalty Loss Weight parameters; S52, By minimizing the total loss function Multi-constraint collaborative optimization iteratively updates Gaussian parameters; during training, every... In the next densification iteration, hard pruning is performed on Gaussian elements that are continuously in the artifact mask region and whose opacity is below a preset threshold until the model converges. S53. Input the trained Gaussian primitives into the differentiable rendering pipeline to achieve high-fidelity image rendering and geometry generation from any new perspective.

Images