A sparse-view single-tree three-dimensional reconstruction method, medium, device and product

Abstract

The application discloses a sparse-view single-tree three-dimensional reconstruction method, medium, equipment and product, comprising: generating an initial 2D Gaussian model of a single-tree three-dimensional scene based on sparse-view single-tree images; performing joint optimization on the 2D Gaussian model to obtain a final 2D Gaussian model, the joint optimization comprising: camera pose correction; constructing a depth consistency constraint of a rendered depth and a camera monocular depth in a local image block of the image after the pose correction; generating a new view according to the view image of the camera after the pose correction, calculating a reliable pixel area of the new view image as internal prior information according to the rendered depth and the back-projection depth of the new view, and combining external prior information of a pre-trained diffusion model to guide the prior alignment of the rendering result; constructing importance score, depth protection weight and pruning scheduling function to determine Gaussian pruning probability, and performing pruning based on the Gaussian pruning probability. The application can guarantee the geometric structure integrity of single-tree reconstruction and reduce redundant Gaussian quantity.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional reconstruction technology, and in particular to a single-tree three-dimensional reconstruction method, medium, equipment and products with sparse perspective. Background Technology

[0002] Trees, as key structural units in ecosystems, play a crucial role in forest stand structure analysis, ecological monitoring and assessment, and low-altitude forestry operations. With the increasing demand for refined management, establishing highly realistic, detailed, and structurally accurate tree models is becoming increasingly important. However, the complex layers of tree branches and leaves, and the varied shading relationships, make traditional parametric tree models relying on rule-based modeling or prior knowledge insufficient for meeting the needs of refined reconstruction.

[0003] The introduction of neural radiation fields has brought about a major technological revolution in the field of 3D reconstruction. Neural radiation fields take 3D coordinates and viewpoint direction as input, and predict the volume density and corresponding volume radiation value at the current location using a multilayer perceptron. Neural radiation fields can accurately model the geometry and appearance of complex 3D scenes using continuous implicit fields; however, its training and rendering processes are computationally intensive, and reconstruction efficiency needs improvement.

[0004] To improve the efficiency of generating new perspectives, 3DGS (3D Gaussian Splatting) proposes to explicitly model the scene using a 3D Gaussian ellipsoid and achieve high-quality reconstruction through differentiable rendering, demonstrating significant advantages in real-time rendering and high-resolution reconstruction. This method relies on SfM (Structure from Motion) to provide initial points and camera pose, and then gradually densifies the scene structure through a density control mechanism, making it suitable for shooting conditions with clear textures and ample perspectives. However, when the number of input images is limited, the canopy texture is repetitive, and there is severe local occlusion, the initial point cloud and pose output by SfM often lack stability, and errors accumulate continuously during the training process, causing a significant decrease in the structure recovery accuracy of 3DGS under sparse perspectives.

[0005] 2DGS (2D Gaussian Splatting) proposes replacing volumetric Gaussians with two-dimensional oriented ellipses, achieving accurate perspective reconstruction through explicit ray-spatial intersection. This surface-oriented rather than volume-oriented modeling approach makes the geometric representation of thin-layer structures more accurate. Each 2D Gaussian primitive contains optimizable surface normals, allowing the model to recover the true shape of continuous surfaces by leveraging normal consistency constraints and depth distortion constraints. This geometric representation mechanism is highly suitable for the characteristics of local thin-layer structures such as branches and leaves in trees. Although research on sparse perspective reconstruction based on 2DGS is still in its early stages, its surface-priority modeling approach, natural constraints on depth consistency, and explicit handling of perspective geometry provide a valuable approach to solving geometric drift and depth distortion under sparse input conditions in tree scenes.

[0006] Overall, sparse-view tree reconstruction still faces challenges: the high irregularity of branch and leaf structure increases the ill-posedness of multi-view geometric solutions; SfM initialization performs poorly in sparse images; existing implicit or explicit reconstruction methods all suffer from missing details or structural distortions under sparse input. Maintaining the continuity of tree surfaces and avoiding the propagation of erroneous depth information within a limited number of views remains a key challenge for accurate tree reconstruction. Summary of the Invention

[0007] The purpose of this invention is to address the problems of inaccurate pose calculation and incomplete geometric reconstruction results for single-tree scenes under sparse viewpoint conditions in existing tree reconstruction methods. This invention proposes a sparse viewpoint method for 3D reconstruction of single trees, comprising the following steps: S1. Obtain a sparse view image of a single tree captured by the camera, and generate the initial camera pose and sparse point cloud. S2. Based on sparse point cloud, generate an initial 2D Gaussian model of a single-tree 3D scene; S3. Perform joint optimization on the initial 2D Gaussian model of the single-tree 3D scene to obtain the final 2D Gaussian model. The joint optimization includes: Camera pose correction based on initial camera pose; Based on the initial 2D Gaussian model, the view of the camera after pose correction is rendered to obtain a rendered depth map. Depth consistency constraints between the rendered depth and the estimated camera monocular depth are constructed in local image patches of the view image of the camera after pose correction. A new viewpoint is generated based on the viewpoint image of the camera after pose correction, and an image and back-projection depth map of the new viewpoint are generated. The reliable pixel region of the new viewpoint image is calculated as internal prior information based on the rendered depth map of the camera viewpoint after pose correction and the back-projection depth map of the new viewpoint. The external prior information of the pre-trained diffusion model is combined to perform prior guidance alignment on the rendering result. Calculate the Gaussian density and depth in a 2D Gaussian field, and construct an importance score based on the Gaussian density and depth; construct depth protection weights based on the depth; introduce a pruning scheduling function that gradually increases with the training process; determine the Gaussian pruning probability based on the importance score, depth protection weights, and pruning scheduling function, and perform pruning based on the Gaussian pruning probability.

[0008] Furthermore, camera pose correction is performed based on the initial camera pose, specifically as follows: (1) A learnable pose embedding vector is introduced for each camera, and the corresponding pose increment is predicted by mapping through a multilayer perceptron. The predicted pose increment is represented as follows:

[0009] in, This represents the rotation correction amount for the i-th camera. This represents the displacement correction amount for the i-th camera. Indicates the first The pose embedding features of each camera, where MLP stands for Multilayer Perceptron; The initial pose is updated based on the predicted pose increment, as shown below:

[0010] in, and This represents the rotation matrix and translation vector updated by the i-th camera. Indicates the first The initial rotation matrix of each camera. Indicates the first The initial translation vector of each camera; (2) Obtain the set of matching pixel pairs between view pairs, obtain the epipolar distance of the matching pixel pairs according to the updated pose, and construct pose consistency constraints based on epipolar geometry based on the set of matching pixel pairs, the set of view pairs with matching relationships and the epipolar distance of the matching pixel pairs to jointly correct the camera pose. The pose consistency constraint based on epipolar geometry is expressed as:

[0011] in, This indicates pose consistency constraints based on epipolar geometry. This represents a set of viewpoints that have a matching relationship. To match confidence weights, Indicates the first With the The number of pixel pairs matched between each viewpoint Indicates the first With the Matching pixel pairs between viewpoints Indicates the first With the Epipolar distance between the k-th matching pixel pair from each viewpoint.

[0012] Furthermore, the rendered depth map is represented as:

[0013]

[0014] in, This represents the rendering depth map of pixel p. Let the depth be the i-th Gaussian. Let be the contribution weight of the i-th Gaussian to pixel p. Let be the cumulative transmittance of the i-th Gaussian pair of pixel p. Let N be the contribution value of the i-th Gaussian to the opacity of pixel p, where N represents the number of Gaussians.

[0015] Furthermore, the loss function for deep consistency constraints is expressed as follows:

[0016]

[0017] in, This represents the depth consistency loss, where K represents the number of local image patches. express , Describing covariance, Represents variance. This represents the rendering depth corresponding to the i-th local image patch. This represents the monocular depth corresponding to the i-th local image patch.

[0018] Furthermore, an adjacent pixel smoothing loss is introduced to enhance the smoothness and continuity at the global scale. The formula for the adjacent pixel smoothing loss is as follows:

[0019] in, This represents the smoothing loss between adjacent pixels. Represents pixels The set of adjacent pixels, Represents pixels Rendering depth The rendering depth of pixel k. For ReLU function, These are parameters used to avoid over-smoothing.

[0020] Furthermore, prior-guided alignment specifically refers to: New perspectives are generated by randomly rotating and transforming the input sparse perspective camera; The rendered depth map of the input sparse viewpoint is back-projected into the new viewpoint space to obtain a distorted depth map. The absolute error between the rendered depth map and the distorted depth map is compared, and a threshold for the absolute error is set to generate a consistency mask. Based on the consistency mask, reliable pixel regions of the new perspective image are selected to obtain the three-dimensional Gaussian model of the new perspective. In this process, a guided loss function is constructed within the reliable pixel region to guide the rendering result to converge toward the prior distribution. The guided loss function is defined as follows:

[0021] in, Indicates guided loss, To balance hyperparameters, It is a time-step related weight function. This indicates a search for information about time step t and a new perspective. Expected values ​​of relevant parameters Denotes KL divergence, express The rendering distribution of the parameterized 3D representation. Indicates perspective and time step Gaussian under the following conditions express The parameterized three-dimensional representation is a corrected distribution established through inline prior conditions. Indicates the parameters of the conditional diffusion model; A consistency constraint is constructed within the reliable pixel region, and the consistency constraint formula is as follows:

[0022] in, This represents a consistency constraint. For a uniform mask, For inputting a rendered image with a sparse viewpoint, It is a perspective image. This indicates a pixel-by-pixel multiplication operation.

[0023] Furthermore, pruning based on Gaussian pruning probability specifically involves: The density of each Gaussian is calculated using kernel density estimation; Normalize the density and depth of the Gaussian; The importance score of the Gaussian is obtained by weighted summing of the density and depth of the normalized Gaussian. The depth protection weights are defined as follows:

[0024] in, Indicates the first A Gaussian depth protection weight, Indicates the first A Gaussian depth value, and These represent the minimum and maximum depths in the current scene, respectively. Indicates the depth attenuation control coefficient; Introduce a pruning scheduling function that progressively increases with the training process:

[0025] in, This represents the pruning scheduling factor during the current training phase. Indicates the current training steps. This represents the total number of training steps; The probability of Gaussian pruning is determined by combining the comprehensive importance score, the depth protection weight, and the dynamic scheduling function, and a gradual transparency decay strategy is adopted to achieve soft pruning. The pruning probability is defined as follows:

[0026] in, Indicates the first The pruning probability of a Gaussian; This represents the importance score of the i-th Gaussian; For the selected Gaussian, its opacity parameter decreases exponentially, as shown in the formula:

[0027] in, Indicates the first Gauss in the th... Opacity at the next iteration This represents the attenuation coefficient.

[0028] The present invention also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described sparse-view single-tree 3D reconstruction method.

[0029] The present invention also proposes an electronic device including a processor and a memory, wherein the processor is interconnected with the memory, the memory is used to store a computer program, the computer program including computer-readable instructions, and the processor is configured to invoke the computer-readable instructions to execute the above-described sparse perspective single-tree 3D reconstruction method.

[0030] The present invention also proposes a computer program product, including a computer program / instruction that, when executed by a processor, implements the steps of the sparse perspective single-tree 3D reconstruction method described above.

[0031] The beneficial effects of the technical solution provided by this invention are: (1) This invention solves the problem of inaccurate pose in traditional SfM under sparse tree images by introducing a pose update method with sparse view pose increment and epipolar geometric constraints, and provides a reliable geometric basis for subsequent reconstruction. (2) Divide the input image into multiple local image blocks, and construct a depth consistency constraint between monocular depth and rendering depth within the local image blocks to ensure depth stability and geometric continuity under sparse view conditions without explicit scale alignment. (3) Generate images and back-projected depth maps of the new perspective, select reliable pixel regions of the new perspective depth map, and combine the pre-trained diffusion model to perform prior guidance alignment on the rendering results of the Gaussian model of the new perspective to enhance cross-view geometric consistency.

[0032] (4) Construct an importance score based on the local density and depth normalization results of Gaussian layers, calculate the protection weights of Gaussian layers at different depths, and introduce a dynamic pruning scheduling function. Based on the importance score, depth protection weights, and pruning scheduling function, obtain the pruning probability of each Gaussian layer. In each iteration, randomly select some Gaussians to delete based on the pruning probability. This reduces the number of redundant Gaussians while maintaining structural integrity, thereby improving reconstruction quality and training efficiency. Attached Figure Description

[0033] Figure 1 This is a flowchart of a sparse-view single-tree 3D reconstruction method according to an example of the present invention; Figure 2 This is a block diagram of an electronic device according to an exemplary embodiment of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0035] A flowchart of a sparse-view single-tree 3D reconstruction method according to an example of the present invention is shown below. Figure 1 Specifically, it includes: S1. Obtain sparse view images of trees captured by the camera, and use the Sparse3R method to generate the initial camera pose and sparse point cloud.

[0036] In this embodiment of the invention, specifically: S11. Using Sparse3R, multi-view matching and structure restoration are performed on the input sparse tree images to generate an initial set of camera poses and the corresponding SfM sparse point cloud, which serves as the geometric constraint benchmark for subsequent pose optimization. Sparse3R, by fusing multi-view matching relationships and sparse structure restoration mechanisms, outputs a set of initial camera poses under the condition of a limited number of images.

[0037] in, This represents the initial set of camera poses. Indicates the first The initial rotation matrix for each viewpoint. Indicates the first The initial translation vector for each viewpoint. Enter the number of views.

[0038] Meanwhile, the SfM sparse point cloud output by Sparse3R can be represented as:

[0039] in, Represents a set of SfM sparse point clouds. Let M represent the k-th point cloud, and M represent the number of point clouds. The point cloud set serves as the geometric constraint benchmark for subsequent camera pose optimization.

[0040] S2. After acquiring sparse point cloud data, the 3D scene is initialized using the 2DGS method to obtain an initialized 2D Gaussian model.

[0041] In this embodiment of the invention, specifically: Each 2D Gaussian element is defined by its center point location, two mutually orthogonal tangent vectors, and corresponding scaling parameters, which together define its orientation and shape in 3D space. Specifically, let the coordinates of the center point of the k-th 2D Gaussian element be... Its local coordinate system consists of two unit tangent vectors and The composition, and the corresponding anisotropic scaling factors are respectively and normal vector The basic element is obtained by the cross product of two tangent vectors, thus fully describing the local tangent plane in which the element resides. This two-dimensional Gaussian basic element can be represented in the world coordinate system as:

[0042] in, Midpoint of the world coordinate system The coordinate components projected onto the local tangent plane of the k-th primitive.

[0043] During initialization, the sparse point cloud obtained in step S1 is used as the geometric basis, and each 3D point is directly mapped to the center position of a 2D Gaussian element, i.e., let , This represents the k-th 3D point in the sparse point cloud. The direction of the local tangent plane is primarily determined based on the observation relationship of this point across multiple viewpoints. In one implementation, a reference viewpoint for observing this point is selected, and the direction from the camera center to this 3D point is used as an initial normal approximation. Two tangent vectors are constructed within their orthogonal plane to ensure that the Gaussian vector aligns with the main observation direction during the initialization phase, thus guaranteeing the stability of subsequent rendering and optimization processes.

[0044] S3. 3D Scene Optimization: Pose correction, local-global depth consistency constraints, internal and external prior-guided geometric constraints, and Gaussian adaptive pruning optimization are introduced to jointly optimize the 2D Gaussian model and obtain the final 2D Gaussian model.

[0045] In this embodiment of the invention, specifically: (1) Posture correction Based on the initial camera pose, a learnable pose embedding vector is introduced for each camera. After predicting rotation and translation increments using a multilayer perceptron, the pose is updated exponentially. Simultaneously, epipolar geometric consistency constraints are constructed based on pixel-level matching between view pairs. By minimizing the epipolar distance loss, the camera pose is jointly optimized during 2DGS training. Specifically, this includes: ① A learnable pose embedding vector is introduced for each camera. The rotation and translation increments of the camera pose are predicted by a multilayer perceptron, and the initial pose is updated using an exponential mapping method to limit the magnitude of a single pose correction and maintain geometric stability. The corresponding pose increment is predicted as follows:

[0046] in, This represents the rotation correction amount for the i-th camera. This represents the displacement correction amount for the i-th camera. Indicates the first The pose embedding features of each camera, where MLP stands for Multilayer Perceptron.

[0047] The initial pose is updated based on the predicted pose increment, as shown below:

[0048] in, and This represents the rotation matrix and translation vector updated by the i-th camera. Indicates the first The initial rotation matrix of each camera. Indicates the first The initial translation vector of each camera.

[0049] ② Obtain the set of matching pixel pairs between view pairs. Based on the updated pose, obtain the epipolar distance of the matching pixel pairs. Based on the set of matching pixel pairs, the set of view pairs with matching relationships, and the epipolar distance of the matching pixel pairs, construct a pose consistency constraint based on epipolar geometry. By minimizing the epipolar distance loss of the matching point pairs, jointly correct the camera pose, thereby reducing the impact of sparse matching error on the subsequent reconstruction process.

[0050] Let the first With the There is a set of matching pixel pairs between each viewpoint. :

[0051] in, Indicates the first With the The k-th matching pixel pair between viewpoints Indicates the first With the The number of pixel pairs matched between viewpoints.

[0052] According to the With the Current pose estimation from each viewpoint and Construct the corresponding fundamental matrix The epipolar distance of a single matching point pair is defined as:

[0053] in, Indicates the first With the Epipolar distance between the k-th matching pixel pair from each viewpoint.

[0054] The pose consistency constraint based on epipolar geometry is expressed as:

[0055] in, This indicates pose consistency constraints based on epipolar geometry. This represents a set of viewpoints that have a matching relationship. To match confidence weights, which are used to reduce the interference of unreliable correspondences on pose optimization, Indicates the first With the The number of pixel pairs matched between each viewpoint Indicates the first With the A set of matching pixel pairs between viewpoints.

[0056] The pose optimization process does not introduce additional photometric loss or dense supervision signals, but only relies on epipolar geometric constraints of sparse matching, which ensures the stability of camera geometry while maintaining computational efficiency.

[0057] (2) Local-global depth consistency constraint The sparse viewpoint image after camera correction is used as the input image. The viewpoint of the input image is rendered, and the pixel depth is calculated to obtain the rendered depth map. For pixel p, the rendered depth map... Represented as:

[0058]

[0059] in, This represents the depth map being rendered, where N represents the number of Gaussian swathes. Let the depth of the i-th Gaussian be the depth from the current perspective. Let be the contribution weight of the i-th Gaussian to pixel p. Let be the cumulative transmittance of the i-th Gaussian pair of pixel p. The contribution value of the i-th Gaussian to the opacity of pixel p.

[0060] The monocular depth estimation result is introduced as a relative depth prior, and the monocular depth of the camera is estimated by the pre-trained monocular depth estimation model.

[0061] The input image is divided into multiple local image patches, and depth consistency constraints are calculated independently within each patch to enhance the stability of local geometry and mitigate the impact of global scale errors. The Pearson correlation coefficient is used to measure the distribution consistency between the rendered depth and monocular depth within each local image patch.

[0062] Extract the rendering depth corresponding to each block and monocular depth Calculate the intra-block Pearson correlation coefficient:

[0063] in, express and The Pearson correlation coefficient, Describing covariance, Represents variance. This represents the rendering depth corresponding to the i-th local image patch. This represents the monocular depth corresponding to the i-th local image patch.

[0064] The loss function for local depth consistency constraints is expressed as follows:

[0065] in, This represents the depth consistency loss, and K represents the number of local image patches.

[0066] This loss encourages local depth distribution alignment without requiring explicit scale alignment, thus avoiding the introduction of additional errors.

[0067] To enhance global-scale smoothness and continuity, an adjacent-pixel smoothing loss is introduced. This loss addresses potential abrupt depth changes in texture-deficient regions by constraining the depth difference between adjacent pixels to strengthen local consistency. For depth maps... Consider the position of each pixel and its adjacent pixels (right neighbor) and the next neighbor ).

[0068] Non-edge regions are identified based on monocular depth: if the monocular depth difference between adjacent pixels is less than a threshold. ( If a region is considered smooth, then a smoothing constraint should be applied. The formula for the smoothing loss between adjacent pixels is as follows:

[0069] in, This represents the smoothing loss between adjacent pixels. Represents pixels The set of adjacent pixels, Represents pixels Rendering depth This represents the rendering depth of pixel k. For ReLU function, For small threshold ( The loss function is used to avoid over-smoothing. It activates only in smooth regions, ensuring that the depth gradient changes gently while preserving edge details.

[0070] (3) Geometric constraints guided by internal and external priors The process generates a new, unseen viewpoint from the input sparse image, along with an image and back-projected depth map of the new viewpoint. Reliable pixel regions of the new viewpoint image are calculated based on the rendered depth map of the sparse viewpoint from the input image and the back-projected depth map of the new viewpoint, serving as internal prior information. This internal prior information is then combined with external prior information from a pre-trained diffusion model to provide prior-guided alignment for the rendering results. Specifically, this includes: ① Synthesize no new perspectives from sparse input views, given a set of sparse input views. ,in For image, For camera pose. Sample new perspectives around the input view. Its camera pose is generated through random rotation transformations. For each new viewpoint... Select the most recently entered view. As a reference, it is back-projected to generate a corresponding new perspective image.

[0071] Specifically, set For a new perspective The rendering depth map will be the input image. Backprojection to the new viewpoint space. Transformed pixel coordinates. The calculation formula is as follows:

[0072] in For the camera intrinsic parameter matrix, From the perspective To view The pose transformation matrix, where u represents the input image. The pixels. A new perspective image is obtained through bilinear sampling. .

[0073] ② Backproject the rendered depth map of the sparse viewpoint of the target to the new viewpoint space to obtain the backprojected depth map. Calculate the absolute error between the rendered depth map and the back-projected depth map of the input view:

[0074] in, This represents the absolute error between the rendered depth map and the back-projected depth map of the input view at point u. This indicates the depth of the input view at coordinate u. This represents the depth of the back-projected depth map at coordinate u.

[0075] Set threshold (like Generate a consistency mask :

[0076] in, This represents the consistency mask at coordinate u, which is used for subsequent loss calculations to ensure that constraints are applied only to geometrically consistent regions.

[0077] ③ Construct a two-stage distribution alignment mechanism based on inline prior and diffusion prior to guide the rendering results to converge toward a geometrically reasonable prior distribution at the distribution level, and introduce pixel-level geometric consistency loss under mask constraints.

[0078] Specifically: The traditional single-stage optimization objective is decomposed into two consecutive distribution alignment processes. First, a corrected distribution is constructed through inline priors, causing the rendering distribution to converge towards the corrected distribution. Then, the corrected distribution is pushed to approximate the diffused prior distribution. At the distribution level, the rendering results are guided to converge towards a geometrically reasonable prior distribution.

[0079] set up Parametric 3D representation in viewpoint and time step The rendering distribution is as follows. The corrected distribution established through inline prior conditions is denoted as... ,in This represents the parameters of the conditional diffusion model. The diffusion prior distribution refers to the data distribution represented by a pre-trained diffusion model. Guiding loss function. Defined as:

[0080] in, Indicates guided loss, To balance hyperparameters, It is a time-step related weight function. This indicates a search for information about time step t and a new perspective. Expected values ​​of relevant parameters KL divergence is used to measure the difference between two probability distributions. express The rendering distribution of the parameterized 3D representation. Indicates perspective and time step Gaussian under the following conditions express The parameterized three-dimensional representation is a corrected distribution established through inline prior conditions. This represents the parameters of the conditional diffusion model. This loss function ensures that the optimization process maintains geometric reasonableness at the distribution level.

[0081] At the pixel level, a binary mask is generated based on depth consistency. Identify reliable pixel regions and construct consistency constraints within these regions. The consistency constraint formula is as follows:

[0082] in, This represents a consistency constraint. For a uniform mask, Rendering images for targets with sparse viewpoints. It is a perspective image. This represents a pixel-by-pixel multiplication operation. This loss function enforces pixel-level consistency constraints within the mask-defined region.

[0083] (4) Gaussian adaptive pruning optimization The Gaussian density and depth in a 2D Gaussian field are calculated. An importance score is constructed using the normalized results of both. A depth-level scaling and training process scheduling mechanism is introduced to determine the probabilistic pruning strategy for Gaussians, gradually removing redundant Gaussians through soft deletion. Specifically: ① Calculate Gaussian density based on spatial neighborhood relationships. For the i-th Gaussian, calculate its kernel density to represent the degree of clustering in its vicinity:

[0084] in, This represents the density of the i-th Gaussian, where N is the number of Gaussians. and This represents the center positions of the i-th and j-th Gaussians in three-dimensional space. This represents the bandwidth of the kernel function.

[0085] ② Normalize the density and depth of the Gaussian. Then, weight and sum the normalized Gaussian density and depth to obtain the importance score of the Gaussian, using the following formula:

[0086] in, This represents the importance score of the i-th Gaussian. and Represents the normalized depth and density of the i-th Gaussian. These are the weighting coefficients.

[0087] ③ To prevent over-pruning of the outer canopy, twigs, and sparsely leafed areas due to insufficient observation, an adaptive protection mechanism based on continuous depth variation is introduced. The pruning intensity is continuously adjusted according to the depth distance from the Gaussian to the camera viewpoint, and the depth protection weight is defined as follows:

[0088] in, Indicates the first A Gaussian depth protection weight, This represents the depth value corresponding to the Gaussian shape. and These represent the minimum and maximum depths in the current scene, respectively. This represents the depth attenuation control coefficient. As the depth increases, the protection weight gradually decreases, thereby reducing the actual pruning intensity of the Gaussian at long distances.

[0089] ④ To avoid irreversible deletions during the early stages of training when the Gaussian structure is not yet stable, and to enhance the model's ability to compress redundant Gaussians and floating noise in later stages, a nonlinear pruning scheduling function that gradually increases with the training process is introduced:

[0090] in, This represents the pruning scheduling factor during the current training phase. Indicates the current training steps. This represents the total number of training steps. The function grows slowly in the early stages of training, as the model primarily achieves a stable fit to the main structure and geometric shape of the tree. As training gradually converges, the pruning scheduling factor gradually increases, resulting in greater pruning intensity for low-contribution regions and redundant Gaussians.

[0091] ⑤ The Gaussian pruning probability is determined jointly based on the comprehensive importance score, depth protection weight, and nonlinear pruning scheduling function, and a gradual transparency decay strategy is used to implement soft pruning. The pruning probability is defined as follows:

[0092] in, Indicates the first The pruning probability of a Gaussian. For the selected Gaussian, its opacity parameter decreases exponentially:

[0093] in, Indicates the first Gauss in the th... Opacity at the next iteration This represents the attenuation coefficient. Unselected Gaussians retain their parameters unchanged.

[0094] In one exemplary embodiment, a computer-readable storage medium is included, which stores a computer program that, when executed by a processor, implements the above-described sparse-view single-tree 3D reconstruction method.

[0095] Please see Figure 2 In one exemplary embodiment, the device further includes an electronic device including at least one processor, at least one memory, and at least one communication bus.

[0096] The memory stores a computer program, which includes computer-readable instructions. The processor calls the computer-readable instructions stored in the memory through the communication bus to execute the sparse perspective single-tree 3D reconstruction method described above.

[0097] In one exemplary embodiment, a computer program product is proposed, including a computer program / instructions that, when executed by a processor, implement the steps of the sparse perspective mono-tree 3D reconstruction method described above.

[0098] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A sparse-viewpoint single-tree 3D reconstruction method, characterized in that, Includes the following steps: S1. Obtain a sparse view image of a single tree captured by the camera, and generate the initial camera pose and sparse point cloud. S2. Based on sparse point cloud, generate an initial 2D Gaussian model of a single-tree 3D scene; S3. Perform joint optimization on the initial 2D Gaussian model of the single-tree 3D scene to obtain the final 2D Gaussian model. The joint optimization includes: Camera pose correction based on initial camera pose; Based on the initial 2D Gaussian model, the view of the camera after pose correction is rendered to obtain a rendered depth map. Depth consistency constraints between the rendered depth and the estimated camera monocular depth are constructed in local image patches of the view image of the camera after pose correction. A new viewpoint is generated based on the viewpoint image of the camera after pose correction, and an image and back-projection depth map of the new viewpoint are generated. The reliable pixel region of the new viewpoint image is calculated as internal prior information based on the rendered depth map of the camera viewpoint after pose correction and the back-projection depth map of the new viewpoint. The external prior information of the pre-trained diffusion model is combined to perform prior guidance alignment on the rendering result. Calculate the Gaussian density and depth in a 2D Gaussian field, and construct an importance score based on the Gaussian density and depth; construct depth protection weights based on the depth; introduce a pruning scheduling function that gradually increases with the training process; determine the Gaussian pruning probability based on the importance score, depth protection weights, and pruning scheduling function, and perform pruning based on the Gaussian pruning probability.

2. The sparse perspective single-tree 3D reconstruction method according to claim 1, characterized in that, Camera pose correction is performed based on the initial camera pose, specifically as follows: (1) A learnable pose embedding vector is introduced for each camera, and the corresponding pose increment is predicted by mapping through a multilayer perceptron. The predicted pose increment is represented as follows: in, This represents the rotation correction amount for the i-th camera. This represents the displacement correction amount for the i-th camera. Indicates the first The pose embedding features of each camera, where MLP stands for Multilayer Perceptron; The initial pose is updated based on the predicted pose increment, as shown below: in, and Indicates the first i The rotation matrix and translation vector updated for each camera. Indicates the first The initial rotation matrix of each camera. Indicates the first The initial translation vector of each camera; (2) Obtain the set of matching pixel pairs between view pairs, obtain the epipolar distance of the matching pixel pairs according to the updated pose, and construct pose consistency constraints based on epipolar geometry based on the set of matching pixel pairs, the set of view pairs with matching relationships and the epipolar distance of the matching pixel pairs to jointly correct the camera pose. The pose consistency constraint based on epipolar geometry is expressed as: in, This indicates pose consistency constraints based on epipolar geometry. This represents a set of viewpoints that have a matching relationship. To match confidence weights, Indicates the first With the The number of pixel pairs matched between each viewpoint Indicates the first With the Matching pixel pairs between viewpoints Indicates the first With the Epipolar distance between the k-th matching pixel pair from each viewpoint.

3. The sparse perspective single-tree 3D reconstruction method according to claim 1, characterized in that, The rendered depth map is represented as follows: in, This represents the rendering depth map of pixel p. Let the depth be the i-th Gaussian. Let be the contribution weight of the i-th Gaussian to pixel p. Let be the cumulative transmittance of the i-th Gaussian pair of pixel p. Let N be the contribution value of the i-th Gaussian to the opacity of pixel p, where N represents the number of Gaussians.

4. The sparse perspective single-tree 3D reconstruction method according to claim 1, characterized in that, The loss function for deep consistency constraints is expressed as follows: in, This represents the depth consistency loss, where K represents the number of local image patches. express , Describing covariance, Represents variance. This represents the rendering depth corresponding to the i-th local image patch. This represents the monocular depth corresponding to the i-th local image patch.

5. The sparse perspective single-tree 3D reconstruction method according to claim 1, characterized in that, To enhance the smoothness and continuity at the global scale, a neighboring pixel smoothing loss is introduced. The formula for the neighboring pixel smoothing loss is as follows: in, This represents the smoothing loss between adjacent pixels. Represents pixels The set of adjacent pixels, Represents pixels Rendering depth The rendering depth of pixel k. For ReLU function, These are parameters used to avoid over-smoothing.

6. The sparse perspective single-tree 3D reconstruction method according to claim 1, characterized in that, Prior guidance alignment specifically refers to: New perspectives are generated by randomly rotating and transforming the input sparse perspective camera; The rendered depth map of the input sparse viewpoint is back-projected into the new viewpoint space to obtain a distorted depth map. The absolute error between the rendered depth map and the distorted depth map is compared, and a threshold for the absolute error is set to generate a consistency mask. Based on the consistency mask, reliable pixel regions of the new perspective image are selected to obtain the three-dimensional Gaussian model of the new perspective. In this process, a guided loss function is constructed within the reliable pixel region to guide the rendering result to converge toward the prior distribution. The guided loss function is defined as follows: in, Indicates guided loss, To balance hyperparameters, It is a time-step related weight function. This indicates a search for information about time step t and a new perspective. Expected values ​​of relevant parameters Denotes KL divergence, express The rendering distribution of the parameterized 3D representation. Indicates perspective and time step Gaussian under the following conditions express The parameterized three-dimensional representation is a corrected distribution established through inline prior conditions. Indicates the parameters of the conditional diffusion model; A consistency constraint is constructed within the reliable pixel region, and the consistency constraint formula is as follows: in, This represents a consistency constraint. For a uniform mask, For inputting a rendered image with a sparse viewpoint, It is a perspective image. This indicates a pixel-by-pixel multiplication operation.

7. The sparse perspective single-tree 3D reconstruction method according to claim 1, characterized in that, Pruning based on Gaussian pruning probability specifically involves: The density of each Gaussian is calculated using kernel density estimation; Normalize the density and depth of the Gaussian; The importance score of the Gaussian is obtained by weighted summing of the density and depth of the normalized Gaussian. The depth protection weights are defined as follows: in, Indicates the first A Gaussian depth protection weight, Indicates the first A Gaussian depth value, and These represent the minimum and maximum depths in the current scene, respectively. Indicates the depth attenuation control coefficient; Introduce a pruning scheduling function that progressively increases with the training process: in, This represents the pruning scheduling factor during the current training phase. Indicates the current training steps. This represents the total number of training steps; The probability of Gaussian pruning is determined by combining the comprehensive importance score, the depth protection weight, and the dynamic scheduling function, and a gradual transparency decay strategy is adopted to achieve soft pruning. The pruning probability is defined as follows: in, Indicates the first The pruning probability of a Gaussian; This represents the importance score of the i-th Gaussian; For the selected Gaussian, its opacity parameter decreases exponentially, as shown in the formula: in, Indicates the first Gauss in the th... Opacity at the next iteration This represents the attenuation coefficient.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

9. An electronic device, characterized in that, The device includes a processor and a memory, the processor being interconnected with the memory, wherein the memory is used to store a computer program, the computer program including computer-readable instructions, and the processor is configured to invoke the computer-readable instructions to perform the method as described in any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the method described in any one of claims 1 to 7.

Images