Three-dimensional reconstruction method based on adaptive dynamic optimization strategy and gradient perception enhancement

Abstract

The invention discloses a three-dimensional reconstruction method based on a 3DGS self-adaptive dynamic optimization strategy and gradient perception enhancement. The compression performance and the rendering quality of a Reduce 3DGS are further optimized through multi-stage hybrid optimization; firstly, a self-adaptive loss strategy is adopted, the weight between SSIM and L1 loss is dynamically adjusted through a Sigmoid function, structure alignment is emphasized in the early stage, and enhancement detail optimization is focused in the later stage; secondly, sparsity regularization is introduced in the optimization stage, transparency entropy maximization constraint is introduced to be combined with attenuation weight, and generation of redundant primitives is actively inhibited while the redundant primitives are trimmed; finally, gradient sensing detail enhancement is carried out, gradient matching loss is increased, high-frequency texture and edge information are reserved, and detail loss caused by trimming and quantization is made up; experiments prove that the method disclosed by the invention achieves better balance in the aspects of compression ratio and visual quality.

Description

Technical Field

[0001] The present invention belongs to the field of computer graphics and three-dimensional reconstruction technology, and specifically relates to a three-dimensional reconstruction method based on 3DGS adaptive dynamic optimization strategy and gradient perception enhancement. Background Art

[0002] In the field of novel perspective synthesis, various methods have emerged to improve rendering quality, speed, and memory efficiency. In recent years, techniques based on explicit and implicit representations have been widely studied. Among them, image-based rendering methods rely on geometric proxies, such as triangular meshes or depth maps from multi-view stereo reconstruction, to achieve novel perspective synthesis. These methods can provide reasonable quality at relatively fast rendering speeds, but their performance is limited by the accuracy of the proxy geometry, and storage requirements increase significantly as scene complexity increases.

[0003] NeRF (Neural Radiance Field) uses a multi-layer perceptron to model the color and density of 3D scenes and utilizes volume rendering techniques to generate images from new perspectives. Although NeRF provides excellent rendering quality, it suffers from extremely high computational overhead, long training times, and slow rendering speeds. Subsequently, numerous improved methods have been proposed, such as Mip-NeRF360 (Unbounded Anti-Aliased Neural Radiance Field) and Zip-NeRF (Fast Neural Radiance Field). These methods improve efficiency by optimizing sampling strategies or introducing more compact data structures, but still struggle to render high-quality images at interactive rates.

[0004] Another emerging method is 3D Gaussian splatting (3DGS) [Kerbl, B, Kopanas, G, Leimku¨hler, T, and Drettakis, G. 3d gaussian splatting for real-time radiance ffeld rendering. ACM Transactions on Graphics, 42(4), 2023]. It uses a rasterization-based method to project a 3D Gaussian distribution onto a 2D screen and calculates pixel color through depth sorting and alpha blending. Compared with NeRF, 3DGS avoids the complex ray marching steps and achieves extremely fast rendering speed. However, 3DGS still has some challenges, such as sampling sensitivity issues that may cause rendering artifacts, and due to the lack of clear geometric constraints, its Gaussian distribution may grow excessively during the optimization process, thereby increasing storage overhead.

[0005] To address these issues, various optimization strategies have been proposed. For example, GaussianCube proposes a structured explicit radiance representation, arranging Gaussians into a predefined voxel grid through optimal transmission. GaussianPro (Gaussian Progressive Propagation) optimizes low-texture regions by proposing a progressive propagation strategy. LightGaussian reduces storage through a three-stage optimization process involving Gaussian pruning, SH (spherical harmonics) distillation, and VecTree (vector tree) quantization. Furthermore, some methods attempt to calculate Gaussian contributions to reduce redundant Gaussians and improve storage efficiency.

[0006] In addition, the Reduce (lightweight) 3DGS strategy in the literature [Papantonakis, P., Kopanas, G., Kerbl, B., Lanvin, A., & Drettakis, G. (2024). Reducing the memory footprint of 3d gaussian splatting. Proceedings of the ACM on Computer Graphics and Interactive Techniques, 7(1), 1-17] further proposed resolution-aware pruning, adaptive SH adjustment, and codebook quantization, which greatly reduced the representation of 3DGS, but also caused the loss of high-frequency details.

[0007] In recent years, the introduction of 3DGS has marked a revival of the explicit representation paradigm for 3D reconstruction. This method models the scene as a learnable 3D Gaussian primitive, achieving for the first time the unification of high-fidelity rendering (>30fps) and real-time performance. However, 3DGS still faces challenges. Although heuristic control is introduced into the optimization process, there is significant redundancy in the distribution of Gaussian primitives. Some areas produce a large number of low-contribution primitives (up to millions for a single scene) due to over-subdivision, while other areas still have blurred reconstructions due to insufficient coverage, resulting in very high storage requirements for typical scenes. To alleviate this problem, Reduce 3DGS has made great efforts. Through resolution-aware primitive pruning, adaptive spherical harmonic band adjustment, and codebook quantization, the 3DGS representation has been significantly reduced. However, this method still has the following limitations:

[0008] 1. Fixed optimization objective: The loss function weights are statically assigned, which cannot balance the requirements of different training stages (such as early reconstruction and late compression), affecting convergence efficiency and model compactness.

[0009] 2. Loss of high-frequency details: Quantization and pruning may cause loss of edge texture and reflective details, resulting in a decrease in the LPIPS (Learning Perceptual Patch Similarity) indicator.

[0010] 3. Reliance on passive post-processing: Redundant primitives need to be removed through post-training pruning, and the lack of active sparsification constraints during the training phase limits compression efficiency. Summary of the Invention

[0011] In view of the above, the present invention provides a three-dimensional reconstruction method based on 3DGS adaptive dynamic optimization strategy and gradient-aware enhancement. A new dynamic optimization framework is used to enhance the compactness and high-frequency features of Reduce 3DGS, so that the three-dimensional reconstruction results have better compression rate and higher quality.

[0012] A 3D reconstruction method based on 3DGS adaptive dynamic optimization strategy and gradient perception enhancement includes the following steps:

[0013] (1) Obtain multi-view images of the reconstructed scene and extract the initial point cloud from the images;

[0014] (2) Divide all images into training and test sets in proportion;

[0015] (3) Constructing the ReducePro (lightweight optimized version) 3DGS model, which includes:

[0016] The densification module is used to densify the initial point cloud of the image to fill the blank areas of the scene and obtain a dense point cloud;

[0017] The redundant point pruning module calculates the redundancy value of each Gaussian point in the dense point cloud and prunes out overly dense Gaussian points according to the set redundancy threshold to reduce the memory usage of the dense point cloud;

[0018] The codebook quantization module uses K-means clustering to create a codebook to further compress the memory usage of the dense point cloud, thereby generating a reconstructed point cloud;

[0019] (4) The ReducePro 3DGS model is trained using the training set. In the densification phase, the adaptive loss is applied to dynamically adjust the weights for geometric reconstruction. In the redundant point pruning phase, the adaptive loss, dynamic sparse loss, and gradient-aware loss are jointly optimized to constrain the geometric consistency between the rendered result and the real result, and to compensate for the detail loss caused by codebook quantization.

[0020] (5) Input the initial point cloud of the test set image into the trained ReducePro 3DGS model to automatically generate the reconstructed point cloud.

[0021] Furthermore, the codebook quantization module uses K-means clustering to create a codebook, uses a 1-byte index to point to the approximate value of the attribute in the codebook, and establishes codebooks for the attributes of Gaussian points, including opacity, scaling component, and rotation component. Then, the position and codebook entries are quantized to 16-bit half floating point, further reducing the memory usage of dense point clouds.

[0022] Furthermore, in the process of training the ReducePro 3DGS model in step (4), the loss function loss1 is used in the early stage of training (such as the first 15,000 rounds), and the loss function loss2 is used in the late stage of training. The specific expressions are as follows:

[0023]

[0024] in: is the adaptive weight, is the mean absolute error loss, is the structural similarity loss, is the regularization loss, is the dynamic sparse loss, is the gradient matching loss, and λ1~λ3 are the set weight coefficients.

[0025] Furthermore, the adaptive weight λ dssim The expression is as follows:

[0026]

[0027] Where: start is the initial value of the weight, Δλ is the difference between the initial value and the final value of the weight, k is the coefficient that controls the speed of change, a is the current iteration round, T is the total number of training rounds, e is a natural constant, and p is the set progress.

[0028] Furthermore, the mean absolute error loss The expression is as follows:

[0029]

[0030] Where: i and j are the index numbers of the row and column in the original image (any image in the training set), is the pixel value of the i-th row and j-th column in the original image, I i,j It represents the pixel value of the i-th row and j-th column in the rendered image corresponding to the reconstructed point cloud generated by the ReducePro 3DGS model when the initial point cloud of the original image is input. N represents the total number of pixels in the original image.

[0031] Furthermore, the structural similarity loss The expression is as follows:

[0032]

[0033] Among them: I gt and I represent the original image and its initial point cloud input, respectively. The ReducePro 3DGS model generates the rendered image corresponding to the reconstructed point cloud. SSIM(I,I gt ) is I and I gt The structural similarity of x and μ y I and I respectively gt The local mean of (calculated by convolution), and I and I respectively gt The local variance, σ xy for I and I gt The local covariance of , C1 and C2 are set constants.

[0034] Furthermore, the regularization loss The expression is as follows:

[0035]

[0036] Where M is the number of visible points in the reconstructed point cloud, α m is the transparency of the mth visible point.

[0037] Furthermore, the dynamic sparse loss The expression is as follows:

[0038]

[0039]

[0040] w cons i st =β·t

[0041] w stability =γ(1-t)

[0042] in: is the color difference loss, is the sparse regularization loss, w consist and w stability are dynamic weights, H and W are the width and height of the original image, i and j are the index numbers of the row and column in the original image, is the pixel value of the i-th row and j-th column in the original image, I i,jrepresents the pixel value of the i-th row and j-th column in the rendered image corresponding to the initial point cloud input of the original image into the ReducePro 3DGS model to generate the reconstructed point cloud. t represents the current iteration progress and t = a / T, where a is the current iteration round, T is the total training rounds, M is the number of visible points in the reconstructed point cloud, and α m is the transparency of the mth visible point, and β and γ are set constants.

[0043] Furthermore, the gradient matching loss The expression is as follows:

[0044]

[0045] Where: H and W are the width and height of the original image, i and j are the index numbers of the row and column in the original image, respectively. is the pixel value of the i-th row and j-th column in the original image, I i,j Indicates the pixel value of row i and column j in the rendered image corresponding to the reconstructed point cloud generated by the ReducePro3DGS model when the initial point cloud of the original image is input. and Represent the gradient operators in the horizontal and vertical directions respectively.

[0046] This paper proposes a new dynamic optimization framework to enhance the compactness and rendering quality of Reduce 3DGS. Through adaptive sparsity constraints and gradient-aware detail enhancement, this framework improves the fidelity of textureless regions and high-frequency details while maintaining low model memory usage. The core concept of this method is to guide Gaussian primitives to generate a more compact distribution while preserving key visual features through dynamic loss scheduling and geometry-aware regularization. Experiments show that this method achieves competitive results on multiple datasets, validating its effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 Schematic diagram of the ReducePro 3DGS model framework in the present invention.

[0048] Figure 2 Schematic diagram of the loss comparison between the model framework of the present invention and Reduce 3DGS.

[0049] Figure 3 Comparison results of the reconstructed rendered images and real images of the model framework of the present invention and Reduce 3DGS on the MipNeRF360 dataset. DETAILED DESCRIPTION

[0050] In order to describe the present invention more specifically, the technical solution of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0051] The present invention is based on a 3DGS adaptive dynamic optimization strategy and a gradient-sensing enhanced 3D reconstruction method, comprising the following steps:

[0052] (1) Obtain multi-view images of the scene to be reconstructed and use the 3D reconstruction software COLMAP to extract the initial point cloud from the scene at each viewpoint of the image.

[0053] (2) Divide all images into training and test sets in proportion.

[0054] (3) Construct Figure 1 The ReducePro 3DGS model framework shown in FIG, includes a densification module, a redundant pruning module, and a codebook quantization module, where:

[0055] The densification module helps the model fill in gaps in the sparse point cloud. This implementation uses the densification module in Reduce3DGS. Starting with an initial sparse point set from SfM (Structure from Motion), the densification module adaptively controls the number and density of Gaussian points per unit volume, enabling it to progress from the initial sparse Gaussian set to a denser set that better represents the scene and has the correct parameters. Densification is performed every 100 iterations. Any inherently transparent Gaussian distributions below a threshold are removed. For small Gaussian functions in underreconstructed regions, new geometry must be created to cover them. In these cases, it is best to clone the Gaussian function by simply creating a copy of the same size and shifting it in the direction of the position gradient. On the other hand, large Gaussian distributions in high-variance regions need to be split into smaller Gaussian distributions. These are replaced by two new Gaussian functions, and their scales are divided by an experimentally determined factor of 1.6. This results in a dense point cloud, but this point cloud model contains a large number of unnecessary redundant points.

[0056] The redundant point pruning module is used to prune redundant points. This implementation uses the pruning module in Reduce 3DGS. For each 3D Gaussian primitive, its pixel footprint across all views is calculated. The pixel footprint refers to the size of the pixel area occupied by the primitive when viewed from a particular view. The smallest pixel footprint is selected to determine the spatial region surrounding the primitive. A spherical region is constructed with the primitive as the center and a radius of half the diagonal length of the smallest pixel footprint. The amount of overlap with other Gaussian primitives within this region is then calculated. If a primitive has too many other primitives in the region surrounding it, it may be redundant. The redundancy score is recorded and propagated. An adaptive threshold is then set based on the mean and standard deviation of the scores (maximum of 3). After sorting, 50% of the primitives with scores exceeding the threshold are deleted (prioritizing those with low opacity).

[0057] The codebook quantization module uses K-means clustering to create a codebook to further compress the memory usage of the point cloud model. This embodiment is implemented using the quantization module in Reduce 3DGS. It uses K-means clustering to create a codebook. That is, it does not store the exact value of each primitive attribute. Instead, it uses a 1-byte index to point to the approximate value of the attribute in the codebook. Codebooks are created for attributes such as the opacity, scaling component, and rotation component of the Gaussian point. Then, 16-bit half-floating point quantization is performed on the position and codebook entries to further compress the memory usage of the point cloud model.

[0058] (4) Use the image and initial point cloud obtained in step (1) to train the ReducePro 3DGS model. The specific process is as follows:

[0059] 4.1 Initialize model parameters, initialize the point cloud through SFM (Structure from Motion), and use 3D Gaussian to represent each point in the scene;

[0060] 4.2 Input the prepared multi-view point cloud into the model for reconstruction, and obtain the rendered images from each viewpoint through rendering;

[0061] 4.3 Based on the rendered images and the real images, the model is optimized using a multi-stage mixed loss function until convergence, and the training is completed.

[0062] The multi-stage hybrid loss function used in the training process applies adaptive loss to dynamically adjust weights in the densification stage for geometric reconstruction. In the redundant point pruning stage, adaptive loss, dynamic sparse loss, and gradient-aware loss are jointly optimized to constrain the geometric consistency of the rendered results and the real results, compensating for the detail loss caused by codebook quantization.

[0063] The multi-stage hybrid loss function is applied in stages throughout the training cycle. There are two loss functions in total. This implementation uses 15,000 rounds as the dividing line. The training function for the first stage is as follows:

[0064]

[0065] The loss function mainly consists of three parts. The first part is the mean absolute error loss.

[0066]

[0067] This part mainly calculates the average value of the absolute difference between the output and the true value, where I i,j Represents a specified pixel of the rendered image, Represents a specified pixel of the real image, N represents the total number of pixels, and λ1 = 0.001.

[0068] This implementation applies loss1 before a specified training round. In the early stages of training, the geometric structure needs to be quickly established, while in the later stages of training, details need to be refined. Fixed weights prevent the dynamic adaptation of the two and also cause the loss of some high-frequency details. Therefore, this implementation adopts an adaptive loss weight adjustment strategy based on the Sigmoid function:

[0069]

[0070] Where: start is the initial value of the weight, Δλ is the difference between the initial value and the final value (set), and k controls the speed of change. Entering the rapid adjustment stage, Represents the current iteration progress, a represents the current iteration round, T represents the total training rounds, and p = 0.4.

[0071] When training is early, λ dssim A larger value focuses on global structure alignment, accelerates scene geometry initialization, and can speed up the convergence of basic geometry; as training progresses, λ dssim Smoothly descend and transition to the detail optimization stage. In the later stages of training, it can better enhance pixel-level accuracy and retain high-frequency details.

[0072] The second part is the structural similarity loss

[0073]

[0074] Among them: I, I gt Represent the rendered image and the real image respectively, μ x and μ y is image I,I gt The local mean of (calculated by convolution), and is image I,I gt The local variance, σ xy is image I,I gt Local covariance, C1 = (0.01) 2 ,C2=(0.03) 2 , used to improve visual quality.

[0075] The third part is the regularization loss

[0076]

[0077] Where: α mis the transparency of visible points, M is the number of visible points (points with transparency greater than 0), which is used to push the transparency of unimportant Gaussian points close to zero.

[0078] The present invention applies loss2 after a specified number of training rounds. The loss function for the second stage is as follows:

[0079]

[0080] The loss function of the second stage adds dynamic sparse optimization and gradient-aware enhancement to the loss function of the first stage, where:

[0081]

[0082] To maintain color accuracy during model pruning of Gaussian primitives, a color difference loss is introduced during the sparsification process. And it is optimized in coordination with the dynamic weight scheduling mechanism:

[0083]

[0084] Where H × W is the image resolution. The color consistency constraint uses dynamic weight scheduling to balance color fidelity and geometric optimization objectives at different stages of training, avoiding color casts caused by quantization and pruning. It also incorporates sparsity regularization to maintain high compression rates while maintaining high reconstruction quality. β = 0.001, λ2 = 1.

[0085] Dynamic sparse optimization interacts with the redundant point pruning module to avoid insufficient scene coverage caused by excessive redundant pruning. Sparse dynamic optimization is based on the distribution statistics of visible point transparency:

[0086]

[0087] Where: γ=0.001, when α m When it is close to 0 or 1, the entropy is the smallest ( maximized); on the contrary, when α m = 0.5 when the entropy is the largest ( By minimizing the loss, the transparency of the primitives is forced to be distributed towards the two extremes, and the idea of dynamic adjustment is introduced again.

[0088] Gradient-aware detail enhancement In order to alleviate the problem of high-frequency detail loss caused by model compression (quantization and primitive pruning), the present invention proposes a gradient matching loss. By constraining the gradient consistency between the rendered image and the real image, the perceptual similarity before and after reconstruction is improved. By forcing the rendered image to retain high-frequency information such as edges and textures in the real image, the blurring and detail degradation caused by quantization or primitive sparsification are avoided.

[0089]

[0090] in: Represent the gradient operators in the horizontal and vertical directions respectively, λ3 = 0.3.

[0091] like Figure 2 As shown, the adaptive loss weight adjustment strategy based on the Sigmoid function of the present invention has a larger loss in the early stage of training, but can be adjusted quickly during the adaptive adjustment process and obtains a better convergence effect during the entire training process.

[0092] In summary, the present invention is based on Reduce 3DGS and combines it with our carefully designed dynamic optimization module, and the newly generated framework is named ReducePro 3DGS. Specifically, the model first represents the initial point cloud with a three-dimensional Gaussian, then densifies the sparse three-dimensional Gaussian point cloud, and uses an adaptive loss strategy to quickly construct the geometry in the densification stage. After the densification is completed, each three-dimensional Gaussian is scored for redundancy, and then these Gaussians with high redundancy values are cropped. At the same time, dynamic sparsification is applied to protect Gaussian points with large contributions, and gradient-aware enhancement is applied to alleviate the problem of high-frequency detail loss caused by model compression (quantization and primitive pruning). Then, images of each perspective of the reconstructed scene are obtained through rasterization rendering.

[0093] Verification example:

[0094] In the experimental section, we conducted validation on three representative datasets: Tanks & Temples, Deep Blending, and MipNeRF360. These datasets contain two, two, and nine scenes, respectively. These datasets cover both bounded indoor and unbounded outdoor scenes, forming a diverse test scenario system. To comprehensively evaluate the performance of our method, we used common quality evaluation metrics such as Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), and Perceptual Similarity (LPIPS). We also analyzed the model's final memory usage, aiming to systematically analyze the effectiveness of our method from multiple dimensions, including visual quality, perceptual consistency, and resource efficiency.

[0095] This method is based on the open-source Reduce 3DGS framework. Building on its core architecture, it dynamically improves the loss function for storage efficiency and reconstruction accuracy. The overall experimental process follows the Reduce 3DGS framework while incorporating an adaptive loss strategy, dynamic sparse optimization, and gradient-aware detail enhancement to form a complete experimental verification process. In this method, the adaptive loss strategy and the complete training process are performed simultaneously, and all experiments were conducted on an RTX 4090 GPU.

[0096] As shown in Table 1, the effectiveness of various solutions is demonstrated on the Tanks&Temples dataset. We set λ start =0.5, Δλ=0.3, k=15, p=0.4, γ=0.05, β=0.2. Compared to Reduce 3DGS, our method consistently reduces primitives and has a positive impact on PSNR, SSIM, and LPIPS. Furthermore, compared to previous state-of-the-art methods, we achieve the desired balance between reconstruction quality and storage efficiency.

[0097] Table 1

[0098]

[0099]

[0100] As shown in Table 2, the performance of various solutions on the Deep Blending dataset, we set λ start =0.2, Δλ=0.1, k=15, p=0.4, γ=0.001, β=0.001. Similarly, the present invention outperforms the previous methods in terms of SSIM and PSNR, and ranks second in the LPIPS index. The primitives also continue to decrease compared to Reduce 3DGS.

[0101] Table 2

[0102]

[0103] As shown in Table 3, the performance of various solutions on ReducePro 3DGS is evaluated on the MipNeRF360 dataset. We set λ start =0.3, Δλ=0.2, k=15, p=0.4, γ=0.001, β=0.001. Since the MipNeRF360 dataset is a 360-degree surround scene, it is suitable for the reconstruction of unbounded or large-scale scenes. More primitives are needed to maintain a better reconstruction effect. However, the method of the present invention still reaches the suboptimal level among all the methods, and the required memory usage is still lower than that of the Reduce3DGS method.

[0104] Table 3

[0105]

[0106] We also conducted an ablation study on the Tanks & Temples dataset for each of the proposed optimization steps. As shown in Table 4, the adaptive loss strategy, dynamic sparsity optimization, and gradient-aware detail enhancement proposed in this paper are all effective. We found that the adaptive loss strategy had a positive impact on PSNR, SSIM, and LPIPS, but with a slight decrease in storage efficiency. However, dynamic sparsity optimization allowed us to prune more redundant primitives, but at the same time, some high-frequency details were lost. Gradient-aware enhancement compensated for this shortcoming, ultimately achieving a balance between reconstruction quality and storage efficiency.

[0107] Table 4

[0108]

[0109] In addition, we also conducted qualitative experimental results analysis and visualized the test results of the proposed method and the baseline model Reduce3DGS on the MipNeRF360 dataset. Figure 3 As shown, the reconstruction quality of the proposed method is better than that of the baseline method.

[0110] In summary, the method of the present invention is based on dynamic optimization strategy and gradient-aware enhancement as its core. By inputting the initial sparse point cloud into the densification module, it fills the blank areas in the scene and uses an adaptive loss mechanism to accelerate the geometric initialization in the early stage of training, while enhancing the detail modeling effect in the later stage of training. At the same time, the present invention introduces redundant point pruning and dynamic sparse optimization modules to effectively compress the model memory overhead while ensuring the reconstruction quality, and enhance the sparsity and coverage accuracy of the primitive distribution. In addition, the present invention constrains the edge consistency between the rendered image and the real image through the gradient-aware detail enhancement module, improves the ability to restore high-frequency information, avoids the problem of blur degradation, and combines color difference loss with sparse regularization to further balance color fidelity and compression rate. The method of the present invention was trained and verified on multi-view images of real scenes, achieving excellent reconstruction results. At the same time, we also conducted sufficient modular ablation experiments to make the effectiveness of the overall framework and the necessity of each component more convincing. In addition, we also visualized the experimental results to further demonstrate the significant improvement of the present invention in three-dimensional reconstruction performance.

[0111] The above description of the embodiments is intended to facilitate understanding and application of the present invention by those skilled in the art. It is apparent that those skilled in the art can readily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without requiring creative effort. Therefore, the present invention is not limited to the above embodiments. Any improvements or modifications made by those skilled in the art based on the disclosure of the present invention should fall within the scope of protection of the present invention.

Claims

1. A 3D reconstruction method based on 3DGS adaptive dynamic optimization strategy and gradient perception enhancement, comprising the following steps: (1) Obtain multi-view images of the reconstructed scene and extract the initial point cloud from the images; (2) Divide all images into training and test sets in proportion; (3) Construct the ReducePro 3DGS model, which includes: The densification module is used to densify the initial point cloud of the image to fill the blank areas of the scene and obtain a dense point cloud; The redundant point pruning module calculates the redundancy value of each Gaussian point in the dense point cloud and prunes out overly dense Gaussian points according to the set redundancy threshold to reduce the memory usage of the dense point cloud; The codebook quantization module uses K-means clustering to create a codebook to further compress the memory usage of the dense point cloud, thereby generating a reconstructed point cloud; (4) The ReducePro 3DGS model is trained using the training set. In the densification phase, the adaptive loss is applied to dynamically adjust the weights for geometric reconstruction. In the redundant point pruning phase, the adaptive loss, dynamic sparse loss, and gradient-aware loss are jointly optimized to constrain the geometric consistency between the rendered result and the real result, and to compensate for the detail loss caused by codebook quantization. (5) Input the initial point cloud of the test set image into the trained ReducePro 3DGS model to automatically generate the reconstructed point cloud.

2. The 3DGS adaptive dynamic optimization strategy and gradient-aware enhanced 3D reconstruction method according to claim 1, characterized in that: The codebook quantization module uses K-means clustering to create a codebook, using 1-byte indexes to point to the approximate values of attributes in the codebook. Codebooks are established for Gaussian point attributes including opacity, scaling components, and rotation components. Positions and codebook entries are then quantized to 16-bit half-floating points to further reduce the memory usage of dense point clouds.

3. The 3DGS adaptive dynamic optimization strategy and gradient-aware enhanced 3D reconstruction method according to claim 1, characterized in that: In the process of training the ReducePro 3DGS model in step (4), the loss function loss1 is used in the early stage of training, and the loss function loss2 is used in the late stage of training. The specific expressions are as follows: Where: dssim is the adaptive weight, is the mean absolute error loss, is the structural similarity loss, is the regularization loss, is the dynamic sparse loss, is the gradient matching loss, and λ1~λ3 are the set weight coefficients.

4. The 3DGS adaptive dynamic optimization strategy and gradient-aware enhanced 3D reconstruction method according to claim 2, characterized in that: The adaptive weight λ dssim The expression is as follows: Where: start is the initial value of the weight, Δλ is the difference between the initial value and the final value of the weight, k is the coefficient that controls the speed of change, a is the current iteration round, T is the total number of training rounds, e is a natural constant, and p is the set progress.

5. The 3DGS adaptive dynamic optimization strategy and gradient-aware enhanced 3D reconstruction method according to claim 2, characterized in that: The mean absolute error loss The expression is as follows: Where: i and j are the index numbers of the row and column in the original image respectively, is the pixel value of the i-th row and j-th column in the original image, I i,j It represents the pixel value of the i-th row and j-th column in the rendered image corresponding to the reconstructed point cloud generated by the ReducePro 3DGS model when the initial point cloud of the original image is input. N represents the total number of pixels in the original image.

6. The 3DGS adaptive dynamic optimization strategy and gradient-aware enhanced 3D reconstruction method according to claim 2, characterized in that: The structural similarity loss The expression is as follows: Among them: I gt and I represent the original image and its initial point cloud input, respectively. The ReducePro 3DGS model generates the rendered image corresponding to the reconstructed point cloud. SSIM(I,I gt ) is I and I gt The structural similarity of x and μ y I and I respectively gt The local mean of and I and I respectively gt The local variance, σ xy for I and I gt The local covariance of , C1 and C2 are set constants.

7. The 3DGS adaptive dynamic optimization strategy and gradient-aware enhanced 3D reconstruction method according to claim 2, characterized in that: The regularization loss The expression is as follows: Where M is the number of visible points in the reconstructed point cloud, α m is the transparency of the mth visible point.

8. The 3DGS adaptive dynamic optimization strategy and gradient-aware enhanced 3D reconstruction method according to claim 2, characterized in that: The dynamic sparsity loss The expression is as follows: w consist =β·t w stability =γ(1-t) in: is the color difference loss, is the sparse regularization loss, w consist and w stability are dynamic weights, H and W are the width and height of the original image, i and j are the index numbers of the row and column in the original image, is the pixel value of the i-th row and j-th column in the original image, I i,j represents the pixel value of the i-th row and j-th column in the rendered image corresponding to the initial point cloud input of the original image into the ReducePro 3DGS model to generate the reconstructed point cloud. t represents the current iteration progress and t = a / T, where a is the current iteration round, T is the total training rounds, M is the number of visible points in the reconstructed point cloud, and α m is the transparency of the mth visible point, and β and γ are set constants.

9. The 3DGS adaptive dynamic optimization strategy and gradient-aware enhanced 3D reconstruction method according to claim 2, characterized in that: The gradient matching loss The expression is as follows: Where: H and W are the width and height of the original image, i and j are the index numbers of the row and column in the original image, respectively. is the pixel value of the i-th row and j-th column in the original image, I i,j Indicates the pixel value of row i and column j in the rendered image corresponding to the reconstructed point cloud generated by the ReducePro 3DGS model when the original image initial point cloud is input. and Represent the gradient operators in the horizontal and vertical directions respectively.

Images