Underwater scene 3D reconstruction method based on 3DGS and surface aggregation optimization

Abstract

This invention proposes a 3D underwater scene reconstruction method based on 3DGS and surface aggregation optimization, belonging to the field of computer vision and 3D reconstruction. The method first preprocesses the underwater image and constructs an initial Gaussian set; through a single-surface Gaussian redistribution strategy, the optimization attention is focused on the geometric surface, effectively suppressing medium obfuscation and levitation artifacts; a composite loss function integrating depth weighting, backscattering constraints, and color correction is designed to improve color restoration and detail reconstruction capabilities; a gradient modulation mechanism based on viewpoint depth is introduced to enhance training stability. This invention significantly improves the geometric accuracy and visual fidelity of underwater scene reconstruction while maintaining the advantages of efficient real-time rendering of 3DGS, and requires no additional annotation, demonstrating good practicality and application prospects.

Description

Technical Field

[0001] This invention relates to the field of computer vision and 3D reconstruction, specifically to a method for 3D reconstruction of underwater scenes based on 3DGS and surface aggregation optimization, belonging to the field of underwater visual enhancement and 3D scene reconstruction. Background Technology

[0002] Underwater 3D reconstruction technology is a core technology in fields such as marine exploration, underwater archaeology, and ecological monitoring. However, the absorption and scattering effects of light by the underwater medium cause problems such as color distortion and decreased contrast in images, posing a significant challenge to traditional reconstruction methods.

[0003] Traditional underwater reconstruction processes based on Structure of Motion (SfM) and Multi-View Stereo Vision (MVS) are complex, and the reconstruction results often exhibit phenomena such as model holes and texture blurring. While Neural Radiation Field (NeRF) and its variants improve reconstruction quality, their implicit representation can easily confuse water scattering effects with scene geometry, leading to geometric "bloat" and "floating object" artifacts. Furthermore, their training and rendering speeds are slow, making them impractical.

[0004] 3D Gaussian sputtering (3DGS), as an emerging explicit representation method, has achieved a good balance between training speed and rendering quality. However, its direct application to underwater environments still faces three major challenges: 1) Underwater optical distortion interferes with the optimization of the geometric and color attributes of Gaussian elements, resulting in media confusion artifacts; 2) Dynamic interferences such as suspended particles and plankton in the water can disrupt image consistency, leading to erroneous geometry in reconstruction; 3) The initial point cloud is sparse and uneven, affecting the integrity and accuracy of the reconstruction.

[0005] Existing technologies lack a 3DGS solution that can collaboratively handle media effects and dynamic disturbances in underwater environments while ensuring efficient and high-precision reconstruction. Summary of the Invention

[0006] To address the above problems, this invention proposes a 3D underwater scene reconstruction method based on 3DGS and surface aggregation optimization. This method integrates the advantages of explicit geometric representation and volumetric media rendering, and effectively suppresses dynamic interference such as suspended particles through a surface aggregation optimization strategy, significantly improving the reconstruction quality of underwater scenes while maintaining real-time rendering capabilities. The specific solution of this invention is as follows:

[0007] The underwater scene 3D reconstruction method based on 3DGS and surface aggregation optimization includes the following steps:

[0008] S1: Underwater image data acquisition and preprocessing;

[0009] S2: 3DGS initialization and media parameter estimation;

[0010] S3: Construct a weight redistribution strategy based on surface aggregation optimization;

[0011] S4: Construct a weighted reconstruction loss function that incorporates medium constraints;

[0012] S5: Enables adaptive optimization and scene reconstruction.

[0013] Furthermore, in the experiment of step S1, the SeaThru-NeRF dataset was used as the experimental data source, which contains image sequences of different underwater scenes. Data preprocessing included white balance correction, image denoising, resolution normalization, and COLMAP was used for camera pose estimation and point cloud initialization.

[0014] Furthermore, step S2 implements 3DGS initialization and media parameter estimation, specifically as follows:

[0015] S21. The initial point cloud is obtained using the Structure for Motion Reconstruction (SfM) method and initialized as a set of 3D Gaussian elements;

[0016] S22. Construct a lightweight multilayer perceptron (MLP) network to estimate global medium parameters, including attenuation coefficients. Backscattering coefficient and infinitely far scattering colors .

[0017] Furthermore, in step S3, a weight redistribution strategy based on surface aggregation optimization is constructed, specifically as follows:

[0018] S31. To reduce the interference of the medium on geometric reconstruction, we introduce a single-surface aggregation strategy. For each ray, we calculate its accumulated weight distribution function along the ray, defined as the weight distribution function from the near-plane. To depth Volume rendering weight integral:

[0019]

[0020] in For depth The volume density at that location, Transmittance. Define surface location. For cumulative weights Depth value when it first reaches 0.5:

[0021]

[0022] S32. The weight distribution is redistributed using a single Gaussian distribution, causing optimization attention to focus on the surface. In the continuous domain, the redistributed weight function is expressed as:

[0023]

[0024] in The standard deviation is 0.3-0.6, with the optimal experimental value being 0.5.

[0025] S33, Introduce a fixed offset Low-density representation of the enhanced medium region:

[0026]

[0027] This strategy focuses the optimization process more on the geometric surface, reducing overfitting to the medium region.

[0028] Furthermore, the construction of the weighted reconstruction loss function with fusion medium constraints in step S4 is specifically as follows:

[0029] S41, 3DGS core loss: ,in ;

[0030] The L1 reconstruction loss is defined as follows: ,in For the image height and width, For real images in pixels RGB values ​​at that location To render an image in pixels RGB values ​​at that location It represents the absolute value.

[0031] The structural similarity loss is defined as follows:

[0032]

[0033] The formula for calculating the SSIM index is as follows:

[0034]

[0035] in and It is an image and The mean, and Represents variance. It is covariance. and It is a stability constant to avoid a denominator of 0.

[0036] S42. Introduce depth-weighted loss: ,in Render images for the model. For real images, This is a depth map (gradient truncation) rendered from the current 3D Gaussian model. This represents the L1 norm.

[0037] S43. Introducing medium constraint loss:

[0038] Backscattering loss (constraint and (Rationality)

[0039]

[0040] in To remove the backscattered direct image, ( Gradient cutoff (to avoid interfering with geometric learning), Hyperparameters are used to balance positive and negative biases.

[0041] Grayscale color loss (Correcting color channel imbalance):

[0042]

[0043] in This method generates media-free, true-color images using 3D Gaussian rendering, forcing the mean value of each channel to be close to 0.5, thus alleviating the problem of underwater images being too blue-green and lacking red.

[0044] S44. Overall Loss Function:

[0045]

[0046] The weighting coefficients are set as follows: (Original depth loss weights) (Backscattering loss). (Gray world loss) has a moderate weight configuration, avoids new losses dominating optimization, and has strong synergy with the original loss.

[0047] Furthermore, in step S5, adaptive optimization and scene reconstruction are implemented, specifically as follows:

[0048] S51. Use the RAdam optimizer, with the initial learning rate set to the original 3DGS learning rate. This is multiplied to compensate for the gradient magnitude change caused by the sum of the loss function coefficients not being equal to 1;

[0049] S52. During backpropagation, to further optimize the model's ability to learn distant geometry, a deep gradient guidance strategy is introduced to stabilize the learning of distant geometry and prevent overfitting:

[0050] First, calculate the average Euclidean depth of all visible Gaussian elements from the current training perspective. :

[0051]

[0052] in The center position of the camera. Let i be the three-dimensional position of the i-th Gaussian center. This represents the Gaussian density from this viewpoint. A depth threshold T is set. h =20.0 (unit consistent with scene scale, e.g., meters). When At that time, assuming that the current viewpoint mainly includes the distant view, the gradients of all Gaussian meta-parameters under this viewpoint are modulated:

[0053]

[0054] in Let represent the learnable parameters of the i-th Gaussian element. This mechanism decays the gradient based on the squared relative distance between the Gaussian element and the camera; the decay is stronger as the distance increases, thereby suppressing over-optimization of distant geometry and improving training stability.

[0055] S53. Periodically perform Gaussian unit densification and pruning operations to remove Gaussian units with transparency below 0.005;

[0056] S54 outputs the final 3D Gaussian scene representation, supporting real-time rendering and color restoration. Attached Figure Description

[0057] Figure 1 Single-surface aggregation optimization strategy flowchart

[0058] Figure 2 Schematic diagram of underwater scene 3D reconstruction framework based on 3DGS and surface aggregation optimization

[0059] Figure 3 Comparison of underwater scene reconstruction results Detailed Implementation

[0060] S1: Underwater image data acquisition and preprocessing;

[0061] S11, Data Acquisition Phase:

[0062] In the experiments of this invention, the SeaThru-NeRF dataset was used as the experimental data source. This dataset contains four real underwater scenarios: IU13 Red Sea, Curaçao, Japan's Garden-like Red Sea, and Panama, covering different water quality conditions and scenario complexity. Each scenario contains 18-29 high-resolution images with a resolution of 900×1400 pixels.

[0063] S12, Data Preprocessing Stage:

[0064] First, white balance correction was performed on the RAW format image, and extreme noise pixels were removed by 0.5% channel cropping. Then, COLMAP software was used for camera pose estimation and lens distortion correction. Finally, the image was downsampled to a uniform resolution (800×1200) and contrast enhancement was performed.

[0065] S2, 3DGS initialization and media parameter estimation;

[0066] S21. Convert the sparse point cloud generated by COLMAP into a 3D Gaussian set, and initialize the Gaussian parameters:

[0067] Location: The three-dimensional coordinates of the sparse point cloud are directly used as the Gaussian center location;

[0068] Covariance: The initial scale factor is calculated based on the k nearest neighbor distance for each point;

[0069] Opacity: Initialize to 0.1 to avoid the initial rendering being too sparse or dense;

[0070] Color: Represented using zero-order spherical harmonic coefficients (no viewpoint dependence), initialized to neutral gray (0.5 for each RGB channel);

[0071] Quantity: The initial number of Gaussian elements is approximately 5 × 10⁻⁶. 4 Up to 1×10 5 One, covering the entire scene space.

[0072] S22. Construct a media parameter estimation module to learn global media parameters:

[0073] Parameter to be estimated: Channel-specific attenuation coefficient (Independent RGB three channels), channel-specific backscattering coefficient (Independent RGB three channels), infinity backscattered color (RGB three channels, value range [0,1]);

[0074] Implementation: Construct a lightweight MLP with two linear layers (hidden units 128), with the ray direction as the input and the above three types of medium parameters as the output; and All are implemented using convolution kernels of size (1,1,1,3), which can be efficiently applied to depth maps; Learned directly as a differentiable parameter;

[0075] Activation constraints: and The Softplus activation function is used to ensure that it is a positive number; for The Sigmoid activation function is used to constrain the system to the [0,1] interval to ensure physical plausibility.

[0076] S3. Construct a weight redistribution strategy based on surface aggregation optimization:

[0077] In the 3DGS framework, the scene is composed of a set of learnable Gaussian meta-meta ... ,in Indicates the center position. Let covariance matrix be the variance matrix. Indicates opacity. Color (represented by spherical harmonic coefficients).

[0078] In traditional 3DGS, the color of each ray of light is determined by... Mixed calculation:

[0079]

[0080] S31. When rendering each ray, first assign depth values ​​to all Gaussian elements that intersect with that ray. Sort the z-values ​​in camera coordinates in ascending order, then calculate the contribution weight of each Gaussian along the ray. :

[0081]

[0082] in For Gauss at point The evaluation value at that location.

[0083] S32. Calculate the cumulative weight function and find the depth index when the cumulative weight reaches 0.5. and the corresponding Gaussian depth As an estimate of the surface position of the light source :

[0084] ,

[0085] S33, Based on estimated surface position Reweight the opacity of each Gaussian and implement a surface clustering strategy:

[0086]

[0087] η controls the aggregation intensity (set to 0.5). This is essentially a spatial attention mechanism that focuses optimization efforts on the surface region.

[0088] S34. Apply a fixed offset base=0.2 to fine-tune the opacity after reweighting to ensure that the low-density Gaussian in the medium region still retains a certain contribution:

[0089]

[0090] S35. Render the final pixel color image using the reweighted Gaussian elements and the learned medium model:

[0091]

[0092] in It is calculated from the spherical harmonic function and the direction of observation.

[0093] S4. Construct a new weighted reconstruction loss function:

[0094] S41. In each forward propagation, render the color image in parallel. and depth map ;

[0095] S42. Synthesizing images containing media using physical models:

[0096] To enhance the model's ability to model underwater media effects, the following physical model was used during training to synthesize images containing the medium:

[0097]

[0098] in For clean scene images (no media), For depth map, , , These are the medium parameters.

[0099] S43. Construct a weighted reconstruction loss function that integrates core medium constraints. Perform steps S41-S43. All loss terms are backpropagated through differentiable calculations. The medium constraint loss term specifically guides the learning of medium parameters and the physical rationality of the 3D Gaussian model.

[0100] S5, Adaptive Optimization and Scene Reconstruction:

[0101] S51. Use the RAdam optimizer and set the learning rate to 0.77 times the initial 3DGS.

[0102] S52. Implement a depth gradient guidance strategy: In each training iteration, calculate the average depth of the viewpoint corresponding to the current batch. :like T h Then, a modulation factor is applied to the gradient of all Gaussian metaparameters from that perspective.

[0103]

[0104] S53. Perform Gaussian cloning, splitting, and pruning (removing Gaussians with transparency <0.005) operations according to the original 3DGS cycle.

[0105] S54. After multiple iterations and optimizations, the final 3D Gaussian scene representation is output. This representation supports real-time, high-quality viewpoint rendering and can perform color correction through its built-in media model.

[0106] Training and validation were performed using the SeaThru-NeRF dataset, as follows: Figure 3 The ablation experiment results are as follows:

[0107] Table 1. Comparison of reconstruction results with different methods added to 3DGS (Surface Aggregation 'S', Constraint Loss 'D', Gradient Modulation 'G').

[0108] method Average PSNR (dB) Average SSIM Average LPIPS 3DGS 24.533 0.850 0.255 3DGS+ Surface Aggregation (S) 27.456 0.882 0.213 3DGS+S+Constraint Loss (D) 28.812 0.895 0.196 3DGS+S+D+G (This invention) 32.851 0.932 0.123

[0109] This invention, based on the 3DGS framework, proposes an optimization method that integrates surface aggregation, medium constraint loss, and adaptive gradient modulation to address the problems of medium interference and dynamic noise in underwater scene reconstruction. As shown in Table 1, in the quantitative evaluation on the SeaThru-NeRF dataset, this method significantly outperforms the basic 3DGS and some improved schemes in PSNR (32.851 dB), SSIM (0.932), and LPIPS (0.123), comprehensively improving the geometric accuracy, texture fidelity, and visual perception quality of underwater reconstruction.

Claims

1. A method for three-dimensional reconstruction of underwater scenes based on 3DGS and surface aggregation optimization, characterized in that, Includes the following steps: S1: Underwater image data acquisition and preprocessing; S2: 3DGS initialization and media parameter estimation; S3: Construct a weight redistribution strategy based on surface aggregation optimization; S4: Construct a weighted reconstruction loss function that incorporates medium constraints; S5: Enables adaptive optimization and scene reconstruction.

2. The method according to claim 1, characterized in that: In step S1, the preprocessing of the image sequences in the SeaThru-NeRF dataset includes: white balance correction, noise suppression, and resolution normalization; the camera pose estimation and sparse point cloud initialization are performed using the COLMAP tool.

3. The method according to claim 1, characterized in that, Step S3 includes: S31: For each ray, calculate its cumulative weight distribution function along the ray, and determine the depth value at which the cumulative weight first reaches 0.5 as the surface position. ; S32: Based on surface location The weight distribution is redistributed using a single Gaussian distribution to obtain the redistributed weight function; S33: Introduce a fixed offset on the basis of redistribution weights to enhance the low-density representation of non-surface regions.

4. The method according to claim 1, characterized in that: the weighted reconstruction loss function in step S4 is defined as:

5. Among them This is the core loss of 3DGS. For deep weighted reconstruction loss, For backscattering constraint, For grayscale color correction loss, , , These are the corresponding weighting coefficients.

6. The method according to claim 4, characterized in that, The backscattering constraint loss The calculation method is as follows:

7. Among them To remove the backscattered direct image, ( Gradient cutoff (to avoid interfering with geometric learning), Hyperparameters are used to balance positive and negative biases.

8. The method according to claim 1, characterized in that, The adaptive optimization strategy in step S5 includes: dynamically adjusting the gradient magnitude based on the average depth of the current training viewpoint; when the average depth is greater than a set threshold... At that time, the gradient is weighted and decayed based on the squared distance, with the decay factor being:

9. Among them For camera position, For the position of Gausky's element, The average depth of the viewpoint.

Images