End-to-end underwater three-dimensional reconstruction method and system based on underwater imaging model

Abstract

The application discloses an end-to-end underwater three-dimensional reconstruction method and system based on an underwater imaging model, and belongs to the technical field of underwater image processing.The application combines deep learning pose estimation with an underwater imaging model; firstly, a plurality of underwater image sequences are collected as input to construct a deep neural network, the network mainly comprises two core sub-modules: one is a pose estimation network; and the other is a three-dimensional reconstruction network, according to the predicted pose and image content, dense point cloud or voxel reconstruction is completed, the pose estimation and the three-dimensional reconstruction process are integrated in the same system to realize overall joint optimization; and the adaptive underwater imaging model is combined to model the physical processes such as underwater light attenuation and scattering, so that the realism and accuracy of the reconstruction result are improved.The application realizes high-quality three-dimensional reconstruction of images in a complex underwater environment, and can be widely applied to ocean engineering, underwater robots, seabed topographic mapping and underwater cultural relic protection.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, image processing, and 3D reconstruction technology, and specifically to an end-to-end underwater 3D reconstruction method and system based on an underwater imaging model. Background Technology

[0002] With the rapid development of marine resource development, seabed exploration, marine engineering, underwater robotics, and other fields, higher demands are being placed on the perception and 3D modeling of the underwater environment. As an important component of underwater perception, underwater 3D reconstruction technology restores the geometric structure of target objects through image information, which is of great significance for ensuring the safety of underwater operations and improving automated operation capabilities.

[0003] However, compared to terrestrial environments, underwater scenes are characterized by complex lighting conditions, variable medium properties, and significant imaging degradation, primarily including rapid light attenuation, color cast, scattering blur, and overall reduced contrast. These imaging degradation factors severely degrade the quality of underwater images, further impacting key steps such as image feature extraction, cross-view matching, and camera pose estimation, thus significantly limiting the accuracy and robustness of 3D reconstruction.

[0004] Traditional underwater 3D reconstruction methods are typically based on techniques such as structured light, laser scanning, stereo vision, or SfM (Structure-from-Motion). However, these methods have the following main shortcomings:

[0005] 1. Lack of modeling of the physical mechanisms of underwater imaging: It cannot adapt to the imaging degradation characteristics under different water quality environments, affecting image quality restoration and subsequent modeling accuracy;

[0006] 2. Highly dependent on camera pose estimation accuracy: Traditional methods rely on high-quality image features and accurate camera calibration, which can easily lead to positioning errors or reconstruction failures in dynamic scenes or areas with weak textures.

[0007] 3. Complex processing flow and poor robustness: It usually requires cumbersome image preprocessing steps (such as color correction, white balance, scatter removal, etc.), resulting in insufficient system stability and adaptability;

[0008] 4. Insufficient end-to-end capabilities: Most methods are modularly designed, and the various stages cannot be jointly optimized, which can easily lead to error propagation and information loss.

[0009] In recent years, the rapid development of deep learning technology, especially methods based on Neural Radiation Field (NeRF) and 3D Gaussian Splatting, has provided new insights for underwater 3D reconstruction. These methods allow researchers to simulate underwater light propagation and 3D volumetric structures with greater precision, thereby generating high-quality 3D reconstruction results. However, these methods typically require traditional camera pose estimation techniques (such as COLMAP) to obtain accurate camera pose information, which poses a significant challenge to the real-time requirements of dynamically changing underwater environments, especially in situations involving moving equipment and complex scenes. Summary of the Invention

[0010] The purpose of this invention is to propose an end-to-end underwater 3D reconstruction method and system based on an underwater imaging model, in order to solve the problems of strong dependence on underwater image quality and camera pose estimation accuracy, poor adaptability, and insufficient robustness in existing underwater 3D reconstruction methods.

[0011] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0012] An end-to-end underwater 3D reconstruction method based on an underwater imaging model includes the following steps:

[0013] S1: Acquire multiple frames of underwater image sequences to obtain raw image data containing complex underwater environmental features such as light attenuation, scattering blur and color shift;

[0014] S2: Construct an end-to-end learnable pose estimation network to perform spatiotemporal modeling and feature extraction on the input image sequence and predict the relative six-degree-of-freedom pose between image frames; the pose estimation network structure includes: an input module, an image feature extraction module, a fusion encoding module, a Transformer encoding module, and a pose regression module;

[0015] S3: Construct an underwater imaging model to simulate the underwater image degradation process caused by the medium, and embed it into the 3D rendering process to enhance color reproduction and imaging physical consistency;

[0016] S4: Based on the pose estimation network and underwater imaging model, a dense reconstruction module based on a three-dimensional Gaussian distribution is constructed to perform three-dimensional reconstruction and output a three-dimensional reconstructed image.

[0017] Furthermore, the pose estimation network in S2 includes the following modules:

[0018] (1) Input module: Receives multiple multi-view images from underwater scenes as observation input for pose estimation, providing basic data for subsequent feature extraction and inference;

[0019] (2) Image feature extraction module: High-dimensional spatial features of the input image are extracted through a deep visual coding network to obtain a feature map containing rich spatial structure information. Then the feature map is flattened and positional encoding is added to form an image token with spatial position information, which is convenient for subsequent Transformer processing and global context modeling;

[0020] (3) Fusion coding module: Combines image tokens with a set of learnable camera tokens, which serve as an implicit representation of camera pose; This module achieves the fusion of visual information and camera priors, constructs a joint contextual representation, provides rich input semantic and geometric information for Transformer, and assists in pose inference;

[0021] (4) Transformer Encoding Module: Composed of multiple Transformer layers, including multi-head self attention, multi-head cross attention, feedforward network, residual connections, and layer normalization, among other standard components. This module performs deep modeling on the joint sequence of image tokens and camera tokens, mining spatiotemporal dependencies and structural matching cues between multiple viewpoints to achieve context-aware pose estimation.

[0022] (5) Pose Regression Module: Based on the joint features encoded by Transformer, the relative rotation (usually in the form of quaternions) and translation parameters of the image are predicted by a fully connected regression network to achieve high-precision camera pose estimation.

[0023] Furthermore, in step S2, to achieve data compatibility and seamless integration between the pose estimation network and the 3D reconstruction module, a standardized data conversion process is designed to uniformly convert the camera pose, intrinsic parameters, and sparse point cloud obtained by the pose estimation network into structured inputs that can be used by the subsequent reconstruction module; specifically, it includes the following steps:

[0024] (1) Pose transformation and format normalization: The relative pose sequence predicted by the network is converted into the global camera pose (external parameter in world coordinate system) of each frame through matrix multiplication, and then the rotation matrix is ​​transformed by quaternion to adapt to the image-pose expression format required by standard 3D reconstruction system such as COLMAP.

[0025] (2) Camera intrinsic parameter initialization and normalization: If there is no explicit calibration data, the default parameters (such as focal length fx=fy=500, principal point located at the center of the image) are used, and the size and intrinsic parameter format of all images are unified to generate the PINHOLE model parameters corresponding to each image.

[0026] (3) Image and pose synchronization packaging: Traverse the image sequence, correspond the image file name order with the predicted pose one by one, and write them into cameras.txt and images.txt to describe the camera model, external parameters and file name information of each frame image, to ensure that the image frame time sequence is consistent with the pose transformation.

[0027] (4) Sparse point cloud extraction and color addition: Read the spatial point position information from the generated initial point cloud (e.g., .ply format), extract the color of each point (if available), and generate a sparse point set with color, which is written into the standard points3D.txt format as the initial scene structure representation.

[0028] (5) Unified output standard three data format: The final output consists of three files: cameras.bin, images.txt and points3D.txt, which constitute a sparse reconstruction input format that meets the input requirements of COLMAP / 3DGS, for subsequent loading and use by the differentiable rendering module based on Gaussian representation.

[0029] The above processing flow, through structured transformation and format standardization, enables the feedforward network to directly generate sparse model input data for 3D reconstruction without the need for manual calibration or third-party SfM tools, supporting the complete closed-loop operation of the SfM-Free 3D reconstruction system.

[0030] Furthermore, the underwater imaging model includes a medium modeling module (MLP) to model the underwater light propagation and imaging process in an end-to-end learnable manner.

[0031] ;

[0032] : The pixel color ultimately observed in the camera image on channel c∈{R,G,B};

[0033] : This is the true reflected color of the 3D Gaussian point corresponding to this pixel, generated by the 3D reconstruction module;

[0034] d(x): represents the depth value corresponding to a 3D point, that is, the distance from that point to the center of the camera;

[0035] Absorption coefficient, used to simulate the exponential decay of light intensity during propagation;

[0036] Scattering coefficient, used to simulate the contribution of background light from water to the superposition of colors;

[0037] : The background light color of the medium (or the inherent color of the water body) reflects the basic offset of the water quality to the image color;

[0038] The underwater imaging model takes three-dimensional spatial coordinates (x, y, z) as input and outputs the above parameters through a multilayer perceptron structure to model the light propagation characteristics in different water environments.

[0039] Furthermore, the medium modeling module is implemented as a multilayer perceptron (MLP), receiving a three-dimensional spatial position (x, y, z) as input and outputting underwater imaging parameters for that position in each color channel, including:

[0040] a. Absorption coefficient Used to describe the energy attenuation of incident light as it propagates in water;

[0041] b. Scattering coefficient Used to model the color overlay and blurring phenomena caused by backscattering;

[0042] c. Background light color : Indicates the color of the water itself or the intensity of the background light.

[0043] Furthermore, the specific steps of S4 are as follows:

[0044] S4-1: Initialize Gaussian point set: Based on the number of input images and pose information, initialize a sparse 3D point set. Each point is represented by a learnable 3D Gaussian distribution, including parameters such as spatial location, covariance, color attributes, and opacity.

[0045] S4-2: Constructing an underwater projection mechanism: Using the camera pose and the parameters of the underwater imaging model, project three-dimensional Gaussian points onto the image plane to construct a differentiable projection rendering model that conforms to the laws of physical propagation.

[0046] S4-3: Gaussian Differentiable Rendering: Based on the 3D Gaussian differentiable rendering framework, color, transparency and coverage are fused in the image space to generate a synthetic image, and the following joint loss function is calculated.

[0047] S4-4: Color consistency loss: Minimize the pixel error between the rendered image and the real image;

[0048] S4-5: Transparency Regularization: Constrains redundant points and improves efficiency;

[0049] S4-6: Depth Consistency Loss (Optional): Constrains the geometry of the point cloud to improve the accuracy of 3D reconstruction.

[0050] S4-7: Layered optimization strategy: adopt a coarse-to-fine optimization process, increase the number of Gaussian points and image resolution in round by round, and supplement it with a redundancy removal mechanism to retain only the points that contribute significantly to the reconstruction quality.

[0051] S4-8: The final output is a high-fidelity dense point cloud with complete geometric structure, texture, and boundary details. This module can be trained in conjunction with the pose estimation network and imaging modeling module to build an end-to-end 3D reconstruction system that does not require manual calibration or third-party SfM tools.

[0052] An end-to-end underwater 3D reconstruction system based on an underwater imaging model includes the following modules:

[0053] Image data acquisition module: This module is used to acquire multiple frames of underwater image sequences and obtain raw image data;

[0054] Pose estimation network module: This module performs spatiotemporal modeling and feature extraction on the input image sequence to predict the pose between image frames;

[0055] Underwater Imaging Model Module: Used to simulate the underwater image degradation process caused by the medium and embedded into the 3D rendering process to enhance color reproduction and imaging physical consistency;

[0056] 3D Reconstruction Module: Performs 3D reconstruction based on dense reconstruction of a 3D Gaussian distribution and outputs a 3D reconstructed image.

[0057] Compared with the prior art, the advantages and technical effects of the present invention are as follows:

[0058] This invention offers several significant advantages. First, by integrating pose estimation and 3D reconstruction into the same deep neural network architecture through an end-to-end optimization framework, joint optimization is achieved. This reduces the reliance on pose prediction and image preprocessing in traditional methods, thereby improving reconstruction accuracy and robustness. Second, the introduction of an adaptive underwater imaging model enables the invention to dynamically simulate imaging degradation factors such as illumination attenuation, scattering blur, and color shift in the underwater environment. This enhances the system's adaptability to different water qualities and environmental conditions, thus improving the realism and accuracy of the reconstruction results. Furthermore, the pose estimation module, which requires no external calibration, can directly derive the camera pose from underwater image sequences, simplifying system setup and operation and improving ease of use. Regarding robustness, the system optimizes pose estimation and underwater imaging modeling, successfully addressing issues such as low image quality and scattering blur in complex underwater environments, enhancing the system's robustness and stability in dynamic and low-contrast environments. Finally, by combining 3D Gaussian Splatting and volumetric rendering technology, the generated 3D reconstruction results are highly accurate and rich in detail, accurately restoring the geometry and texture of underwater scenes. This provides high-precision 3D data support for fields such as ocean exploration, underwater robot navigation, and structural health checks, significantly improving the feasibility and reliability of practical applications.

[0059] This invention will help improve the accuracy and robustness of underwater 3D reconstruction, especially in complex dynamic environments, and promote its implementation and development in practical applications such as offshore wind power, marine aquaculture, and port construction. Therefore, this invention not only has important theoretical significance, but also plays a positive role in promoting underwater data acquisition, environmental monitoring, and resource management in practical applications. Attached Figure Description

[0060] Figure 1 This is the overall flowchart of the present invention.

[0061] Figure 2 This invention provides underwater multi-view technology. Figure 3 A schematic diagram of the framework of the 3D reconstruction method.

[0062] Figure 3 This is a diagram of the pose estimation network architecture.

[0063] Figure 4 This is a flowchart of a 3D reconstruction system.

[0064] Figure 5 This is a diagram showing the result of the three-dimensional reconstruction of the present invention. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this invention clearer, the following detailed description, in conjunction with specific embodiments and accompanying drawings, further illustrates the invention. Obviously, the described embodiments are only a portion, not all, of the embodiments disclosed in this invention. All other embodiments obtained by those skilled in the art based on the embodiments disclosed in this invention without inventive effort are within the scope of protection of this invention.

[0066] Example 1:

[0067] This embodiment proposes an end-to-end underwater 3D reconstruction method based on an underwater imaging model. By constructing a unified deep learning framework, it integrates camera pose estimation, underwater image modeling, and the 3D reconstruction process, effectively adapting to different underwater environmental changes and improving reconstruction accuracy and system robustness. This method eliminates the need for traditional external calibration or multi-stage processing, and can directly and automatically recover camera motion trajectories and dense 3D structures from multi-frame underwater image sequences, making it suitable for high-quality 3D modeling tasks in complex underwater environments. Figure 1 , Figure 2 As shown, the three-dimensional reconstruction method includes the following steps:

[0068] Step 1: Multi-frame image input and overall network construction:

[0069] First, a multi-frame underwater image sequence is continuously acquired using a multi-view underwater camera to ensure temporal continuity and spatial coverage. This yields raw image data containing complex underwater environmental features such as illumination attenuation, scattering blur, and color shift, providing rich perceptual information for subsequent pose estimation and 3D reconstruction. The acquired multi-frame underwater image sequence is used as input data, denoted as... , where t represents the time frame number.

[0070] The input images are processed via a unified end-to-end deep neural network. This network architecture comprises two core functional sub-modules: a pose estimation module, used to estimate the six-DOF camera transform parameters between adjacent image frames; and a 3D reconstruction module, which combines the estimated pose with image content to generate dense point clouds or voxel representations. The network employs a joint optimization mechanism to ensure that the high dependence of the reconstruction task on the pose estimation results is collaboratively optimized during end-to-end training, thereby significantly improving overall accuracy and stability.

[0071] Step 2: As Figure 3 As shown, the implementation of the camera pose estimation module is as follows:

[0072] To achieve a 3D reconstruction workflow without the need for external SfM tools, an end-to-end camera pose estimation network based on Transformer was designed to directly regress the camera pose from multiple frames of underwater images. Its core structure, input / output definitions, and training mechanism are as follows:

[0073] (1) Input and output definition: Input image sequence, T consecutive frames of images, size (T,3,H,W), resolution H=384, W=512; finally output the global pose corresponding to each frame of image, for a total of T poses.

[0074] (2) Network structure:

[0075] a. Image feature extraction module: ResNet-18 with the last fully connected layer removed is used as the encoder to extract multi-scale deep visual features of the input image. The number of output feature channels is 128, and the feature map size corresponds to the spatial size after downsampling (e.g., 48×64).

[0076] b. Image Patch Encoding + Transformer Encoder: Each image is divided into 16×16 image blocks. Each image block is linearly transformed into a fixed-dimensional vector (e.g., dimension 128), and then positional encoding is added to form a sequence token. Assuming each image size is H×W, each frame is encoded as H / 16×H / 16 patch tokens. The token sequences of multiple image frames are concatenated in chronological order to form the input sequence of the entire Transformer encoder.

[0077] c. Cross-frame feature fusion and spatiotemporal modeling: The input sequence passes through a multi-layer (four layers in this embodiment) standard Transformer encoder, with each layer containing multi-head self-attention, a multi-layer feedforward network, residual connections, and layer normalization. The Transformer module implicitly models the spatiotemporal dependencies and structural correspondences between multiple frames through a self-attention mechanism, achieving cross-frame feature fusion. This mechanism effectively captures geometric and content matching under viewpoint changes without requiring explicit computation of matching maps or optical flow.

[0078] d. Pose Regression Module: Finally, the token sequence output by the Transformer is reshaped into a frame-wise descriptor for each frame, meaning each frame corresponds to a global semantic vector or token representation. This representation is input into the pose regression module (composed of two MLPs), which predict the relative rotation parameters (quaternions) and translation vectors between each pair of image frames, thereby deriving the global pose sequence relative to the first frame.

[0079] e. Data structure conversion: Export to COLMAP style format (cameras.txt / images.txt) for 3D reconstruction input; if there are no intrinsic parameters, use PINHOLE model parameters uniformly: fx=fy=500, cx=W / 2, cy=H / 2; finally, the pose and image sequence are packaged in order to output a scene representation that can be used by 3D Gaussian Splatting.

[0080] Step S3: As Figure 4 As shown, underwater imaging modeling and optimization:

[0081] To enhance the realism and adaptability of 3D reconstruction results in different underwater environments, a learnable underwater imaging model was introduced to simulate the propagation and attenuation of light in water. This model is deeply integrated with the 3D differentiable rendering module, providing a physically sound imaging basis for image synthesis and effectively improving the system's robustness to factors such as water quality changes, depth attenuation, and color shifts.

[0082] The underwater imaging model includes a Medium Modeling Module (Medium MLP): This module is implemented in the form of a Multilayer Perceptron (MLP), receiving a three-dimensional spatial position (x,y,z) as input and outputting underwater imaging parameters for that position in each color channel, including:

[0083] d. Absorption coefficient Used to describe the energy attenuation of incident light as it propagates in water;

[0084] e. Scattering coefficient Used to model the color overlay and blurring phenomena caused by backscattering;

[0085] f. Background light color : Indicates the color of the water itself or the intensity of the background light.

[0086] The parameters mentioned above are generated by Medium MLP in the form of continuous spatial functions and are embedded in the physics rendering process as a physical field. They can be optimized during training to adapt to the real image characteristics under different water conditions.

[0087] To realistically simulate image degradation processes in underwater environments and improve the adaptability and robustness of the reconstruction system under different water quality conditions, an adaptive underwater imaging model is introduced. This model models the underwater light propagation and imaging process in an end-to-end learnable manner.

[0088] ;

[0089] : The pixel color ultimately observed in the camera image on channel c∈{R,G,B};

[0090] : This is the true reflected color of the 3D Gaussian point corresponding to this pixel, generated by the 3D reconstruction module;

[0091] d(x): represents the depth value corresponding to a 3D point, that is, the distance from that point to the center of the camera;

[0092] Absorption coefficient, used to simulate the exponential decay of light intensity during propagation;

[0093] Scattering coefficient, used to simulate the contribution of background light from water to the superposition of colors;

[0094] : Background light color of the medium (or inherent color of the water body), which reflects the basic offset of the water quality to the image color.

[0095] The above media parameters , and All are generated by a learnable medium modeling module (Medium MLP). This module takes three-dimensional spatial coordinates (x,y,z) as input and outputs the above parameters through a multilayer perceptron structure to model the light propagation characteristics in different aquatic environments.

[0096] The underwater imaging model employs a physically parameterized representation, primarily including imaging parameters such as illumination attenuation coefficient, medium background brightness, and optional scattering factor. These imaging parameters can be estimated through image content statistics or learned end-to-end as optimizable variables during training, and are encapsulated and invoked in a one-to-one correspondence with image frame indices. During 3D rendering, these imaging parameters are integrated with a 3D Gaussian point projection mechanism, achieving differentiable physical rendering of colors through a color modulation function based on medium transmittance, thereby improving the consistency and realism between the rendered image and the actual underwater image. This module significantly enhances the robustness and accuracy of 3D reconstruction under different water quality, depth, and illumination conditions, providing strong support for subsequent end-to-end optimization.

[0097] Step S4: Implementation of 3D reconstruction based on Gaussian rendering:

[0098] After completing image input, pose estimation, and adaptive underwater imaging modeling, the system enters the dense reconstruction stage based on a three-dimensional Gaussian distribution. The core of this stage lies in constructing a physically consistent and differentiable image synthesis mechanism to realize the entire process from sparse point initialization to high-quality point cloud restoration.

[0099] First, initialize a set of Gaussian points in 3D space. Each Gaussian point is represented by learnable parameters, including: 3D position coordinates μ=(x,y,z), covariance matrix Σ, color vector c=(r,g,b), and opacity α. Then, initialize a point cloud. Next, begin the rendering process:

[0100] (1) Differentiable rendering process based on physical modeling: In the rendering stage, the system projects Gaussian points onto the image plane and integrates the medium absorption and scattering parameters obtained in the previous stage (S3) to complete physically consistent rendering based on the light propagation model.

[0101] (2) Joint Loss Function Design: To achieve end-to-end optimization and image reconstruction consistency, this system employs a joint loss function during training, including reconstruction error (L1) and structural similarity (SSIM) loss, to measure the color and structural consistency between the synthesized image and the real image. The L1 loss ensures accurate color restoration, while the SSIM loss helps maintain the stability of image edges and texture structures. The weighted combination of these two loss functions constitutes the final loss function, guiding the system to achieve an optimal balance between 3D structure, image appearance, and physical consistency. The loss function is composed as follows:

[0102] a. Reconstruction Loss (L1 Loss): ;

[0103] in The system renders an image, where I(x) is the real image and N is the total number of pixels.

[0104] b. Structural Similarity Loss (SSIM Loss): To enhance structural fidelity, an SSIM loss term is introduced to measure the similarity of image structures in local regions. SSIM evaluates image consistency by comprehensively considering brightness, contrast, and structural similarity.

[0105] c. Total Loss Function (Weighted Combination): Combining the above two indicators, the final optimization objective is: ,in , The weighting coefficient is usually set to 1. =1.0, =0.1~0.5, to balance reconstruction accuracy and structural fidelity.

[0106] This embodiment employs 3D Gaussian Splatting or volumetric rendering to perform high-quality 3D reconstruction. By combining image content with estimated pose information, it generates dense point clouds, meshes, or voxelized 3D representations, effectively restoring the geometric and textural information of underwater scenes. The system supports real-time visualization and various post-processing operations, such as filtering and noise reduction, texture mapping, and model optimization, improving the model's usability and expressiveness in complex engineering environments. The final output is a complete and detailed 3D model, widely applicable to various underwater mission scenarios such as marine resource exploration, underwater robot navigation, structural health checks, and emergency operation analysis.

[0107] Example 2:

[0108] To verify the effectiveness of the end-to-end underwater 3D reconstruction method based on underwater imaging model proposed in this invention, a set of typical experiments were designed, covering 3D modeling tasks under different water conditions.

[0109] This embodiment uses some publicly available datasets and some datasets taken by ourselves. This example demonstrates a publicly available underwater dataset, which is processed into images with a resolution of 512×384 and then used as input. In the experiment, the system adopts the complete end-to-end network architecture described in Embodiment 1. Without the need for external camera calibration, depth maps, or structured light assistance, the system directly inputs the image sequence and executes the following processes in sequence: using the Transformer pose estimation network to regress the relative camera pose between consecutive image frames and convert it into a global pose sequence; constructing an adaptive underwater imaging module to estimate illumination attenuation, scattering, and background light parameters based on 3D point coordinates; performing physically based differentiable 3D image synthesis in the Gaussian rendering module; and jointly optimizing the pose and Gaussian point attributes in the network through pixel-level L1 loss and SSIM loss to finally output a high-fidelity dense 3D point cloud.

[0110] To further verify the effectiveness of the proposed physical modeling module, Figure 5 The image rendering results in typical underwater scenarios are compared: the left image shows the rendering result without underwater imaging modeling, and the right image shows the rendering result with water optical modeling. It can be observed that by introducing physical modeling, the system can more accurately reproduce the light attenuation and scattering phenomena in underwater images, enhancing the realism and visual consistency of the synthesized images, and fully verifying the effectiveness and advantages of the proposed method in complex aquatic environments.

[0111] Figure 5The image rendering results in a typical underwater scene are shown in the comparison: the left image is the result without considering the underwater imaging model (render), and the right image is the rendering result with physical modeling (with_water). It can be observed that after introducing the model, the system more accurately reproduces the underwater light attenuation and scattering phenomena, enhancing the realism and consistency of the image.

[0112] To verify the effectiveness of this invention, it was compared with some existing methods, and the results are shown in the table below:

[0113] ;

[0114] The table above clearly shows that the method of this invention has significant advantages over existing technologies in several key indicators. Regarding PSNR, this method achieves 26.1, 23.7, and 30.8 dB in the Coral, JAN, and Curasao scenes, respectively, all superior to Seasonplat and CF-GS. The improvement is particularly significant in the complex underwater environment of Curasao, demonstrating higher image fidelity. In terms of SSIM, this method is slightly lower than Seasonplat but significantly higher than CF-GS in the Coral and JAN scenes, while significantly better than both in the Curasao scene, indicating more stable structure preservation and detail restoration capabilities. Furthermore, the LPIPS indicator shows that this invention is significantly lower than the comparative methods in all scenes, especially showing a significant improvement in perceived quality and realism.

[0115] In summary, the method of this invention achieves comprehensive optimization in three dimensions: image fidelity, structural restoration, and visual realism. In particular, it demonstrates stronger robustness and adaptability in complex underwater environments, fully verifying its advanced nature and practical value in the field of underwater 3D reconstruction.

[0116] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An end-to-end underwater 3D reconstruction method based on an underwater imaging model, characterized in that, Includes the following steps: S1: Acquire multiple frames of underwater image sequences to obtain raw image data containing complex underwater environmental features such as light attenuation, scattering blur and color shift; S2: Construct an end-to-end learnable pose estimation network to perform spatiotemporal modeling and feature extraction on the input image sequence and predict the relative six-degree-of-freedom pose between image frames; The pose estimation network structure includes: an input module, an image feature extraction module, a fusion encoding module, a Transformer encoding module, and a pose regression module; the pose estimation network in S2 includes the following modules: (1) Input module: Receives underwater multi-view images as observation input for pose estimation; (2) Image feature extraction module: High-dimensional spatial features of the input image are extracted through a deep visual coding network to obtain a feature map containing spatial structure information; then the feature map is flattened and positional encoding is added to form an image token with spatial position information, which is convenient for subsequent Transformer processing and global context modeling. (3) Fusion coding module: Combines image tokens with learnable camera tokens, the latter serving as an implicit representation of camera pose; this module achieves the fusion of visual information and camera priors, constructs a joint contextual representation, provides the Transformer with rich input semantic and geometric information, and assists in pose inference; (4) Transformer encoding module: It consists of multiple Transformer layers, including multi-head self-attention, multi-head cross-attention, feedforward network, residual connection and layer normalization; This module performs deep modeling on the joint sequence of image token and camera token to realize context-aware estimation of pose; (5) Pose Regression Module: Based on the joint features encoded by Transformer, the relative rotation and translation parameters corresponding to the image are predicted through a fully connected regression network to achieve high-precision camera pose estimation; S2 also includes standardized data transformation, which specifically includes the following steps: (1) Pose transformation and format normalization: The relative pose sequence predicted by the pose estimation network is transformed into the global camera pose of each frame through matrix multiplication, and then the rotation matrix is ​​transformed by quaternion. (2) Camera intrinsic parameter initialization and normalization: If there is no explicit calibration data, the default parameters are used, and the size and intrinsic parameter format are uniform for all images to generate the PINHOLE model parameters corresponding to each image; (3) Image and pose synchronization packaging: Traverse the image sequence and match the image file name order with the predicted pose one by one; (4) Sparse point cloud extraction and color addition: Read the spatial point position information from the generated initial point cloud, extract the color of each point, generate a sparse point set with color, and write it into a standard format as the initial scene structure representation; (5) Unified output standard three-data format: The final output consists of three files: cameras.bin, images.txt and points3D.txt, which constitute a sparse reconstruction input format that meets the input requirements of COLMAP / 3DGS, for subsequent loading and use by the differentiable rendering module based on Gaussian representation; S3: Construct an underwater imaging model to simulate the underwater image degradation process caused by the medium, and embed it into the 3D rendering process to enhance color reproduction and imaging physical consistency; S4: Based on the pose estimation network and underwater imaging model, a dense reconstruction module based on a three-dimensional Gaussian distribution is constructed to perform three-dimensional reconstruction and output a three-dimensional reconstructed image.

2. The end-to-end underwater three-dimensional reconstruction method as described in claim 1, characterized in that, The underwater imaging model includes a medium modeling module (MLP) that models the underwater light propagation and imaging process in an end-to-end learnable manner. ； : The pixel color ultimately observed in the camera image on channel c∈{R,G,B}; : This is the true reflected color of the 3D Gaussian point corresponding to this pixel, generated by the 3D reconstruction module; d(x): represents the depth value corresponding to a 3D point, that is, the distance from that point to the center of the camera; Absorption coefficient, used to simulate the exponential decay of light intensity during propagation; Scattering coefficient, used to simulate the contribution of background light from water to the superposition of colors; : The background light color of the medium reflects the basic offset of water quality to image color; The underwater imaging model takes three-dimensional spatial coordinates (x, y, z) as input and outputs the above parameters through a multilayer perceptron (MLP) to model the light propagation characteristics in different aquatic environments.

3. The end-to-end underwater three-dimensional reconstruction method as described in claim 2, characterized in that, The media modeling module is implemented in the form of a multilayer perceptron (MLP), receiving a three-dimensional spatial position (x, y, z) as input and outputting underwater imaging parameters for that position in each color channel, including: a. Absorption coefficient Used to describe the energy attenuation of incident light as it propagates in water; b. Scattering coefficient Used to model the color overlay and blurring phenomena caused by backscattering; c. Background light color : Indicates the color of the water itself or the intensity of the background light.

4. The end-to-end underwater three-dimensional reconstruction method as described in claim 2, characterized in that, The specific steps of S4 are as follows: S4-1: Initialize Gaussian point set: Based on the number of input images and pose information, initialize a sparse 3D point set; each point is represented by a learnable 3D Gaussian distribution, including spatial location, covariance, color attribute and opacity parameter; S4-2: Constructing an underwater projection mechanism: Using the camera pose and the parameters of the underwater imaging model, project three-dimensional Gaussian points onto the image plane to construct a differentiable projection rendering model; S4-3: Gaussian Differentiable Rendering: Based on the 3D Gaussian differentiable rendering framework, color, transparency and coverage are fused in the image space to generate a synthetic image, and the following joint loss function is calculated; S4-4: Color consistency loss: Minimize the pixel error between the rendered image and the real image; S4-5: Transparency Regularization: Constrains redundant points and improves efficiency; S4-6: Depth Consistency Loss: Constrains the geometry of point clouds to improve the accuracy of 3D reconstruction; S4-7: Layered optimization strategy: adopt a coarse-to-fine optimization process, increase the number of Gaussian points and image resolution in round by round, and supplement it with a redundancy removal mechanism; S4-8: The final output is a high-fidelity dense point cloud with complete geometric structure, texture and boundary details.

5. An end-to-end underwater 3D reconstruction system based on the underwater imaging model of the end-to-end underwater 3D reconstruction method based on the underwater imaging model as described in claim 1, characterized in that, Includes the following modules: Image data acquisition module: This module is used to acquire multiple frames of underwater image sequences and obtain raw image data; Pose estimation network module: This module performs spatiotemporal modeling and feature extraction on the input image sequence and predicts the relative six-degree-of-freedom pose between image frames; Underwater Imaging Model Module: Used to simulate the underwater image degradation process caused by the medium and embedded into the 3D rendering process to enhance color reproduction and imaging physical consistency; 3D Reconstruction Module: Performs 3D reconstruction based on dense reconstruction of a 3D Gaussian distribution and outputs a 3D reconstructed image.

Images