Underwater multi-view three-dimensional reconstruction method

Abstract

The invention provides an underwater multi-view three-dimensional reconstruction method. The underwater multi-view three-dimensional reconstruction method mainly comprises the following steps: S1, acquiring a high-quality multi-view underwater image through a physical perception image enhancement technology PUIE; s2, initializing a neural symbol distance function SDF, and implicitly representing the three-dimensional shape of an object by a neural network; s3, generating a two-dimensional foreground mask by using few-sample multi-view target segmentation, and obtaining three-dimensional geometric information in combination with monocular depth estimation and normal vector prediction to form mixed geometric prior; and S4, in combination with an image enhancement result and mixed geometric priori, through a joint loss function, training a neural network optimization SDF, and completing accurate three-dimensional reconstruction. According to the underwater three-dimensional reconstruction method, a more efficient and more accurate underwater three-dimensional reconstruction solution is provided by introducing mixed geometric prior and optimizing the reconstruction process of the neural symbol distance function SDF.

Description

Technical Field

[0001] The invention relates to three-dimensional reconstruction technology, and in particular to an underwater multi-view three-dimensional reconstruction method. Background Art

[0002] In underwater environments, due to the unique characteristics of light absorption and scattering, obtaining high-quality images and accurate 3D reconstruction is a very challenging task. Existing 3D reconstruction technologies mainly rely on multi-sensor fusion and deep learning methods, which have their own advantages and disadvantages, but still have many shortcomings.

[0003] Multi-sensor fusion technology usually combines sonar and optical sensors to obtain geometric information of underwater objects. For example, sonar sensors can penetrate turbid water, but their resolution is low and it is difficult to provide fine geometric details. Optical sensors (such as underwater cameras) can capture high-resolution images, but are limited by the optical properties of the underwater environment, and the image quality is easily affected by light absorption and scattering. In order to overcome their respective shortcomings, researchers have tried to fuse the data of sonar and optical sensors to improve the overall reconstruction accuracy.

[0004] However, the multi-sensor fusion method has the following shortcomings:

[0005] 1. High cost and complexity: The equipment cost of multi-sensor systems is high, and the system integration and calibration are complex, which increases the difficulty of deployment and maintenance.

[0006] 2. Difficulties in data synchronization and fusion: The working principles of sonar and optical sensors are different, which makes it difficult to synchronize the time and space of data, and affects the fusion accuracy.

[0007] 3. Strong dependence on the environment: The multi-sensor system is highly dependent on the underwater environment. Especially in turbid water and insufficient light, the data quality will be significantly reduced.

[0008] In recent years, deep learning technology has made significant progress in the fields of computer vision and image processing. In underwater image processing, deep learning models such as convolutional neural networks (CNNs) have been widely used in tasks such as image enhancement, dehazing, and object detection. For example, some studies have used generative adversarial networks (GANs) to enhance underwater images and improve image quality. However, in the field of 3D reconstruction, especially in underwater environments, deep learning technology still faces challenges.

[0009] Specifically, existing deep learning methods have the following shortcomings:

[0010] 1. Limited image enhancement effect: Although deep learning models can improve the quality of underwater images to a certain extent, the enhancement effect is limited in the case of severe light absorption and scattering.

[0011] 2. Insufficient reconstruction accuracy: When deep learning-based 3D reconstruction methods, such as Neural Radiance Field (NeRF), are applied in underwater environments, the reconstruction results are often not accurate enough and the details are not fully restored due to the lack of effective geometric priors and illumination models.

[0012] 3. Large demand for training data: Deep learning models usually require a large amount of labeled data for training. However, in underwater environments, it is costly and difficult to obtain and label high-quality training data.

[0013] A typical underwater 3D reconstruction technology is an implicit representation method based on neural radiance fields (NeRF). NeRF uses multi-view images to train a neural network to implicitly represent the voxel color and density of the scene. However, when NeRF is directly applied to underwater scenes, the effect is not ideal. The NeRF method proposed by Mildenhall et al. performs well in aerial environments, but in underwater environments, due to optical properties and lack of geometric priors, the reconstruction results are blurred and inaccurate. UWSLAM method of Bi et al.: The UWSLAM method proposed by Bi et al. combines underwater image enhancement and 3D reconstruction technology, but still relies on multi-sensor data, the system is complex, and the reconstruction accuracy is limited.

[0014] In summary, the existing multi-sensor fusion technology and deep learning methods still have many shortcomings in underwater 3D reconstruction, especially in terms of equipment cost, system complexity, data fusion, environmental dependence, image enhancement effect and reconstruction accuracy.

[0015] It should be noted that the information disclosed in the above background technology section is only used for understanding the background of the present application, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the invention

[0016] The main purpose of the present invention is to solve the problems existing in the above-mentioned background technology and to provide an underwater multi-view 3D reconstruction method.

[0017] To achieve the above object, the present invention adopts the following technical solutions:

[0018] In a first aspect of the present invention, an underwater multi-view 3D reconstruction method comprises the following steps:

[0019] S1. Acquire multi-view underwater images and use the underwater image enhancement (PUIE) method based on physical perception to enhance the images.

[0020] S2, initializing a neural signed distance function SDF, wherein the initialized SDF is represented by a neural network, and the neural network parameterizes an implicit three-dimensional field for describing the geometric shape of the object;

[0021] S3. Based on image enhancement, a foreground mask is generated through a few-sample multi-view target segmentation strategy as a 2D geometric prior, and 3D geometric information is obtained through monocular depth estimation and normal vector prediction as a 3D geometric prior.

[0022] S4. Combining the image enhancement results and geometric prior information, a neural signed distance function (SDF) is trained. A joint loss function is constructed using a mixed geometric prior of two-dimensional foreground mask and three-dimensional depth and normal vector information. The enhanced multi-view images and their corresponding geometric prior information are used to train the neural network to minimize the loss function between the reconstruction error and the geometric prior constraint. The trained neural network is used for three-dimensional reconstruction.

[0023] In a second aspect of the present invention, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the underwater multi-view three-dimensional reconstruction method is implemented.

[0024] In a third aspect of the present invention, a computer program product includes a computer program, and when the computer program is executed by a processor, the underwater multi-view three-dimensional reconstruction method is implemented.

[0025] The present invention has the following beneficial effects:

[0026] The present invention provides a more efficient and accurate underwater 3D reconstruction solution by introducing a hybrid geometric prior and optimizing the neural signed distance function (SDF) reconstruction process.

[0027] The underwater multi-view 3D reconstruction method of the present invention significantly improves the quality and efficiency of 3D reconstruction of underwater environment through a series of innovative technical steps. First, the underwater image is enhanced by using the physical perception-based image enhancement method PUIE, which effectively improves the clarity and contrast of the image and solves the problem of image color distortion and blur caused by complex underwater lighting conditions. Secondly, a high-quality foreground mask is generated as a two-dimensional geometric prior through a few-sample multi-view target segmentation strategy, and the three-dimensional geometric information obtained by combining monocular depth estimation and normal vector prediction provides rich geometric constraints for the training of neural networks. In addition, the present invention introduces a hybrid geometric prior, and optimizes the reconstruction process of the neural signed distance function SDF through a joint loss function, which significantly improves the reconstruction accuracy and detail recovery. Compared with traditional multi-sensor fusion technology and deep learning methods, the present invention reduces equipment cost and system complexity, reduces dependence on a large amount of labeled data, and improves the degree of automation, providing an efficient and accurate 3D reconstruction solution for fields such as marine scientific research, underwater engineering maintenance, underwater archaeology, and autonomous underwater vehicles.

[0028] Other beneficial effects of the embodiments of the present invention will be further described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Schematic diagram of the framework of the underwater multi-view 3D reconstruction method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope and application of the present invention.

[0031] The present invention optimizes the reconstruction process of the neural signed distance function (SDF) by combining mixed geometric prior information. Physically-aware underwater image enhancement (PUIE) method: Based on the physical model, the illumination attenuation and scattering parameters are estimated to perform image enhancement. SDF reconstruction technology with joint geometric prior: Combines two-dimensional foreground mask and three-dimensional geometric information to construct a joint loss function and optimize neural network training. Few-sample multi-view target segmentation strategy: Based on a small amount of annotation, multi-view target automatic segmentation is achieved through data enhancement and transfer learning.

[0032] The method of the embodiment of the present invention mainly includes: Image acquisition: using a multi-view optical camera to collect underwater environment images. Image enhancement: enhancing the image through the PUIE method to improve clarity and contrast. Specifically including: Light distribution estimation: using a physical model to estimate the light attenuation and scattering parameters in the image. Color distortion correction: correcting the color distortion in the image according to the estimated parameters. Geometric prior acquisition: Two-dimensional foreground mask: initially annotate a small number of images, train the segmentation model, and automatically segment the remaining images. Three-dimensional geometric information: obtain depth and normal vector information through monocular depth estimation and normal vector prediction. SDF initialization and training: initialize the SDF model and parameterize it into a neural network. Combined with geometric prior information, a joint loss function is constructed to optimize neural network training to generate high-precision three-dimensional reconstruction results.

[0033] See also Figure 1 The embodiment of the present invention provides an underwater multi-view 3D reconstruction method, comprising the following steps:

[0034] S1. Image acquisition and enhancement: Acquire multi-view underwater images and use the underwater image enhancement (PUIE) method based on physical perception to enhance the images, providing high-quality image data for subsequent geometric prior information acquisition and 3D reconstruction;

[0035] S2. Initialize the neural SDF: Initialize the neural signed distance function SDF by pre-training or randomly, and the initialized SDF is represented by a neural network, and the neural network parameterizes an implicit three-dimensional field for describing the geometric shape of the object;

[0036] S3, geometric prior acquisition: Based on image enhancement, a foreground mask is generated through a few-sample multi-view target segmentation strategy as a 2D geometric prior, and 3D geometric information is obtained through monocular depth estimation and normal vector prediction as a 3D geometric prior;

[0037] S4. Neural SDF reconstruction: Use a neural network combined with image enhancement results and geometric prior information to perform neural signed distance function (SDF) and training; use the trained neural network model to perform three-dimensional reconstruction; among them, use a mixed geometric prior of two-dimensional foreground mask and three-dimensional depth and normal vector information to construct a joint loss function, use the enhanced multi-view images and their corresponding geometric prior information to train the neural network to minimize the loss function between the reconstruction error and the geometric prior constraints.

[0038] In a preferred embodiment, in step S1, the use of the PUIE method for image enhancement specifically includes: distribution estimation: using a physical model to model the underwater illumination and imaging process, estimating the illumination distribution and color distortion in the image; calculating the illumination attenuation and scattering parameters by analyzing the color histogram and illumination distribution of the image. Consensus process: correcting and enhancing the image based on the estimated illumination and color distortion parameters; optimizing the image enhancement effect through the consistency of information between multi-view images to ensure that the enhanced image has higher clarity and contrast.

[0039] In a preferred embodiment, in step S3, the few-sample multi-view target segmentation strategy specifically includes: initial labeling: manually labeling on images of a small number of selected viewpoints to generate an initial target mask, and the labeling is used as the few-sample data for training the segmentation model; segmentation model training: using the segmentation model to train on the few-sample data, and improving the segmentation ability of the model through data enhancement and transfer learning techniques; automatic segmentation: automatically segmenting the remaining viewpoint images to generate a foreground mask, and the mask is used as a two-dimensional geometric prior to constrain the training process of the neural SDF.

[0040] In a preferred embodiment, in step S4, the multi-view images used for SDF modeling are preprocessed, including operations such as image alignment, denoising and normalization, to ensure the consistency and quality of the input data. Furthermore, in step S4, the foreground mask generated by the few-sample multi-view object segmentation strategy is used as a two-dimensional geometric prior to constrain the training process of the neural network; the additional three-dimensional geometric information obtained by monocular depth estimation and normal vector prediction is used to further optimize the training of the SDF.

[0041] See also Figure 1In a preferred embodiment, step S4 specifically includes: extracting a depth map and a normal map from the enhanced multi-view RGB image, wherein these feature maps are used to capture the geometric properties of the object surface; using a few-sample multi-view target segmentation strategy to generate a foreground mask to distinguish the object surface from other background elements; combining the two-dimensional foreground mask and the three-dimensional geometric information obtained by monocular depth estimation and normal vector prediction to form a complete geometric prior; learning the SDF representation of the object from the extracted features and the geometric prior through a multi-layer perceptron MLP network; wherein the definition of the loss function combines the constraints of the reconstruction error and the geometric prior and is used as the optimization target in the training process; during the training process, the MLP network is trained through light sampling and volume rendering technology to minimize the loss function, thereby learning an accurate SDF representation; and the network parameters are adjusted through iterative optimization until a high-precision three-dimensional reconstruction effect is achieved.

[0042] In some embodiments, a multispectral camera is used to collect data, and multispectral information is combined to perform image enhancement to improve image quality under different spectra.

[0043] In some embodiments, multi-sensor fusion can be performed: sonar or lidar data can be combined with optical images for three-dimensional reconstruction to further improve reconstruction accuracy.

[0044] In some embodiments, the real-time image enhancement and three-dimensional reconstruction system obtained by using the present invention is applied to an autonomous underwater vehicle (AUV) or a remotely operated vehicle (ROV) to achieve real-time environmental perception and navigation.

[0045] In some embodiments, the present invention can be used for augmented reality (AR) applications: applying the reconstructed three-dimensional model to the AR system to assist underwater engineering or archaeological research and improve work efficiency and accuracy.

[0046] The significant advantages of the present invention are:

[0047] Significantly improved image quality: The PUIE method is based on a physical model and can more accurately correct color distortion and improve image clarity. High reconstruction accuracy: The SDF reconstruction technology combined with geometric priors significantly improves the geometric details and overall accuracy of 3D reconstruction. Efficient automation: The few-sample, multi-view target segmentation strategy reduces the need for manual labeling and improves the efficiency of foreground mask acquisition. Low system cost: Relying on a single optical sensor reduces equipment cost and system complexity. Advantages compared to existing solutions: Multi-sensor fusion technology has high equipment cost and complex system; the present invention reduces cost and system complexity. Deep learning methods have a large demand for a large amount of labeled data; the present invention reduces data requirements through a few-sample strategy.

[0048] The specific embodiments of the present invention are further described below.

[0049] This paper proposes a neural signed distance function (SDF) reconstruction method based on hybrid geometric priors, which is particularly suitable for 3D reconstruction of underwater environments. This paper proposes a framework called UW-SDF, which optimizes the reconstruction process of the neural signed distance function (SDF) by introducing hybrid geometric priors, thereby significantly improving the quality and efficiency of underwater 3D reconstruction.

[0050] The method specifically includes:

[0051] 1. Physically Perceived Underwater Image Enhancement (PUIE)

[0052] Illumination distribution estimation: Use physical models to model the underwater illumination and imaging process and estimate the illumination attenuation and scattering parameters in the image.

[0053] Color distortion correction: Correct the color distortion in the image based on the estimated lighting and color distortion parameters to improve the color reproduction of the image.

[0054] Multi-view information consistency optimization: By analyzing the information consistency between multi-view images, the image enhancement effect is optimized to ensure that the enhanced image has higher clarity and contrast.

[0055] 2. Neural Signed Distance Function (SDF) Reconstruction

[0056] SDF initialization method: The initial SDF is represented by a neural network that parameterizes an implicit three-dimensional field to describe the geometry of the object.

[0057] Loss function design with joint geometric prior: A joint loss function is constructed by combining 2D foreground mask and 3D geometric information (depth and normal vector) to optimize the training process of the neural network.

[0058] As a specific example, the joint loss function includes the following RGB reconstruction loss, Eikonal loss, 2D mask loss, and 3D geometric information loss:

[0059]

[0060] RGB reconstruction loss:

[0061]

[0062] The SDF value can be standardized by the Eikonal term, and the Eikonal loss function is as follows:

[0063]

[0064] Binary cross-entropy loss (BCELoss) is used to supervise the predicted two-dimensional foreground mask. The two-dimensional mask loss function is as follows:

[0065]

[0066] When dealing with shape radiation blur, the network is supervised using depth prior and normal prior. By calculating the depth and surface normal, and supervising them using L1loss and angleL1loss respectively, the 3D geometric information loss function is as follows:

[0067]

[0068] Among them, x is a batch of points uniformly sampled in three-dimensional space and near the surface, C represents the RGB image, M represents the mask, D represents the depth map, N represents the normal map, and the symbols above the letters represent the predicted value and - represents the prior.

[0069] Neural network training strategy: Use incremental learning and transfer learning techniques to improve the training efficiency and reconstruction accuracy of neural networks.

[0070] 3. Few-sample multi-view target segmentation strategy

[0071] Initial annotation method: Manual annotation is performed on a small number of view images to generate an initial target mask.

[0072] Data augmentation and transfer learning: Use data augmentation techniques to expand the training dataset and use transfer learning to improve the generalization ability of the segmentation model.

[0073] Automatic segmentation technology: Use the trained segmentation model to automatically segment multi-view images and generate high-quality foreground masks.

[0074] 4. Utilization of mixed geometric priors

[0075] Acquisition and application of two-dimensional geometric prior: The foreground mask generated by the few-sample multi-view target segmentation strategy is used as a two-dimensional geometric prior to constrain the training process of the neural network.

[0076] Acquisition and application of 3D geometric priors: Through monocular depth estimation and normal vector prediction, Additional 3D geometric information further optimizes the training of SDF.

[0077] Geometric prior joint constraint method: Combine the two-dimensional foreground mask and three-dimensional geometric information to jointly constrain the training process of the neural network and improve reconstruction accuracy and detail recovery.

[0078] The underwater 3D reconstruction system architecture of the embodiment of the present invention includes: an image acquisition and enhancement module: including a multi-view camera system and an image enhancement technology based on the PUIE method. A geometric prior acquisition module: including a few-sample multi-view target segmentation strategy and a 3D geometric information acquisition technology. A neural SDF reconstruction module: including SDF initialization, loss function design of joint geometric priors, and a neural network training strategy. A reconstruction result display module: used to display and analyze the high-precision 3D reconstruction results finally generated.

[0079] The present invention can achieve high-quality three-dimensional reconstruction in underwater environments, providing an efficient and accurate solution and providing technical support for multiple fields such as marine scientific research, underwater engineering maintenance, underwater archaeology and autonomous underwater vehicles.

[0080] In some specific embodiments, the following main steps are included:

[0081] First, the captured multi-view underwater images are enhanced to improve the image quality. We adopt a physical perception-based underwater image enhancement (PUIE) method, which is divided into two parts: distribution estimation and consensus process: Distribution estimation: The physical model is used to model the underwater illumination and imaging process, and the illumination distribution and color distortion in the image are estimated. By analyzing the color histogram and illumination distribution of the image, the illumination attenuation and scattering parameters are calculated. Consensus process: Based on the estimated illumination and color distortion parameters, the image is corrected and enhanced. Through the consistency of information between multi-view images, the image enhancement effect is optimized to ensure that the enhanced image has higher clarity and contrast.

[0082] Next, based on the enhanced image, the neural signed distance function (SDF) is trained to represent the surface of the object using the concept of neural radiance field (NeRF). The specific steps are as follows: SDF initialization: The initial SDF is represented by a neural network that parameterizes an implicit three-dimensional field to describe the geometry of the object. The initial network parameters can be pre-trained or randomly initialized. Image preprocessing: Preprocess the multi-view images used for SDF neural network modeling, including operations such as image alignment, denoising, and normalization to ensure the consistency and quality of the input data. Introduction of geometric prior: During the SDF training process, two-dimensional and three-dimensional geometric prior information are combined for optimization; Obtain a two-dimensional foreground mask: The foreground mask generated by the few-sample multi-view target segmentation strategy is used as a two-dimensional geometric prior to constrain the training process of the neural network; Obtain three-dimensional depth and normal vector: Obtain additional three-dimensional geometric information through monocular depth estimation and normal vector prediction to further optimize the SDF. of Training. Neural network training: using enhanced multi-view images and their corresponding geometric prior information The goal is to minimize the loss function between the reconstruction error and the geometric prior constraints, thereby obtaining a high-precision 3D reconstruction result.

[0083] In order to efficiently obtain the foreground mask, the present invention proposes a few-sample multi-view target segmentation strategy, and the specific steps are as follows: Initial annotation: Manually annotate on a small number of view images to generate an initial target mask. These annotations are used as few-sample data to train the segmentation model. Segmentation model training: Use a general segmentation model (such as SAM, SegmentAnythingModel) to train on few-sample data. Improve the segmentation ability of the model through data enhancement and transfer learning techniques. Automatic segmentation: Automatically segment the remaining multi-view images to generate a foreground mask. These masks are used as two-dimensional geometric priors to constrain the training process of the neural SDF.

[0084] During the training process of the neural SDF, hybrid geometric priors are used to make full use of 2D and 3D geometric prior information to improve reconstruction accuracy and detail recovery: 2D geometric prior: Use foreground masks as constraints to ensure that the network can correctly identify and segment the foreground and background areas of objects during reconstruction to avoid background noise interference. 3D geometric prior: Combined with monocular depth estimation and normal vector prediction, it provides additional 3D geometric information to help the network more accurately restore the geometric details and surface texture of objects.

[0085] Figure 1 This is a technical flow chart of the UW-SDF framework of an embodiment of the present invention, which shows the workflow and main technical links of the entire system as follows:

[0086] Image acquisition and enhancement: Underwater images are collected by a multi-view optical camera, and the PUIE method is used for image enhancement.

[0087] Geometric prior acquisition: A few-sample multi-view target segmentation strategy is used to generate a foreground mask, and three-dimensional geometric prior is acquired through monocular depth estimation and normal vector prediction.

[0088] SDF initialization and training: Initialize the neural signed distance function (SDF), combine the geometric prior information to train the neural network, and finally generate high-precision 3D reconstruction results.

[0089] The implementation of the PUIE method includes: Distribution estimation: using optical models and image statistical characteristics to estimate light attenuation and scattering parameters. Consensus process: based on the consistency of information between multi-view images, the image enhancement effect is optimized.

[0090] The training strategy of Neural SDF includes: combining foreground mask and 3D geometric prior to construct a joint loss function to constrain the training process of the neural network. Using incremental learning and transfer learning techniques to improve the training efficiency and reconstruction accuracy of the neural network. Few-sample multi-view target segmentation strategy: Initial annotation and data enhancement: training on a small amount of annotated data, using data enhancement technology to expand the training data set. Automatic segmentation: using the trained segmentation model, automatically segment the multi-view image to generate a high-quality foreground mask.

[0091] Through the above technical solutions, the present invention has achieved remarkable results in underwater three-dimensional reconstruction, especially in terms of image quality improvement, reconstruction accuracy and efficiency, and has obvious advantages.

[0092] Compared with the prior art, the present invention significantly improves the quality and efficiency of underwater 3D reconstruction by introducing hybrid geometric priors and optimizing the neural signed distance function (SDF) reconstruction process. The specific advantages are as follows:

[0093] 1. Improve image quality

[0094] Physical perception based underwater image enhancement (PUIE) method:

[0095] Distribution Estimation: Estimate light attenuation and scattering parameters through physical models to accurately correct color distortion in images.

[0096] Consensus process: Utilize the information consistency between multi-view images to optimize the image enhancement effect.

[0097] This method significantly improves the clarity and contrast of underwater images, allowing the subsequent three-dimensional reconstruction process to be based on higher quality image data.

[0098] 2. High-precision 3D reconstruction

[0099] Neural Signed Distance Function (SDF) reconstruction technique:

[0100] Combining geometric priors: By introducing 2D foreground masks and 3D geometric prior information (such as depth and normal vectors), the training process of SDF is optimized.

[0101] Neural network training: Use the joint loss function, combined with incremental learning and transfer learning techniques to improve network training efficiency and reconstruction accuracy.

[0102] Compared with traditional multi-sensor fusion technology and existing deep learning methods, the present invention achieves higher geometric detail recovery and overall reconstruction accuracy in underwater environments.

[0103] 3. Efficient target segmentation

[0104] Few-sample multi-view target segmentation strategy:

[0105] Initial annotation and data augmentation: Manual annotation is performed on a small number of images from different viewpoints, combined with data augmentation techniques to expand the training dataset.

[0106] Automatic Segmentation: The Generic Segmentation Model (SAM) is trained to automatically segment multi-view images and generate high-quality foreground masks.

[0107] This strategy greatly reduces the need for manual annotation, improves the efficiency of foreground mask acquisition, and provides reliable geometric priors for the training of neural SDF.

[0108] Main technical advantages:

[0109] 1. Significantly improved image quality: The PUIE method significantly improves the clarity and contrast of underwater images, providing high-quality data input for 3D reconstruction.

[0110] 2. Excellent reconstruction accuracy and detail restoration: Combined with the SDF reconstruction technology based on mixed geometric priors, Higher geometric detail recovery and overall reconstruction accuracy are achieved.

[0111] 3. Improved efficiency: The few-sample, multi-view target segmentation strategy reduces the need for manual labeling and improves the efficiency of foreground mask acquisition, thereby accelerating the overall 3D reconstruction process.

[0112] Comparison with existing technologies: Existing multi-sensor fusion technology has high equipment cost, complex system, and difficult data fusion. In contrast, the present invention relies on a single optical sensor, reducing costs and system complexity. Existing deep learning methods have limited image enhancement effects, insufficient reconstruction accuracy, and large training data requirements. The present invention combines geometric priors to optimize neural network training, significantly improves reconstruction accuracy, and reduces training data requirements.

[0113] The application fields of the present invention include: Underwater detection: Applicable to the three-dimensional reconstruction of underwater objects and environments in the fields of marine scientific research, ecological environment monitoring, etc. Underwater engineering: Applicable to the three-dimensional modeling and analysis of engineering projects such as underwater infrastructure detection, maintenance and repair. Underwater archaeology: Provide high-precision technical support for the three-dimensional reconstruction of underwater relics and cultural relics, and assist in archaeological research and protection work. Autonomous underwater vehicles (AUVs) and remotely operated vehicles (ROVs): Realize high-precision environmental perception and target reconstruction in autonomous navigation, target recognition and operation tasks.

[0114] In summary, the present invention has achieved remarkable technical effects in underwater 3D reconstruction, especially in terms of image quality improvement, reconstruction accuracy and efficiency. By introducing hybrid geometric priors and optimizing the SDF reconstruction process, the present invention provides an efficient and accurate solution for 3D reconstruction of underwater environments.

[0115] In order to demonstrate the specific application effects of the present invention, several examples are provided below, which detail the application and technical advantages of the present invention in different scenarios.

[0116] Example 1: 3D reconstruction of underwater coral reefs

[0117] Background: In marine ecological research, accurate reconstruction of the three-dimensional structure of underwater coral reefs is important for assessing their health status and ecological changes.

[0118] step:

[0119] 1. Image acquisition: Use a multi-view optical camera to photograph coral reefs at different angles and distances to obtain multiple sets of underwater images.

[0120] 2. Image enhancement: Use the physical perception-based underwater image enhancement (PUIE) method to enhance the collected images and improve the image clarity and contrast.

[0121] 3. Geometric prior acquisition:

[0122] 2D foreground mask: Manually annotate the foreground regions of the coral reef on a few viewpoints, train a segmentation model, and automatically segment the remaining images.

[0123] 3D geometry: Obtain the 3D geometry of coral reefs through monocular depth estimation and normal vector prediction Geometry information.

[0124] 4. SDF initialization and training: Initialize the neural signed distance function (SDF), combine geometric prior information, train the neural network, optimize the loss function, and generate high-precision 3D reconstruction results of the coral reef.

[0125] Effect: The present invention achieves fine structural restoration and high-precision reconstruction of geometric details in the three-dimensional reconstruction of coral reefs, providing an accurate three-dimensional model that helps ecologists conduct detailed research and evaluation.

[0126] Example 2: Underwater pipeline inspection and maintenance

[0127] Background: Inspection and maintenance of underwater pipelines require high-precision 3D models in order to identify potential damage and perform precise repair work.

[0128] step:

[0129] 1. Image acquisition: Use the multi-view camera system carried by the underwater robot to capture multiple sets of images of the underwater pipeline.

[0130] 2. Image enhancement: The collected images are enhanced through the PUIE method to eliminate the effects of light attenuation and scattering and improve image quality.

[0131] 3. Geometric prior acquisition:

[0132] 2D foreground mask: Preliminarily annotate pipeline regions in a small number of images, train a segmentation model, and automatically segment the remaining images.

[0133] 3D geometric information: Use depth estimation technology to obtain the 3D geometric information of the pipeline, including depth and normal vector.

[0134] 4. SDF initialization and training: Initialize and train the SDF model, optimize it with geometric priors, and generate a high-precision 3D model of the pipeline.

[0135] Effect: The present invention provides a high-precision three-dimensional model in underwater pipeline inspection, which can clearly display the structure and details of the pipeline, help engineers accurately locate and repair potential damage, and improve the efficiency and accuracy of maintenance work.

[0136] Example 3: 3D reconstruction of underwater archaeological sites

[0137] Background: In underwater archaeological research, accurate reconstruction of the three-dimensional structure of the site is of great significance for protection and research.

[0138] step:

[0139] 1. Image acquisition: Use a multi-view camera system to capture images of underwater sites from different angles.

[0140] 2. Image enhancement: Apply the PUIE method to enhance image quality, eliminate the impact of the underwater environment on the image, and improve clarity.

[0141] 3. Geometric prior acquisition:

[0142] 2D foreground masks: Manually annotate the heritage areas in a small number of images, train a segmentation model, and automatically generate foreground masks for the remaining images.

[0143] Three-dimensional geometric information: Obtain the three-dimensional geometric information of the site through monocular depth estimation technology.

[0144] 4. SDF initialization and training: Initialize and train the SDF model, combine the geometric prior to optimize the reconstruction process, and generate a high-precision three-dimensional model of the site.

[0145] Effect: The present invention performs well in the three-dimensional reconstruction of underwater archaeological sites, providing a detailed three-dimensional model that shows the complete structure and details of the site, helping archaeologists conduct in-depth research and protection work.

[0146] Example 4: Autonomous Underwater Vehicle (AUV) Environmental Perception

[0147] Background: Autonomous underwater vehicles (AUVs) require high-precision environment perception capabilities for autonomous navigation and target recognition.

[0148] step:

[0149] 1. Image acquisition: The AUV is equipped with a multi-view camera system to collect multiple sets of images of the underwater environment during autonomous cruising.

[0150] 2. Image enhancement: Use the PUIE method to enhance image quality and improve the clarity and contrast of environmental images.

[0151] 3. Geometric prior acquisition:

[0152] 2D foreground mask: A small number of key target areas from the AUV’s perspective are annotated, and the segmentation model is trained to automatically segment the rest of the image.

[0153] 3D geometric information: Use depth estimation technology to obtain the 3D geometric information of the environment.

[0154] 4. SDF initialization and training: Initialize and train the SDF model, optimize it with geometric priors, and generate a high-precision 3D model of the environment.

[0155] Effect: The present invention provides a high-precision three-dimensional environmental model in AUV environmental perception, improves the autonomous navigation and target recognition capabilities of AUV, and ensures efficient operation in complex underwater environments.

[0156] Through the above examples, the present invention demonstrates its superior technical effects in different scenarios, including:

[0157] 1. High-quality image enhancement: Through the PUIE method, the clarity and contrast of underwater images are significantly improved.

[0158] 2. High-precision 3D reconstruction: Combined with hybrid geometric priors, SDF model training is optimized to generate high-precision 3D reconstruction results.

[0159] 3. Efficient target segmentation: The few-sample multi-view target segmentation strategy reduces the need for manual labeling and improves the efficiency of foreground mask acquisition.

[0160] 4. Wide range of application fields: It is suitable for marine ecological research (such as 3D reconstruction and monitoring of marine life and ecological environment), underwater engineering maintenance (such as 3D modeling and detection of underwater structures such as pipelines and platforms), underwater archaeology (such as 3D reconstruction and protection of underwater sites and cultural relics) and autonomous underwater vehicles (such as environmental perception, navigation and target recognition), showing its wide application potential and technical advantages.

[0161] These embodiments fully demonstrate the remarkable technical effects of the present invention in underwater three-dimensional reconstruction, and provide effective technical support for the exploration and application of underwater environments.

[0162] An embodiment of the present invention further provides a storage medium for storing a computer program, which at least performs the above method when executed.

[0163] An embodiment of the present invention further provides a control device, comprising a processor and a storage medium for storing a computer program; wherein the processor is configured to execute at least the method described above when executing the computer program.

[0164] An embodiment of the present invention further provides a processor, wherein the processor executes a computer program and at least executes the method described above.

[0165] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a ferromagnetic random access memory, a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The storage medium described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0166] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are only exemplary. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the components shown or discussed are not mutually exclusive. The coupling, or direct coupling, or communication connection between them can be through some interface, device or unit The indirect coupling or communication connection may be electrical, mechanical or other.

[0167] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0168] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0169] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, and other media that can store program codes.

[0170] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0171] The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0172] The features disclosed in several product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0173] The features disclosed in several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0174] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For technicians in the field, In addition to the concept of the present invention, several other Any substitution or obvious modification with the same performance or use shall be deemed to fall within the protection scope of the present invention.

Claims

1. An underwater multi-view 3D reconstruction method, characterized in that: The following steps are involved: S1. Acquire multi-view underwater images and use the underwater image enhancement (PUIE) method based on physical perception to enhance the images. S2, initializing a neural signed distance function SDF, wherein the initialized SDF is represented by a neural network, and the neural network parameterizes an implicit three-dimensional field for describing the geometric shape of the object; S3. Based on image enhancement, a foreground mask is generated through a few-sample multi-view target segmentation strategy as a 2D geometric prior, and 3D geometric information is obtained through monocular depth estimation and normal vector prediction as a 3D geometric prior. S4. Combining the image enhancement results and geometric prior information, a neural signed distance function (SDF) is trained. A joint loss function is constructed using a hybrid geometric prior of a two-dimensional foreground mask and three-dimensional depth and normal vector information. The enhanced multi-view images and their corresponding geometric prior information are used to train the neural network to minimize the loss function between the reconstruction error and the geometric prior constraint. The trained neural network is used for three-dimensional reconstruction.

2. The underwater multi-view 3D reconstruction method according to claim 1, characterized in that: In step S1, using the PUIE method to perform image enhancement specifically includes: Distribution estimation: Use physical models to model underwater illumination and imaging processes, estimate illumination distribution and color distortion in images; calculate illumination attenuation and scattering parameters by analyzing the color histogram and illumination distribution of images; Consensus process: Correct and enhance the image based on the estimated illumination and color distortion parameters; optimize the image enhancement effect through the consistency of information between multi-view images.

3. The underwater multi-view 3D reconstruction method according to any one of claims 1 to 2, characterized in that: In step S2, the initial network parameters of the neural network are initialized by pre-training or random initialization.

4. The underwater multi-view 3D reconstruction method according to any one of claims 1 to 3, characterized in that: In step S3, the few-sample multi-view target segmentation strategy specifically includes: Manually annotate images of a small number of selected viewpoints to generate initial target masks, and the annotations are used as few-sample data to train the segmentation model; Use the segmentation model to train on a small number of sample data, and improve the segmentation ability of the model through data enhancement and transfer learning technology; The remaining view images are automatically segmented to generate a foreground mask, which is used as a 2D geometric prior to constrain the training process of the neural SDF.

5. The underwater multi-view 3D reconstruction method according to any one of claims 1 to 4, characterized in that: In step S4, the multi-view images used for SDF modeling are first preprocessed, including image alignment, denoising and normalization.

6. The underwater multi-view 3D reconstruction method according to any one of claims 1 to 5, characterized in that: In step S4, the foreground mask generated by the few-sample multi-view target segmentation strategy is used as a two-dimensional geometric prior to constrain the training process of the neural network; the additional three-dimensional geometric information obtained by monocular depth estimation and normal vector prediction is used to further optimize the training of the SDF.

7. The underwater multi-view 3D reconstruction method according to any one of claims 1 to 6, characterized in that: Step S4 specifically includes: Extract depth maps and normal maps from the enhanced multi-view RGB images to capture the geometric properties of the object surface; Utilize the few-sample multi-view target segmentation strategy to generate a foreground mask to distinguish the object surface from the background elements; Combine the 2D foreground mask with the 3D geometry information obtained through monocular depth estimation and normal vector prediction to form a complete geometric prior. Through a multi-layer perceptron MLP network, the SDF representation of the object is learned from the extracted features and geometric priors; The loss function is defined by combining the reconstruction error and geometric prior constraints to optimize the target during training. During the training process, the MLP network is trained by light sampling and volume rendering techniques to minimize the loss function, thereby learning an accurate SDF representation; Through iterative optimization, the network parameters are adjusted until the 3D reconstruction effect with the predetermined accuracy is achieved.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the underwater multi-view three-dimensional reconstruction method as described in any one of claims 1 to 7 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the underwater multi-view three-dimensional reconstruction method as described in any one of claims 1 to 7 is implemented.

Images