Underwater Image Enhancement and Depth Estimation Method, Device, and Storage Medium

Abstract

The present invention provides an underwater image enhancement and depth estimation method, apparatus, and storage medium. The method includes: establishing a self-supervised underwater image enhancement and depth estimation network model, including: an image enhancement network, a depth estimation network, a first self-supervised module, and a second self-supervised module; the first self-supervised module takes the enhanced image and depth map corresponding to the first input underwater image as inputs, and generates a first self-supervised signal according to the image generation mechanism; the second self-supervised module reconstructs the second input underwater image and depth map based on the motion prediction result to obtain a second self-supervised signal; training and establishing the model based on the two self-supervised signals; inputting the obtained real underwater image into the trained model to obtain the enhanced underwater image and depth map. In the present invention, underwater image enhancement and depth estimation are performed in a self-supervised manner, which can eliminate the need for a complex manually annotated dataset and achieve an image enhancement effect equivalent to or even better than that of the fully supervised method.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to an underwater image enhancement and depth estimation method, device and storage medium. Background Art

[0002] In recent years, underwater vision has received increasing attention, such as ocean exploration, deep-sea fishing, marine biological monitoring, and so on. However, due to factors such as the refraction and absorption of light in the underwater environment, suspended particles in the water, and color distortion, the quality of visual data will be affected, resulting in noise and distortion in the images.

[0003] Most traditional underwater image enhancement and depth estimation methods adopt a fully supervised approach. Supervised learning is trained through existing training samples (i.e., known data and their corresponding outputs) to obtain an optimal model, and then this model is used to map all new data samples to corresponding output results, and a simple judgment is made on the output results to achieve the purpose of classification. This type of method can generally suppress the noise in the enhancement results well and obtain a good image enhancement effect.

[0004] However, since the fully supervised underwater image enhancement and depth estimation methods require labeled data during training, and the acquisition of labeled data is expensive and difficult, this makes it often difficult to implement the fully supervised underwater image enhancement and depth estimation methods. Summary of the Invention

[0005] In view of this, the present invention provides an underwater image enhancement and depth estimation method, device and storage medium, which perform underwater image enhancement and depth estimation in a self-supervised manner, can eliminate the need for a complex manually labeled data set, save manpower and material resources, and can achieve the same or even better image enhancement effect as the fully supervised method.

[0006] To this end, the present invention provides the following technical solutions:

[0007] On the one hand, the present invention provides a self-supervised underwater image enhancement and depth estimation method, the method comprising:

[0008] Build a self-supervised underwater image enhancement and depth estimation network model; the self-supervised underwater image enhancement and depth estimation network model includes: an image enhancement network, a depth estimation network, a first self-supervised module, and a second self-supervised module; the image enhancement network enhances the first input underwater image to obtain an enhanced image corresponding to the first input underwater image; the depth estimation network performs depth estimation on the first input underwater image to obtain a depth map corresponding to the first input underwater image; the first self-supervised module takes the enhanced image and depth map corresponding to the first input underwater image as inputs, and generates a first self-supervised signal for self-supervised training of the image enhancement network according to the image generation mechanism; the second self-supervised module reconstructs the second input underwater image and the depth map corresponding to the first input underwater image based on the motion prediction result to obtain a second self-supervised signal for self-supervised training of the depth estimation network;

[0009] A self-supervised underwater image enhancement and depth estimation network model established based on training with the first self-supervised signal and the second self-supervised signal;

[0010] Obtain a real underwater image;

[0011] Input the obtained real underwater image into the trained self-supervised underwater image enhancement and depth estimation network model to obtain an enhanced underwater image and a depth map.

[0012] Furthermore, the first self-supervised module takes the enhanced image and depth map corresponding to the first input underwater image as inputs, and generates a first self-supervised signal for self-supervised training of the image enhancement network according to the image generation mechanism, including:

[0013] Input the enhanced image and depth map corresponding to the first input underwater image into the image generation model IFM to generate a first self-supervised signal for self-supervised training of the image enhancement network; wherein, the image generation model IFM includes:

[0014] J(x) = I(x)t(x) + B(1 - t(x));

[0015] t(x) = e -βd ;

[0016] wherein, J(x) is the generated first self-supervised signal, I(x) represents the true radiation intensity of the underwater scene, t(x) represents the enhanced image, B represents the underwater environmental background light, β represents the attenuation rate of light underwater, and d represents the scene depth reflected in the depth map.

[0017] Furthermore, obtain the underwater environmental background light prior through statistical analysis of the real underwater image dataset.

[0018] Further, the second self-supervised module reconstructs the depth maps corresponding to the second input underwater image and the first input underwater image based on the motion prediction result to obtain a second self-supervised signal for self-supervised training of the depth estimation network, including:

[0019] The second self-supervised module includes: a warping module based on the Unet structure; the warping module takes the first input underwater image and the second input underwater image as inputs, predicts camera parameters and inter-frame motion; and reconstructs the depth maps of the second input underwater image and the first input underwater image based on the prediction result to obtain a second self-supervised signal for self-supervised training of the depth estimation network; wherein, the first input underwater image and the second input underwater image are adjacent frame images.

[0020] Further, predicting camera parameters and inter-frame motion includes:

[0021] Predicting the camera intrinsic parameters, rotation angle, and camera displacement.

[0022] Further, reconstructing the depth maps of the second input underwater image and the first input underwater image based on the prediction result includes:

[0023] Reconstruct according to the following formula:

[0024] z′p′ = KRK -1 zp + Kt;

[0025] I′ = W(I1, p′);

[0026] Wherein, z′ represents the new depth map, p′ represents the new homogeneous coordinates, K represents the camera intrinsic parameters, R represents the rotation angle, z represents the depth map, t represents the camera displacement, p represents the pixel coordinates, W() represents the warping operation, I′ represents the second self-supervised signal, and I1 represents the second input underwater image.

[0027] Further, the self-supervised underwater image enhancement and depth estimation network model established based on the first self-supervised signal and the second self-supervised signal includes:

[0028] Construct a loss function using the similarity between the first input underwater image and the first supervision signal, and update the gradient of the loss function and the parameters of the image enhancement network;

[0029] Construct a loss function using the similarity between the first input underwater image and the second supervision signal, and update the gradient of the loss function and the parameters of the depth estimation network.

[0030] Further, construct a style loss function between the enhanced image corresponding to the first input underwater image and the air_image of the NYU dataset.

[0031] On the other hand, the present invention also provides a self-supervised underwater image enhancement and depth estimation device, which includes:

[0032] A model establishment unit, configured to establish a self-supervised underwater image enhancement and depth estimation network model; the self-supervised underwater image enhancement and depth estimation network model includes: an image enhancement network, a depth estimation network, a first self-supervised module, and a second self-supervised module; the image enhancement network enhances a first input underwater image to obtain an enhanced image corresponding to the first input underwater image; the depth estimation network performs depth estimation on the first input underwater image to obtain a depth map corresponding to the first input underwater image; the first self-supervised module takes the enhanced image and the depth map corresponding to the first input underwater image as inputs, and generates a first self-supervised signal for self-supervised training of the image enhancement network according to an image generation mechanism; the second self-supervised module reconstructs a second input underwater image and the depth map corresponding to the first input underwater image based on a motion prediction result to obtain a second self-supervised signal for self-supervised training of the depth estimation network;

[0033] A model training unit, configured to train the established self-supervised underwater image enhancement and depth estimation network model based on the first self-supervised signal and the second self-supervised signal;

[0034] An image acquisition unit, configured to acquire real underwater images;

[0035] An image enhancement and depth estimation unit, configured to input the acquired real underwater images into the trained self-supervised underwater image enhancement and depth estimation network model to obtain enhanced underwater images and depth maps.

[0036] On the other hand, the present invention also provides a computer-readable storage medium, which stores a computer instruction set therein. When the computer instruction set is executed by a processor, it implements the above-mentioned self-supervised underwater image enhancement and depth estimation method.

[0037] Advantages and positive effects of the present invention:

[0038] In terms of the data set, the present invention solves the problem that the underwater vision model cannot be trained due to the lack of ground truth. Since the present invention is a self-supervised deep learning model, there is no need to collect the original pictures and ground truth corresponding to underwater image enhancement and depth estimation in a time-consuming and laborious manner. At the same time, for underwater image enhancement and depth estimation, it is very difficult and costly to obtain effective ground truth. The self-supervised image enhancement and depth estimation algorithm of the present invention only requires the original continuous underwater video data to complete the training of the model, and can solve the problem that the model cannot be trained due to the lack of ground truth.

[0039] In terms of multi - tasking, the complex operations of multiple models for multi - tasking are reduced. The present invention uses an interconnected model structure to combine the two tasks of image enhancement and depth estimation. Through the IFM formula, a connection is formed between the depth map and the enhanced image, which is used to better optimize the model parameters during training. At the same time, only one end - to - end model can complete the two tasks of image enhancement and depth estimation, reducing the complex operations when multiple models handle multiple tasks.

[0040] In terms of depth estimation, since the camera internal parameter K of the publicly available underwater video data is unknown, the present invention uses an internal parameter estimation network to estimate the internal parameter K, the rotation angle R, and the translation t, and then uses them to construct a self - supervised signal for the depth map, solving the problem of unknown camera internal parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0042] Figure 1 It is a flowchart of a self - supervised underwater image enhancement and depth estimation method in an embodiment of the present invention

[0043] Figure 2 It is a network structure diagram of a self - supervised underwater image enhancement and depth estimation network model in an embodiment of the present invention;

[0044] Figure 3 It is a Unet structure diagram of the warping module U in an embodiment of the present invention;

[0045] Figure 4 It is a schematic diagram of obtaining the background light prior by analyzing a large number of underwater images in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] In order to enable those skilled in the art to better understand the solution of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0047] It should be noted that the terms "first", "second", etc. in the description, claims and above-mentioned drawings of the present invention are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0048] As Figure 1 shown, it shows a flowchart of a self-supervised underwater image enhancement and depth estimation method in an embodiment of the present invention. The method includes:

[0049] S1. Establish a self-supervised underwater image enhancement and depth estimation network model;

[0050] In the present invention, an end-to-end model is used to simultaneously perform underwater image enhancement and depth estimation. Only one model needs to be trained to complete two underwater vision tasks, which is a two-for-one approach. Compared with using multiple models to perform different tasks separately, it not only saves training time and GPU computing power expenses, but also only requires running a single model to complete the tasks of image enhancement and depth estimation when outputting results, without the need to laboriously call multiple models.

[0051] As Figure 2 shown, the self-supervised underwater image enhancement and depth estimation network model consists of two parts. Two branches are used to perform underwater image enhancement and depth estimation tasks respectively. The first part, the image enhancement branch, adopts an Encoder-Decoder structure based on ResNet18. The first input underwater image T1 is input into the Encoder module of the convolutional network module. After convolution and downsampling to extract features, the obtained features are then upsampled by the Decoder to obtain the image enhancement result. In addition, the first self-supervised module inputs the enhanced image of the first input underwater image T1 output by the image enhancement network and the depth map of the first input underwater image T1 output by the depth estimation network into the image formation model IFM to generate the first reconstructed image T1_1, which serves as the first self-supervised signal for model training.

[0052] The second part, the depth estimation branch, consists of a depth estimation network and a warping module, which are respectively used for single-image depth estimation and predicted motion tasks. Among them, the depth estimation network adopts the same network structure as the image enhancement network. The first input underwater image T1 is input into the Encoder module of the convolutional network module. After convolution and downsampling to extract features, the low-dimensional feature map is then upsampled to the same dimension as the original image through the Decoder module, and the depth result map of the first input underwater image T1 is output. The warping module adopts a network with a Unet structure, which is used to estimate camera parameters and inter-frame motion. By inputting two pictures of adjacent frames, the camera internal parameter K, rotation angle R, and camera displacement t are predicted. Then, the depth maps of the second input underwater image T2 and the first input underwater image T1 are reconstructed into the second reconstructed image T1_2, that is, the second self-supervised signal, to form the self-supervised training loop of the depth estimation network. At the same time, the depth estimation network and the image enhancement network are connected together through the depth map, making the two branches of the entire network integrated, avoiding the need to train the two modules separately.

[0053] Among them, the image enhancement network and the depth estimation network can adopt the above-mentioned general Encoder-Decoder structure, or other networks such as the Unet structure.

[0054] S2. Establish a self-supervised underwater image enhancement and depth estimation network model through training;

[0055] The depth estimation branch constructs a self-supervised signal for the depth estimation network by adding a prediction network for the camera internal parameter K, camera rotation angle R, and camera translation t. The depth map of the current frame output by the depth estimation network is reconstructed into the next frame together with the current frame and K, R, t. The SSIM and MSE losses are calculated through the real next frame and the reconstructed next frame. Then, the network parameters are optimized through the Adam optimizer. After repeated iterations, the parameters of the depth estimation network reach the optimal point of the model, which is applicable to the depth estimation task. At the same time, the depth output by the depth estimation network can also be used in the image enhancement network. The predicted depth map and the enhanced map output by the enhancement network are reconstructed into the current frame (the original underwater image) through the IFM. After calculating the SSIM and MSE losses, the parameters of the enhancement network are updated. At the same time, the style loss between the air image and the enhanced map is calculated using the style loss to supervise the style of the enhancement result, and the enhancement result has a higher style similarity with the land image. The two self-supervised signals strengthen the supervision of underwater image enhancement and depth estimation. Through the IFM, the depth network and the enhancement network are combined. While outputting their respective tasks, they are also related to a certain extent, which is better used for the training of the network model.

[0056] More specifically:

[0057] On the image enhancement branch, after obtaining the enhanced image and depth map output by the image enhancement module and the depth estimation module, the image T1_1 can be reconstructed according to the IFM formula, which is the first self-supervised signal. After obtaining the reconstructed underwater original image, a loss can be calculated between it and the first input underwater image T1, that is, the first input underwater image T1 and the reconstructed image T1_1 synthesized by IFM are compared for similarity to construct a loss function.

[0058] The IFM formula is as follows:

[0059] J(x) = I(x)t(x) + B(1 - t(x)) (1)

[0060] t(x) = e -βd (2)

[0061] Where, J(x) is the generated first self-supervised signal, I(x) represents the true radiation intensity of the underwater scene, t(x) represents the enhanced image, B represents the underwater environmental background light, β represents the attenuation rate of light underwater, and d represents the scene depth reflected in the depth map.

[0062] Among them, the underwater environmental background light can be an empirical value or a background light prior obtained by analyzing a large number of datasets (as Figure 4 shown).

[0063] To ensure the effectiveness of training, a style-loss is calculated between the enhanced image output by the image enhancement network and the air_image of the NYU dataset, and the image style of air_image is used to supervise the style of the enhanced image.

[0064] On the depth estimation branch, in order to better train the depth estimation network, a warping module is introduced in the depth estimation branch to construct the second self-supervised signal for the depth estimation network. This module takes the adjacent frames T1 and T2 of the video as inputs, estimates the camera internal parameters, rotation angle, and translation vector by predicting the motion of adjacent frames, and then reconstructs the second input underwater image T2 into T1_2 as the supervision signal during the training of the depth network according to formulas 3 and 4. Formulas 3 and 4 are as follows:

[0065] z′p′ = KRK -1 zp + Kt (3)

[0066] I′ = W(I1, p′) (4)

[0067] Among them, z′ represents the new depth map, p′ represents the new homogeneous coordinates, K represents the camera internal parameters, R represents the rotation angle, z represents the depth map, t represents the camera displacement, p represents the pixel coordinates, W() represents the warping operation, I′ represents the second self-supervised signal, and I1 represents the second input underwater image.

[0068] During the training process, the L1 and SSIM losses are used to update the network parameters. After obtaining the network output images T1_1 and T1_2, substitute T1_1, T1 and T1_2, T1 into the L1 and SSIM loss functions respectively, and calculate the gradients of the combined L1 and SSIM losses in each iteration. The network model parameters are optimized by the Adam optimizer to find the optimal parameters suitable for the task. The formulas of the L1 and SSIM loss functions are as follows:

[0069] L MSE =‖T1_1 - T1‖ (5)

[0070]

[0071] In the formula, μ p and μ g are the means of T1_1 and T1 respectively; σ p and σ g are the variances of T1_1 and T1 respectively; σ pg is the covariance, c1 and c2 are constants, usually taking the values c1 = (0.01 × 255) 2 and c2 = (0.03 × 255) 2 .

[0072] Therefore, the total loss function is as follows:

[0073] L = αL SSIM +(1 - α)L MSE (7)

[0074] where α takes the value of 0.8.

[0075] S3. Obtain the real underwater image;

[0076] S4. Input the obtained real underwater image into the trained self-supervised underwater image enhancement and depth estimation network model to obtain the enhanced underwater image and the depth map.

[0077] The real underwater images are respectively input into two branches of the self-supervised underwater image enhancement and depth estimation network model. In the image enhancement module of the first branch, first, the Encoder extracts features from the input RGB underwater image, and the Decoder upsamples the extracted features to reconstruct the image, and then outputs the enhanced underwater image. In the depth estimation convolutional module of the second branch, the Encoder module extracts features from the input image and downsamples them, and the Decoder module upsamples the feature map extracted by the Encoder module, estimates the depth map using the features extracted by the Encoder, and reshapes the low-dimensional image to the same size as the input image, and finally outputs the depth map.

[0078] In the embodiments of the present invention, a self-supervised method is used for underwater image enhancement, which can eliminate the need for complex manually annotated datasets, saving manpower and material resources, and can achieve the same effect as the fully supervised method, or even outperform the fully supervised method.

[0079] Corresponding to a self-supervised underwater image enhancement and depth estimation method in the present invention, the present invention also provides a self-supervised underwater image enhancement and depth estimation device, including:

[0080] A model establishment unit, configured to establish a self-supervised underwater image enhancement and depth estimation network model; the self-supervised underwater image enhancement and depth estimation network model includes: an image enhancement network based on the Encoder-Decoder structure of ResNet18, a depth estimation network based on the Encoder-Decoder structure of ResNet18, a first self-supervised module, and a second self-supervised module; the image enhancement network enhances the first input underwater image to obtain an enhanced image corresponding to the first input underwater image; the depth estimation network performs depth estimation on the first input underwater image to obtain a depth map corresponding to the first input underwater image; the first self-supervised module takes the enhanced image and depth map corresponding to the first input underwater image as inputs, and generates a first self-supervised signal for self-supervised training of the image enhancement network according to the image generation mechanism; the second self-supervised module reconstructs the second input underwater image and the depth map corresponding to the first input underwater image based on the motion prediction result to obtain a second self-supervised signal for self-supervised training of the depth estimation network;

[0081] A model training unit, configured to train the established self-supervised underwater image enhancement and depth estimation network model based on the first self-supervised signal and the second self-supervised signal;

[0082] An image acquisition unit, configured to acquire real underwater images;

[0083] An image enhancement and depth estimation unit, configured to input the acquired real underwater image into a trained self-supervised underwater image enhancement and depth estimation network model to obtain an enhanced underwater image and a depth map.

[0084] For a self-supervised underwater image enhancement and depth estimation device according to an embodiment of the present invention, since it corresponds to a self-supervised underwater image enhancement and depth estimation method in the above embodiment, the description is relatively simple. For related similarities, please refer to the description of the self-supervised underwater image enhancement and depth estimation method in the above embodiment, and details are not repeated here.

[0085] An embodiment of the present invention also discloses a computer-readable storage medium, in which a computer instruction set is stored. When the computer instruction set is executed by a processor, it implements a self-supervised underwater image enhancement and depth estimation method provided in any of the above embodiments.

[0086] In several embodiments provided by the present invention, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of the units can be a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the units or modules can be in electrical or other forms.

[0087] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0088] In addition, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0089] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs.

[0090] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A self-supervised underwater image enhancement and depth estimation method, characterized in that, The method includes: Building a self-supervised underwater image enhancement and depth estimation network model; the self-supervised underwater image enhancement and depth estimation network model includes: an image enhancement network, a depth estimation network, a first self-supervised module, and a second self-supervised module; the image enhancement network enhances the first input underwater image to obtain an enhanced image corresponding to the first input underwater image; the depth estimation network performs depth estimation on the first input underwater image to obtain a depth map corresponding to the first input underwater image; the first self-supervised module takes the enhanced image and depth map corresponding to the first input underwater image as inputs, and generates a first self-supervised signal for self-supervised training of the image enhancement network according to the image generation mechanism; the second self-supervised module reconstructs the second input underwater image and the depth map corresponding to the first input underwater image based on the motion prediction result to obtain a second self-supervised signal for self-supervised training of the depth estimation network; Training and building a self-supervised underwater image enhancement and depth estimation network model based on the first self-supervised signal and the second self-supervised signal; Obtaining a real underwater image; Inputting the obtained real underwater image into the trained self-supervised underwater image enhancement and depth estimation network model to obtain an enhanced underwater image and a depth map; Among them, the second self-supervised module includes: a warping module based on the Unet structure; the warping module takes the first input underwater image and the second input underwater image as inputs, predicts the camera parameters and inter-frame motion; and reconstructs the depth maps of the second input underwater image and the first input underwater image based on the prediction result to obtain a second self-supervised signal for self-supervised training of the depth estimation network; where the first input underwater image and the second input underwater image are adjacent frame images; Among them, predicting the camera parameters and inter-frame motion includes: predicting the camera intrinsic parameters, rotation angle, and camera displacement; Among them, reconstructing the depth maps of the second input underwater image and the first input underwater image based on the prediction result includes: Reconstructing according to the following formula: z′p′ = KRK -1 zp + Kt; I′ = W(I1, p′); Where z′ represents the new depth map, p′ represents the new homogeneous coordinates, K represents the camera intrinsic parameters, R represents the rotation angle, z represents the depth map, t represents the camera displacement, p represents the pixel coordinates, W() represents the warping operation, I′ represents the second self-supervised signal, and I1 represents the second input underwater image.

2. The self-supervised underwater image enhancement and depth estimation method according to claim 1, characterized in that, The first self-supervised module takes the enhanced image and depth map corresponding to the first input underwater image as inputs, and generates a first self-supervised signal for self-supervised training of the image enhancement network according to the image generation mechanism, including: Inputting the enhanced image and depth map corresponding to the first input underwater image into the image generation model IFM to generate a first self-supervised signal for self-supervised training of the image enhancement network; where the image generation model IFM includes: J(x) = I(x)t(x) + B(1 - t(x)); t(x)=e -βd ; Where J(x) is the generated first self-supervised signal, I(x) represents the true radiation intensity of the underwater scene, t(x) represents the enhanced image, B represents the underwater environmental background light, β represents the attenuation rate of light in water, and d represents the scene depth reflected in the depth map.

3. A self-supervised underwater image enhancement and depth estimation method according to claim 2, characterized in that The underwater environmental background light prior is obtained through statistical analysis of a real underwater image dataset.

4. A self-supervised underwater image enhancement and depth estimation method according to claim 1, characterized in that A self-supervised underwater image enhancement and depth estimation network model established based on training with a first self-supervised signal and a second self-supervised signal includes: Construct a loss function using the similarity between the first input underwater image and the first supervised signal, and update the gradient of the loss function and the parameters of the image enhancement network. Construct a loss function using the similarity between the first input underwater image and the second supervised signal, and update the gradient of the loss function and the parameters of the depth estimation network.

5. A self-supervised underwater image enhancement and depth estimation method according to claim 4, characterized in that, Construct a style loss function between the enhanced image corresponding to the first input underwater image and the air_image of the NYU dataset.

6. An underwater image enhancement and depth estimation device based on self-supervision, characterized in that, The device includes: A model establishment unit for establishing a self-supervised underwater image enhancement and depth estimation network model; the self-supervised underwater image enhancement and depth estimation network model includes: an image enhancement network, a depth estimation network, a first self-supervised module, and a second self-supervised module; the image enhancement network enhances the first input underwater image to obtain an enhanced image corresponding to the first input underwater image; the depth estimation network performs depth estimation on the first input underwater image to obtain a depth map corresponding to the first input underwater image; the first self-supervised module takes the enhanced image and depth map corresponding to the first input underwater image as inputs, and generates a first self-supervised signal for self-supervised training of the image enhancement network according to an image generation mechanism; the second self-supervised module reconstructs the second input underwater image and the depth map corresponding to the first input underwater image based on the motion prediction result to obtain a second self-supervised signal for self-supervised training of the depth estimation network; wherein, the second self-supervised module includes: a warping module based on the Unet structure; the warping module takes the first input underwater image and the second input underwater image as inputs, predicts camera parameters and inter-frame motion; and reconstructs the depth maps of the second input underwater image and the first input underwater image based on the prediction result to obtain a second self-supervised signal for self-supervised training of the depth estimation network; wherein, the first input underwater image and the second input underwater image are adjacent frame images; wherein, predicting camera parameters and inter-frame motion includes: predicting camera internal parameters, rotation angle, and camera displacement; wherein, reconstructing the depth maps of the second input underwater image and the first input underwater image based on the prediction result includes: reconstructing according to the following formula: z′p′ = KRK -1 zp + Kt; I′ = W(I1, p′); where, z′ represents the new depth map, p′ represents the new homogeneous coordinates, K represents the camera internal parameters, R represents the rotation angle, z represents the depth map, t represents the camera displacement, p represents the pixel coordinates, W() represents the warping operation, I′ represents the second self-supervised signal, and I1 represents the second input underwater image; A model training unit for training and establishing a self-supervised underwater image enhancement and depth estimation network model based on the first self-supervised signal and the second self-supervised signal; An image acquisition unit for acquiring real underwater images; An image enhancement and depth estimation unit for inputting the acquired real underwater images into the trained self-supervised underwater image enhancement and depth estimation network model to obtain enhanced underwater images and depth maps.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer instruction set, which, when executed by a processor, implements the underwater image enhancement and depth estimation method based on self-supervision as described in any one of claims 1 to 5.

Images