A reversible underwater image enhancement method

Abstract

The application discloses to the technical field of image processing, and particularly relates to a reversible underwater image enhancement method. The method comprises the following steps: constructing an input data set; performing image preprocessing; inputting the images into an InvUIE model in batches, performing forward operation, calling an FGB module to generate a latent space and a high-frequency noise space; calling an IDS module to reduce the distribution gap of the latent space; performing reverse operation to discard the high-frequency noise and output a pure image; constructing an optimization loss function to train the model and save the result; and testing and evaluating the model. The UIE method can effectively reconstruct the degradation area, reduce color deviation, obtain high-quality visual results and SOTA quantitative indicators, has good generalization ability, can guide the distribution of individual noise and clean images, and can balance accuracy and computing resources for a model for a specific task, thereby effectively alleviating the problem of unsuitable UIE caused by insufficient parameters.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a reversible underwater image enhancement method. BACKGROUND

[0002] Underwater image enhancement (UIE) aims to improve the quality of degraded images and correct color distortion. Due to the absorption and scattering of light, underwater images are usually significantly degraded, and different image degradation effects such as distortion, noise and color change are caused by complex underwater environments, which seriously affect the effect of various visual tasks such as classification, detection and tracking. High-quality underwater images are essential for various marine visual tasks, and there is currently a lack of an effective solution to meet the requirements of real-world applications, making it difficult to adapt to complex underwater scenes, and the performance and universality of fixed models and parameters in dynamic underwater environments are poor. Related prior art is as follows:

[0003] (1) Traditional underwater image enhancement method:

[0004] Traditional UIE methods use physical models and prior knowledge to reconstruct clear images. Existing UIE methods are divided into traditional learning methods and deep learning methods. Initially, UIE methods use specialized optical and image formation models (IFM) to enhance the degradation of underwater images. For example, Peng et al. proposed using a dark channel to reduce environmental lighting and turbidity media before changes; Sea-thru is an atmospheric image formation model that can use depth information to correct color distortion; Ancuti et al. proposed a new fusion strategy to reduce noise signal information; Zhuang et al. used a Bayesian retina algorithm with a multi-order gradient prior parameter to correct color conversion. Although the above models can effectively enhance degraded images, they have fixed model parameters and structures, making it difficult to generalize to various complex underwater scenes.

[0005] (2) Convolutional neural network:

[0006] The existing convolutional neural network (CNN) based methods have good representation ability and good generalization ability for nonlinear mapping with a large amount of data, and do not require prior parameters. CNN shows high performance in UIE, which can learn generalizable prior knowledge from large-scale data. For example, Li et al. proposed a new CNN-based model Ucolor to enrich features using multiple spaces in color channels; Guo et al. introduced multi-scale dense data blocks to reconstruct underwater images. However, most CNN-based models require additional supervision for training, and it is very difficult to collect paired training images in the real underwater world. In addition, the CNN model represents the input image with a fixed mapping function, resulting in suboptimal results for any degraded area, so the CNN-based model is difficult to adapt to various complex underwater scenes.

[0007] (3) Adversarial neural network:

[0008] A generative adversarial network (GAN) is used to restore clear underwater images. For example, UGAN enhances images as an image conversion process; WaterGAN uses RGB-D images to estimate depth information and reconstruct clear underwater images; UcycleGAN uses a weak supervision scheme to learn the cross-domain mapping function between input images and reference images. The GAN-based UIE method can produce images with excellent visual effects, but as the size of the existing deep learning model increases and becomes more complex, it is difficult to implement in real-world underwater scenes. The GAN-based method has the following disadvantages: first, it has unstable training and requires longer time to converge; second, it requires a large number of parameters and is computationally expensive. Therefore, the GAN-based UIE method lacks potential space to handle the uncertainty of degraded images and is not suitable for mobile underwater applications.

[0009] (4) Invertible neural network:

[0010] INN was originally designed for unsupervised learning of probabilistic models, which uses a series of differentiable and reversible mapping sequences to convert one unknown distribution to another known distribution without losing information. To reduce the complexity of determinant calculation, researchers modify the flow module using a triangular Jacobian matrix, and the formal probability function that can be processed is: where represents a known random variable. Specifically, f(x) implicitly defines a normalized density model p(x), which can be directly trained by maximum likelihood. The probability density of p(x) and π(z) is related to the following variable formula: logp(x) = logπ(z) + log|det(J f (z))|

[0011] where J f (z) is the Jacobian matrix of the z ~ π(z) latent space distribution, and the computation cost of π(z) is low.

[0012] Inverse neural networks (INN) provide an effective solution for reconstructing high-quality underwater images, and many works have made great progress in various low-level vision tasks, such as denoising, colorization, and super-resolution, etc. For example, Liu et al. proposed a flow-based denoising network (FDN) that can separate noise signals and clear image distributions from degraded images; Zhao et al. designed a double-affine coupling for the image colorization task; Liang et al. proposed a flow-based kernel prior (FKP) to optimize the blind image super-resolution processing. INN also has attractive theoretical and good interpretability in low-level practical characteristics. For example, Xiao et al. developed an inverse re-adjustment network (IRN) for the image super-resolution task; Liu et al. proposed an inverse denoising network (InvDN) to separate noise signal information from input images; An et al. modified an unbiased style transfer framework (ArtFlow) to solve the problem of content leakage. Although these models have achieved ideal results in different tasks, they always need many tuning parameters to improve performance. Therefore, another research direction is to explore the architecture of INN. For example, Ho et al. designed variable dequantization to improve the representability of INN and reduce the complexity of the model; Kingma et al. improved a reversible convolution method, replacing the fixed permutation layer in RealNVP with a new type of simulated synthetic image in RealNVP. However, there are two obvious shortcomings in the INN-based method. First, it is difficult to directly distinguish separate noise distribution and clean image distribution in various underwater scenes; second, there is a lack of task-specific models that can trade off between accuracy and computational resources. When the INN-based method is directly applied to UIE, to ensure accuracy, the number of parameters needs to be increased, losing the advantage of small number of parameters that INN methods should have. Therefore, the INN-based method cannot be directly applied to UIE. SUMMARY

[0013] In view of the problems in the background art, the present application provides a reversible underwater image enhancement method, characterized in that it comprises the following steps:

[0014] 1) Combine the existing underwater image dataset to construct an input data set;

[0015] 2) Randomly rotate and crop the training image block to 256x256, perform image preprocessing, and increase the training data by horizontal and vertical flipping;

[0016] 3) Perform the forward operation of the InvUIE model, input the degraded image x and the reference image y into the flow-based generator in the InvUIE model, generate the latent space and high-frequency noise space

[0017] 4) Reduce the difference between the latent spaces by a supervised method using the reversible downscaling block in the InvUIE model and distribution gap Perform the forward operation of the reversible downscaling block to generate a low-resolution clean image x l and noise distribution z;

[0018] 5) Perform the reverse operation of the InvUIE model, discard the high-frequency noise, and perform image reconstruction to generate a pure image x c ;

[0019] 6) Train the constructed model using the ADAM optimizer and the optimization loss function involved, and save the last trained model weight and result;

[0020] 7) Test the trained model on the test data set and evaluate.

[0021] The existing underwater image data set in step 1) includes a synthetic underwater image data set and a real underwater image data set; the synthetic underwater image data set includes 10 types of water and 1440 pairs of underwater images; the real underwater image data set includes 890 pairs of images and 60 real images without reference images.

[0022] The execution steps of the flow-based generator in step 3) are:

[0023] Use compression operation to maintain local features;

[0024] Initialize data dependency, rescale and bias;

[0025] Reverse the order of feature mapping channels;

[0026] Split the input into x a and x b , extract features using the normalization network and φ, multiply the output of with x b , and add the output of φ to obtain x′ b , concatenate x a and x′ b to obtain the output feature,

[0027] The forward operation is:

[0028] wherein: x a , x b is the split input feature map, x′ b is the output feature map, and φ represent the normalization network;

[0029] The reversible 1x1 convolution is used to make the model have stable generalization ability;

[0030] The split operation is used to connect the feature maps along the channel of the inverse path.

[0031] The execution step of the reversible downscaling block in step 4) is: using discrete wavelet transform to downsample the feature map (H, W, C) to (H / 2, W / 2, 4C), decomposing the feature map into low-frequency and high-frequency representations; using the attention mechanism ResNet block (ρ, μ, ξ) to correct the affine coupling layer, and screening the latent variables that can be beneficial to image enhancement;

[0032] The forward operation is:

[0033] x′ l = ρ(x h ) + x l , x′ h = μ(x′ l ) ⊙ x h + ξ(x′ l )

[0034] wherein: H is the image height, W is the image width, C is the channel number, x l is the low-frequency representation of the feature map,

[0035] x h is the high-frequency representation of the feature map, x′ l is the intermediate latent feature space, x′ h is the clean latent feature space, and ρ, μ, ξ are the ResNet blocks of the attention mechanism.

[0036] In order to avoid the noise that hinders the reconstruction process, the high-frequency noise signal is discarded, and in the reconstruction of the clean image, the normalized z c ~ N(0, 1) distribution is randomly sampled to replace the descending part in the inverse operation, and the inverse operation of the InvUIE model in step 5) is:

[0037] The inverse operation of the flow-based generator is:

[0038]

[0039] wherein: x a , x b is the split input feature map, x′b To output the feature map, And φ represent the normalized network;

[0040] The reverse operation of a reversible downscaling block is:

[0041] x l =x′ l -ρ(x h ),x h =(x′) h -ξ(x′ l )) / μ(x′ l )

[0042] Where: x l For the low-frequency representation of the feature map, x h For the high-frequency representation of the feature map, x′ l Let x′ be the intermediate latent feature space. h For a clean latent feature space, ρ, μ, and ξ are ResNet blocks for the attention mechanism.

[0043] The process of optimizing the loss function and training the constructed model in step 6) is as follows:

[0044] The overall loss function consists of three components for each training phrase, and the loss function during training is as follows:

[0045]

[0046] Where λ1, λ2, and λ3 are the pre-set weights of three loss functions.

[0047] The contrast loss is represented by:

[0048]

[0049] in and φ i (y) is the activation of the i-th layer of the pre-trained VGGNet, N represents the number of training images, and |||1 refers to the L1 norm. Let y be the potential space and y be the reference image;

[0050] Reconstruction loss is defined as:

[0051]

[0052] in, Denotes the potential noise space, x l This represents the low-frequency representation of the feature map;

[0053] represents the latent variable regularization, defined as:

[0054]

[0055] where D is dimension of D, is the log-likelihood value, and ‖‖2 refers to the L2 norm.

[0056] The present application has the beneficial effects that:

[0057] The reversible underwater image enhancement method provided by the present application uses a flow-based generator module to represent an input image by using a latent variable; uses a reversible downscaling block with an attention ResNet block to remove noise signals through readjustment processing; and reconstructs a clean image through backward operation.

[0058] Unlike existing INN, the InvUIE model does not require additional prior knowledge, only uses a set of models to perform forward operation and reverse operation, ensures good image enhancement effect while effectively reducing the parameter amount, and can balance between precision and parameters; through dataset experiments, complexity and parameter experiments, the performance of the InvUIE model disclosed in the present application and other models are compared, and the experimental results show that the InvUIE model is better than other models in qualitative and quantitative indicators.

[0059] The reversible underwater image enhancement method provided by the present application can effectively reconstruct the degradation area, reduce color deviation, obtain high-quality visual results and SOTA quantitative indicators, and has good generalization ability in various underwater scenes, can to a great extent alleviate the problem of insufficient parameters, and effectively solves the problem that the method based on INN cannot be directly applied to UIE. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 A reversible underwater image enhancement method processing step flow chart is disclosed in the present application;

[0061] Figure 2 An InvUIE model schematic diagram is disclosed in the present application;

[0062] Figure 3 A flow-based generation network schematic diagram of an embodiment of the present application is shown in the figure;

[0063] Figure 4 A reversible downscaling block schematic diagram of an embodiment of the present application is shown in the figure;

[0064] Figure 5 A general enhancement framework structure diagram of an embodiment of the present application is shown in the figure;

[0065] Figure 6A framework structure diagram of an InvUIE model of an embodiment of the present application. DETAILED DESCRIPTION

[0066] The present application proposes a reversible underwater image enhancement method, which is further described in detail below in combination with the accompanying drawings.

[0067] As shown in Figure 5 , the purpose of the UIE method is to transform the degraded image x to generate a clean image x c , so that the degraded image x is close to the reference image y. The overall enhancement process can be expressed as x→x c . Since the clean and noise distributions are not isomorphic, x c cannot be directly reconstructed from the degraded image x, so the UIE is regarded as a distribution separation task, assuming that the degraded image x is composed of a noise distribution z and a correlation x c , and a kind of flexible bijective function is used to solve this problem. As shown in Figure 6 , a flow-based generator (FGB) model is used to generate latent spaces and An invertible down-scale block (IDS) is used to reduce the distribution gap between the latent spaces and by a supervised method The specific process is shown in Figure 1 , and the present application discloses a reversible underwater image enhancement method, and the specific steps include:

[0068] 1) Combine the existing underwater image dataset to construct an input data set;

[0069] 2) Randomly rotate and crop the training image block to 256x256, perform image preprocessing, and increase the training data by horizontal and vertical flipping;

[0070] 3) Perform the forward operation of the InvUIE (Invertible Underwater Image Enhancement Network) model, input the degraded image x and the reference image y into the flow-based generator in the InvUIE model, and generate latent spaces and high-frequency noise spaces

[0071] 4) Use the reversible down-scale block in the InvUIE model to reduce the distribution gap between the latent spaces and by a supervised method performing a forward operation of the reversible downscaling block to generate a low-resolution clean image x l and a noise distribution z;

[0072] 5) performing an inverse operation of the InvUIE model, discarding high-frequency noise, and performing image reconstruction to generate a pure image x c ;

[0073] 6) using an ADAM optimizer and an optimization loss function involved to train the constructed model, retaining the last trained model weight and result;

[0074] 7) using the trained model to test the model on a test data set and evaluate.

[0075] The image preprocessing includes steps 1) and 2), and an input data set is constructed in combination with an existing underwater image data set. The underwater image data set includes a synthetic underwater image data set and a real underwater image data set. The synthetic underwater image data set includes 10 water types and 1440 pairs of underwater images; the real underwater image data set includes 890 pairs of images and 60 real images without reference images. In this embodiment, in the training phase, 1250 pairs of synthetic underwater images and 800 pairs of real underwater images are selected to train the InvUIE model; in the test phase, 1000 pairs of synthetic images from UWCNN are selected as Test-S1000; 90 pairs of real underwater images from UIEB are selected as Test-R90; 60 real images without reference images are selected as Test-C60; 16 underwater stereo images from SQUID are selected; 7 color check images from Color-check7 are selected to evaluate the correctness of the color. The number of images in each data set is shown in Table 1. The selected data set contains a large number of paired underwater images with rich scenes. In order to prevent overfitting in training, the images in the data set are augmented, the training data is increased by horizontal and vertical flipping, and the image blocks are randomly rotated and cropped to 256x256.

[0076] Table 1 Summary of the number of images in each data set

[0077]

[0078] As Figure 2 shown, the present application discloses an InvUIE model, which includes l flow cells and m IDS modules, and each flow cell includes k FGB modules. x and y are injected into the flow cell (l x k FGB), denoted as wherein is a latent space, The high-frequency noise space. The input image size HxWxC has spatial dimensions (H, W) and channel dimension C. g(x) maps the input image to the latent space to obtain high-quality invariant feature maps. g -1 (x) represents the inverse process and reconstructs the clean image. The flow downscaling block consists of m IDS, aiming to further downscale the latent noise to IDS generates a low-resolution clean image x l and noise distribution z. The down-sampling process formula is:

[0079] In step 3), the degraded image x and the reference image y are input into the flow-based generator in the InvUIE model to generate the latent space and the high-frequency noise space The specific process is shown in Figure 3

[0080] An FGB module consists of an activation normalization layer (ActNorm), a permutation layer (Permutation), an affine coupling layer (Affine Tran.), and a reversible 1x1 convolution layer (1x1 Conv). To optimize the log-likelihood objective, the FGB model first uses the squeeze operation to maintain local features, and the activation normalization layer can rescale and bias through data-dependent initialization; the permutation layer reverses the order of the channels of the feature map; the affine coupling layer can express complex dependencies on data while keeping the inversion and log-likelihood calculation easy to handle; the reversible 1x1 convolution layer makes the model have stable generalization ability, and the split operation is used to connect the feature maps along the channel of the inverse path, so as to generate the latent space and the high-frequency noise space

[0081] The affine coupling layer splits the input into x a and x b , extracts features using the normalization network and φ, multiplies the output of with x b , and adds the output of φ to obtain x′ b , and connects x a and x′ b to obtain the output feature. In addition, the average latent variable of each channel can be calculated as the direction of the degraded region-aware latent space. The forward calculation is

[0082] The reversible downscaling block in step 4) is as shown in Figure 4 ​As shown, the feature map (H, W, C) is down-sampled to (H / 2, W / 2, 4C) using the discrete wavelet transform to decompose the feature map into low-frequency and high-frequency representations; the affine coupling layer is modified using the ResNet block (p, m, xi) with attention mechanism to screen the latent variables that can be beneficial to image enhancement, and by screening the latent variables, the necessary changes in the latent variables are focused on, the model's interpretability is enhanced, the feature extraction effect is improved, and the ResNet with attention mechanism is used to learn the features in the low-frequency and high-frequency representations to generate the intermediate latent feature space and the pure latent feature space; the supervised method is used to reduce the distribution gap between the latent spaces and distribution gap The forward operation of the reversible downscaling block is performed to generate a low-resolution clean image x l and noise distribution z.

[0083] The forward operation is:

[0084] x′ l = p(x h ) + x l , x′ h = m(x′ l ) o x h + xi(x′ l )

[0085] Where: H is the image height, W is the image width, C is the number of channels, x l is the low-frequency representation of the feature map,

[0086] x h is the high-frequency representation of the feature map, x′ l is the intermediate latent feature space, x′ h is the clean latent feature space, p, m, xi are the ResNet blocks with attention mechanism.

[0087] Unlike other affine couplings, the optimization method described in the present application uses ResNet blocks with channel attention modules to improve the representation of the latent space of the UIE task, and the new coupling flow capability enables IDS to have arbitrary complexity.

[0088] The step 5) performs the inverse operation of the InvUIE model, to avoid the noise that hinders the reconstruction process, the high-frequency noise signal is discarded, and when reconstructing the clean image, the normalized z c ~ N(0, 1) distribution is randomly sampled to replace the descending part in the inverse operation to generate a pure image x c .

[0089] The inverse operation of the flow-based generator is:

[0090]

[0091] wherein: x a , x b is the split input feature map, x′ b is the output feature map, wherein and φ represent the normalization network;

[0092] The inverse operation of the reversible downscaling block is:

[0093] x l = x′ l - ρ(x h ), x h = (x′ h - ξ(x′ l )) / μ(x′ l )

[0094] wherein: x l is the low-frequency representation of the feature map, x h is the high-frequency representation of the feature map, x′ l is the intermediate latent feature space, x′ h is the clean latent feature space, ρ, μ, ξ are the attention mechanism ResNet blocks.

[0095] The process of training the constructed model by optimizing the loss function in step 6) is as follows:

[0096] The overall loss function includes three components of each training phrase, and the loss function in the training process is as follows:

[0097]

[0098] wherein λ1, λ2, λ3 are the preset weights of the three loss functions,

[0099] represents the contrast loss, which minimizes the difference between the two images from the perspective of image derivatives, preserves high-level semantic features and improves visual quality, and is defined as:

[0100]

[0101] wherein and φ i (y) are the activations of the i-th layer of the pre-trained VGGNet, N represents the number of training images, and ‖‖1 refers to the L1 norm, is the latent space, and y is the reference image;

[0102] represents the reconstruction loss, which uses the reconstruction loss to encourage the sample latent distribution closer to low resolution clean x l , is defined as:

[0103]

[0104] where, represents the latent noise space, x l is the low frequency representation of the feature map;

[0105] represents the latent variable regularization, in order to regularize the latent distribution, the obtained latent variable is normalized to follow the probability density function p of the standard Gaussian distribution. The log-likelihood value of the following case is maximized is defined as:

[0106]

[0107] where, D where, is the dimension of, is the log-likelihood value, ‖‖2 refers to the L2 norm.

[0108] The step 6) uses ADAM optimizer, and the specific process of training the constructed model by involving the optimization loss function, reserving the last trained model weight and result is as follows:

[0109] In order to verify the actual effect of the disclosed reversible underwater image enhancement method, a computer configured with an Intel(R) Xeon(R) Silver 4210R CPU and two NVIDIA Tesla V100 GPUs is used to realize the InvUIE model on the PyTorch framework platform. The InvUIE model is an end-to-end model composed of 6 flow units with 4 FGBs and 2 reversible downscaling blocks with 8 reversible blocks. In the training stage, ADAM optimizer is used and the momentum is set to β1=0.9, β2=0.999. The initial learning rate is 2×10 -4 , and it is decayed by multiple steps every 10k iterations. The batch size is set to 26. The training data is increased by horizontal and vertical flipping, and the image blocks are randomly rotated and cropped to 256×256.

[0110] In order to evaluate the effectiveness of the InvUIE model, the step 7) uses the trained model to test and evaluate the test data set, and the specific comparison process is as follows:

[0111] The following UIE methods were selected for comparison, including four traditional methods such as Ancuti et al., Li et al., Peng and Cosman, and GDCP; six deep learning-based methods such as UcycleGAN, Guo et al., Water-Net, UWCNN, Unet-RMT, and Ucolor. To ensure the fairness of the comparison, the retrained UWCNN and Unet were used with the estimated RMT maps to compare with the InvUIE model disclosed herein.

[0112] A quantitative evaluation was performed using the peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and mean squared error (MSE) to evaluate the models. In addition, non-reference metrics were UIQM and UCIQE, which were used to evaluate the performance of these methods in Test-C60 and SQUID without the corresponding reference images. To avoid the large variance of UCIQE in different datasets, the scores were normalized in the range of (0, 1). The NIQE metric indicates the quality of the generated images. The present disclosure invited 30 human subjects to score the perceptual quality ranging from 1 to 5 (worst to best quality). CIEDE2000 measured the relative perception in Color-check 7, and the smaller the value of CIEDE2000, the better the effect.

[0113] The qualitative performance of the average PSNR, SSIM, and MSE values for TestS1000 and TestR90 are shown in Table 2, with the best result in bold and the second best result underlined. Compared with the traditional methods, the deep learning-based models have significant performance in terms of PSNR, SSIM, and MSE results. Ancuti et al., Li et al., Peng and Cosman, GDCP, and UcycleGAN performed poorly in quantitative metrics. WaterNet can effectively enhance the degraded images in Test-r90, but the generalization effect is poor in other underwater datasets. UWCNN requires prior knowledge in the image processing process, but it cannot well restore the degraded images in the actual underwater dataset. UNet-U and Ucolor require additional medium transmission to reconstruct clean images. In contrast, the InvUIE model achieved 1.07 dB / 0.51 dB in PSNR on Test-S1000 and Test-R90 compared to Ucolor, indicating that the InvUIE model described herein can generate high-quality images and preserve more contextual details. In addition, the InvUIE model achieves better results without utilizing additional information and the trade-off between accuracy and parameters.

[0114] Table 2 Evaluation of different methods on Test-S1000 and Test-R90

[0115]

[0116] The results of the comparison methods are shown in Table 3, where the best result is indicated in bold and the second best result is underlined. It is observed that the traditional methods outperform the deep learning based methods in terms of UIQM and UCIQE, etc. The best scores are achieved by UIQM and UCIQE. The InvUIE model achieves the best scores compared to the deep learning based models. For the PS scores, the InvUIE model performs the best in Test-C60 and ranks second in SQUID. This indicates that the InvUIE model is able to produce excellent visual results. In terms of NICE, Ancuti et al. achieve the best scores in Test-C60 and Ucolor achieves the best scores in SQUID dataset. In summary, the InvUIE model achieves good performance on various underwater datasets compared to other deep learning based models, and is able to trade-off perceptual results and background details in the results.

[0117] Table 3 Average perceptual scores (PS), UIQM, UCIQE and NIQE of different methods on Test-C60 and SQUID

[0118]

[0119] To verify the robustness and accuracy of the different methods, the average scores of CIEDE2000 are compared, as shown in Table 4, where the best result is indicated in bold and the second best result is underlined. The corresponding color patches are cropped by the present application and Color-Check7 is used to calculate these values. The InvUIE model achieves the lowest average color values in different cameras and the best color values in W60, W80, CanD10 and FujZ33. Ucolor achieves the lowest scores on OlyT6000 and Ancuti et al. perform the best on OlyT8000 and PanTS1. The results show that the InvUIE model is able to effectively correct color distortion and generalize well to other different camera images. In contrast, the comparison methods fail to recover the true image color information under low contrast.

[0120] Table 4 CIEDE2000 color difference comparison of different methods on Color-Check7

[0121]

[0122] To evaluate the parameter amount and performance of the model, all comparative models were evaluated on the same machine, with a hardware environment using an Intel(R) Xeon(R) Silver 4210R CPU and two NVIDIA Tesla V100 GPUs. The InvUIE model presents a good complexity and performance trade-off, with a parameter amount of only 4.7M, which is significantly lighter than Ucolor (600M) and WaterNet (153.1M). Although the parameter amount of UWCNN is 354.1K, it has poor performance on various underwater datasets and requires prior knowledge. It is unnecessary to compare with physical models in this task, such as Ancuti et al., GDCP, Li et al., and Peng et al. As shown in Table 5, the results of the comparison of each model for images with a size of 256x256 in Test-S1000.

[0123] Table 5 Comparison of InvUIE and other methods in terms of parameters and PSNR (dB)

[0124]

[0125] Through dataset experiments, complexity and parameter experiments, the performance of the InvUIE model disclosed in the present application is compared with that of other models, and the experimental results show that the InvUIE model is superior to other models in qualitative and quantitative indicators. The InvUIE model does not require additional prior knowledge, and compared with traditional general enhancement frameworks, it does not require two sets of models for encoding and decoding, but only uses one set of models, through forward operation and reverse operation, while ensuring good image enhancement effect, effectively reduces the parameter amount, and can balance between precision and parameters. The reversible underwater image enhancement method described in the present application can effectively reconstruct the degradation area, reduce color deviation, obtain high-quality visual results and SOTA quantitative indicators, and has good generalization ability in various underwater scenes, can to a great extent alleviate the problem of insufficient parameters, and effectively solves the problem that the method based on INN cannot be directly applied to UIE.

Claims

1. A method of reversible underwater image enhancement, characterized in that, The method comprises the following steps: 1) constructing an input data set by combining an existing underwater image data set; 2) randomly rotating and cropping the training image blocks to 256x256, performing image preprocessing, and increasing the training data by horizontal and vertical flipping; 3) performing a forward operation of the InvUIE model, inputting the degraded image x and the reference image y into a flow-based generator in the InvUIE model, generating a latent space and high frequency noise space 4) Utilize the reversible downscaling block in the InvUIE model to reduce the distribution gap between latent spaces through a supervised approach and distribution gap Perform the forward operation of the reversible downscaling block to generate a low-resolution clean image x l and noise distribution z; 5) Perform the inverse operation of the InvUIE model, discard high frequency noise, perform image reconstruction, generate clean image x c ; 6) training the constructed model using an ADAM optimizer and the involved optimization loss function, retaining the last trained model weight and result; 7) using the trained model to test the test data set and evaluate the model; The execution steps of the flow-based generator in step 3) are: Use compression operation to maintain local features; Initialize data dependency, rescale and bias; Reverse the order of feature mapping channels; Split input into x a and x b , extract features using normalized network and φ, multiply output of by x b and add to output of φ to get x′ b , concatenate x a and x′ b to get output features, The forward operation is: wherein: x a , x b is the input feature map after segmentation, x′ b is the output feature map, wherein and φ represent the normalization network; Use reversible 1x1 convolution to make the model have stable generalization ability; Use segmentation operation to connect feature maps along the channel of the inverse path; To avoid noise that hinders the reconstruction process, high frequency noise signals are discarded, and in the reconstruction of the clean image, the normalized z c ~N(0,1) distribution to replace the descending part in the inverse operation, the inverse operation of the InvUIE model in step 5) being: The reverse operation of the flow-based generator is: wherein: x a , x b is the input feature map after segmentation, x′ b is the output feature map, wherein and φ represent the normalization network; The reverse operation of the reversible downsampling block is: x l = x' l - p(x h ), x h = (x' h - ξ(x' l )) / μ(x' l ) where: x l is the low-frequency representation of the feature map, x h is the high-frequency representation of the feature map, x′ l is the intermediate latent feature space, x′ h is the clean latent feature space, ρ, μ, ξ are the ResNet blocks of the attention mechanism.

2. A reversible underwater image enhancement method according to claim 1, characterized in that, The existing underwater image data set in step 1) includes a synthetic underwater image data set and a real underwater image data set; the synthetic underwater image data set includes 10 water types and 1440 pairs of underwater images; The real underwater image data set includes 890 pairs of images and 60 real images without reference images.

3. The reversible underwater image enhancement method according to claim 1, characterized in that, The execution steps of the reversible downsampling block in step 4) are: using discrete wavelet transform to downsample the feature map (H, W, C) to (H / 2, W / 2, 4C), decomposing the feature map into low-frequency and high-frequency representations; using an attention mechanism ResNet block (ρ, μ, ξ) to correct the affine coupling layer, and screening potential variables that can benefit image enhancement; The forward operation is: x′ l = ρ(x h ) + x l ,x′ h = μ(x′ l ) ⊙ x h + ξ(x′ l ) where: H is the image height, W is the image width, C is the number of channels, x l is the low-frequency representation of the feature map, x h x' is the high-frequency representation of the feature map l x' is the intermediate latent feature space h x' is the clean latent feature space, ρ, μ, ξ are ResNet blocks of the attention mechanism.

4. The method of claim 1, wherein the method is reversible. The process of training the constructed model using the optimization loss function in step 6) is: The overall loss function includes three components of each training phrase, and the loss function in the training process is as follows: Where λ1, λ2, and λ3 are the weights of the three loss functions preset, represents the contrast loss, defined as: wherein and φ i (y) as the activation of the i-th layer of the pre-trained VGGNet, N denotes the number of training images, and || ||1 denotes the L1 norm, is the latent space, and y is the reference image; denotes the reconstruction loss, defined as: wherein, represents the latent noise space, x l is a low-frequency representation of the feature map; represents the potential variable regularization, defined as: where D = D + D dimension of D, is the log-likelihood value, and || ||2denotes the L2norm.

Images