Underwater image enhancement method and system based on conditional generative adversarial network

Abstract

The application provides an underwater image enhancement method and system based on a conditional generative adversarial network, color of degraded underwater images is corrected by extracting and fusing multi-scale local features and global features, feature extraction effect is improved by constructing an attention module AMU for underwater image enhancement, and in training, a perception loss and a total variation loss are introduced to improve the quality of generated images and inhibit noise from occurring.The method can provide clear underwater environment information for high-level visual tasks such as behavior monitoring and disease identification of intelligent aquaculture, and promote healthy and sustainable development of intelligent intensive aquaculture.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to an underwater image enhancement method and system based on a conditional generative adversarial network. BACKGROUND

[0002] As of 2021, the global aquaculture industry has reached 2094.2 million US dollars. With the rapid development of the aquaculture industry, the demand for fish behavior monitoring, fish disease identification and other work is gradually expanding, so clear underwater images are needed to provide highly usable image resources for these high-level visual tasks. Currently, relevant studies have shown that, compared with original images, enhanced images have better improvement effects on key point matching, target detection, target tracking and other aspects. In the intelligent aquaculture industry, visual work such as underwater biological monitoring and underwater fish tracking requires clear underwater image resources. However, unlike the atmospheric environment, in the underwater environment, water has an absorption and scattering effect on light, and there are also suspended particles in the water, which cause the underwater image to appear color cast, blur and other degradation phenomena, hindering the development of related underwater work.

[0003] The degraded underwater images are difficult to be directly applied to the related underwater work of intelligent aquaculture, and also bring challenges to traditional image processing technology. Therefore, researchers gradually carry out related research on underwater image enhancement. The traditional underwater image enhancement method adopts fixed parameters and physical models, and adjusts the pixel value of the image to enhance the degraded image. However, the above method is only for processing images in a single environment, and cannot adapt to various complex underwater environments. Due to the excellent performance of the convolutional neural network (CNN), it is widely used in many computer vision tasks. Therefore, researchers begin to introduce CNN into the field of underwater image enhancement. Based on CNN, an underwater image enhancement framework UIE-Net is proposed to perform color correction; a residual learning strategy is introduced, and a underwater residual convolutional neural network URCNN is proposed based on CNN. Since the generative adversarial network (GAN) appeared, it has been widely used in image processing, text generation, audio and video generation and other fields. This network can generate similar real data in an antagonistic way, which just makes up for the shortage of the underwater image dataset lacking of images before degradation. Some people also propose WaterGAN to generate paired underwater image dataset and perform color correction; by combining the cycle generative adversarial network (CycleGAN) and the dark channel prior algorithm, some researchers propose an underwater image restoration method based on multi-scale cycle generative adversarial network (MCycle GAN) and a new underwater image enhancement model FUnIE-GAN based on conditional generative adversarial network (CGAN), and also construct a dataset EUVP containing paired and unpaired underwater images. These learning-based methods are trained with a large amount of data to adapt to various underwater environments. Therefore, improving the clarity of underwater image extraction is an urgent problem to be solved at present for intelligent aquaculture. SUMMARY

[0004] In order to solve the above problems, the present application provides an underwater image enhancement method and system based on a conditional generative adversarial network, which is used for color correction of degraded underwater images, thereby providing a clear visual environment for subsequent visual work.

[0005] In one aspect, the present application provides an underwater image enhancement method based on a conditional generative adversarial network, comprising the following steps:

[0006] Step 1: Obtain a paired image set of underwater degraded images and corresponding pure images, and divide them into a training set and a test set;

[0007] Step 2: Scale all images to the same size;

[0008] Step 3: Model construction, including: extracting global and local features from the image based on the encoder-decoder structure; fusing the global features with local features at each scale; restoring the image by upsampling the global features layer by layer, with each upsampling layer connected to the fused features at the corresponding scale; and feeding the generated image into the discriminator network to determine whether it comes from real data, prompting the generator network to make adjustments.

[0009] Step 4: Model training and testing, and saving the tested model;

[0010] Step 5: Process the actual underwater images using the tested model.

[0011] Preferably, the codec structure is an improved U-Net network, which includes 8 downsampling layers and extracts global and local features from the input image through layer-by-layer convolution.

[0012] More preferably, each downsampling layer consists of a LeakyReLU layer, a two-dimensional convolutional layer, and a batch normalization layer.

[0013] Preferably, during the downsampling process, based on SENet and NAM modules, the global average pooling module in SENet is replaced with the batch normalization scaling factor in the NAM module to construct an attention module for underwater image enhancement.

[0014] More preferably, in the attention module, the input feature map is processed by batch normalization layer and 1×1 convolution, multiplied by weight coefficients, and then passed through ReLU activation function, 1×1 convolution layer and sigmoid activation function, and finally connected to the input feature map by skip connection.

[0015] Preferably, global and local features are fused before the upsampling results with the same resolution are connected in a skip-layer manner. The fusion process is as follows:

[0016] Step 4-1: The global features f are processed by a convolutional layer with a kernel size of 1×1 and a stride of 1. g Number of channels c g Adjusted to the local feature map f corresponding to scale i l Same number of channels c i This step is represented as f g1 =F conv (f g W)

[0017] Among them, F conv This represents a convolution operation, where W is the learnable weight.

[0018] Step 4-2: For f g1 Perform replication, with a replication quantity of h. i× w i where h i and w i are the length and width of the local feature map f l of scale i, the operation is denoted as

[0019] f g2 = F copy (f g1 , num = h i × w i )

[0020] Step 4-3: reshape f g2 to the same dimension h l × w i × c i as f i

[0021] f g3 = F re (f g2 , size = h i × w i × c i )

[0022] where F re denotes the reshape operation.

[0023] Step 4-4: concatenate f g3 with f l

[0024] f out = F concat (f l , f g3 ).

[0025] Preferably, the image restoration is based on an improved U-Net network, which includes 8 up-sampling layers and corresponds to the down-sampling layers.

[0026] More preferably, each of the up-sampling layers includes a ReLU layer, a bilinear up-sampling layer, a convolution layer and a batch normalization layer.

[0027] Preferably, the overall objective function of the model training loss is:

[0028]

[0029] where WGAN-GP, L1, L p , L TV are loss functions, λ1=10 -1 , λ2=10 -2 , λ3=10 -3 ; ​

[0030]

[0031] wherein x is a degraded underwater image, gt is a real underwater image with good details, is a uniform sampling between the generated image G(x) and the real image gt, and λ = 10.

[0032] On the other hand, the present application proposes an underwater image enhancement system based on conditional generative adversarial network, comprising:

[0033] A data set construction module is configured to obtain a pair of images set of underwater degraded images and corresponding pure images, and divide them into a training set and a test set;

[0034] An image processing module is configured to scale all images to the same size;

[0035] A model construction module comprises: based on the codec structure, global and local features of the image are extracted; the global features are fused with the local features of each scale respectively; the global features are restored through layer-by-layer upsampling, and each upsampling layer is connected with the corresponding scale of the fused features; the generated image is sent to the discriminator network to determine whether it comes from the real data, so as to prompt the generator network to adjust;

[0036] A model training and testing module is configured to input the image into the model for training and testing, and save the tested model;

[0037] A model application module is configured to process the actual underwater image by using the tested model.

[0038] The present application has the beneficial effects that: for the degradation phenomenon of underwater images, the present application proposes an underwater image enhancement method and system based on conditional generative adversarial network, extracts and fuses global features and multi-scale local features based on the conditional generative adversarial network framework, filters noise in degraded images, and effectively improves the quality of images. An attention module AMU for underwater image enhancement is constructed at the end of the feature extraction network, which improves the feature extraction effect; the trained model weight difference measurement is used to highlight key features, and a weight sparsity penalty is applied to the attention module to improve the calculation efficiency; the perceptual loss and total variation loss are introduced to make the generated image have similar high-level semantic information as the real image, which enhances the image generation effect of the generator network and suppresses image noise. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 is a GAN model structure diagram in the prior art;

[0040] Figure 2 is a conditional generative adversarial network underwater image enhancement method flowchart of an embodiment of the present application;

[0041] Figure 3 is the SE module structure embedded in ResNet;

[0042] Figure 4 is the channel attention submodule structure diagram;

[0043] Figure 5 is the spatial attention submodule structure diagram;

[0044] Figure 6 is the SE module and AMU module structure diagram;

[0045] Figure 7 is the visual comparison result of the image enhanced by the method of the present application in the UGAN dataset. DETAILED DESCRIPTION

[0046] The embodiments will be described in detail below with reference to the accompanying drawings.

[0047] Thanks to the adversarial training method of the generative adversarial network (GAN), it has good performance in the field of text generation, image processing, etc. GAN contains two models, namely the generator and the discriminator. In the network training process, the generator produces instances similar to the original data by receiving random noise z, denoted as G(z), to deceive the discriminator. The discriminator is used to judge whether the instance generated by the generator is artificially forged or from the real data. The input of the discriminator is x, i.e. the instance generated by the generator; the output is D(x), i.e. the probability that x is real data. Both sides are alternately optimized in the continuous iteration process, so as to achieve the balance of the two, i.e. the generator can produce instances with better details, and the discriminator is difficult to judge the output result of the generator. The overall process of GAN is as follows Figure 1 .

[0048] The objective function of the GAN model is as follows:

[0049]

[0050] wherein, means that the parameters of the discriminator D are updated by maximizing the cross-entropy loss V(D, G) under the condition that the generator is fixed. means that the generator minimizes the cross-entropy loss under the condition that the discriminator maximizes the cross-entropy loss V(D, G) of true and false instances. In the training process, the parameters of the discriminator are generally updated first, because in the early stage of training, the performance of the discriminator is poor and cannot play a role in pushing the generator to produce higher quality instances.

[0051] Compared with the traditional generative adversarial network, the input of the conditional generative adversarial network introduces condition information, so that the generation result of the whole network is more stable and controllable. Figure 2 For its flowchart. Including:

[0052] Step 1: data set construction

[0053] UGAN data set is selected as the training and test data set of the method, which is generated by CycleGAN by learning the mapping relationship between degraded images and pure images. In this embodiment, the data set contains 6128 pairs of images. 6000 pairs of images are selected as the training set, and the remaining 128 pairs of images are the test set.

[0054] Step 2: image preprocessing

[0055] Image preprocessing is mainly to unify the image size, and scale the image to the same size. In the embodiment of the application, all images are scaled to a size of 256x256.

[0056] Step 3: multi-scale feature extraction

[0057] Global feature Figure 1 Generally contains the overall information of the image, such as color, texture, shape, etc., which can enhance the model's perception ability to the scene environment. The application refers to the classic U-Net network, and extracts global and local features based on the encoder-decoder structure. In the global and local feature extraction network, the original 4-layer down-sampling in the U-Net network is expanded to 8-layer down-sampling, aiming to extract more scale local features and enrich the semantic information of the global feature map. In addition, unlike the U-Net network which uses maximum pooling to down-sample, the input image is extracted by global and local features through layer-by-layer convolution. Each down-sampling layer is composed of a LeakyRelu layer, a two-dimensional convolution layer (convolution kernel size is 4, step is 2) and a batch normalization layer, and the output size is 1x1xc g , c g is the number of channels. Such a down-sampling method can improve the extraction effect of local features, so that the generated image has more details.

[0058] With the increase of the number of down-sampling layers, the number of features contained gradually increases. In order to make the network focus on the key features of the image, an attention module (AMU) for underwater image enhancement is constructed based on SENet and NAM module in the down-sampling process, which focuses on detail information and context information, thereby improving the feature extraction effect.

[0059] SENet model can be embedded in other network structures more conveniently. SENet pays more attention to the connection in the channel dimension, including Squeeze and Excitation two operations. In the Squeeze operation, the model uses global average pooling to encode the features in the entire space in the channel dimension into a global feature map. In the Excitation operation, SENet can learn the weight coefficients of each channel, enhancing the model's ability to distinguish features in each channel. In related experiments, SE module is embedded in other networks such as ResNet, VGG-16, etc., and obvious improvement in error indicators is achieved. The SE module embedded in ResNet is shown in Figure 3 .

[0060] The normalized channel attention module (NAM) aims to focus on insignificant feature weights, and by imposing a sparse weight penalty on the attention module, it improves the efficiency of weight calculation while maintaining network performance. The NAM module is integrated based on the CBAM module, and the channel and spatial attention sub-modules are redesigned. In the residual network, the module is embedded at the end of the residual structure. In the channel attention sub-module, the module uses the scale factor in batch normalization, and its formula is as follows:

[0061]

[0062] where γ and β are trainable transformation parameters, and are the mean and standard deviation of the mini-batch , respectively. The channel attention sub-module is shown in Figure 4 , M c represents the output, γ is the scale factor of each channel, and ω is the weight of each channel.

[0063] The normalized scale factor is also applied in the spatial attention sub-module, named pixel normalization. The spatial attention sub-module is shown in Figure 5 , M s represents the output, and λ is the scale factor.

[0064] In the present application, the global average pooling module in SENet is replaced by the batch normalization scale factor in the NAM module to improve the suppression effect of insignificant features. The structure of the AMU module is shown in Figure 6 , the input feature map is processed by the batch normalization layer and the 1x1 convolution, and multiplied by the weight coefficient, then passed through the ReLU activation function, the 1x1 convolution layer and the sigmoid activation function, finally connected with the input feature map.

[0065] Step 4: Global and local feature fusion

[0066] To enhance the processing performance of low-resolution images and improve the color and detail of the enhanced images using global feature maps with high-level semantic information, a global and local feature fusion module is constructed before the skip-layer connection between the upsampling results and the downsampling results with the same resolution. This suppresses artifacts in the enhanced images. The module's workflow is as follows:

[0067] First, the global feature map f is processed by a convolutional layer with a kernel size of 1×1 and a stride of 1. g Number of channels c g Adjusted to the local feature map f corresponding to scale i l Same number of channels c i This step is represented as f g1 =F conv (f g W)

[0068] Among them, F conv This represents the convolution operation, where W is the learnable weight.

[0069] Subsequently, for f g1 Perform replication, with a replication quantity of h. i ×w i , where h i with w i For the local feature map f at scale i l The length and width, this operation is represented as

[0070] f g2 =F copy (f g1 , num = h i ×w i )

[0071] Then, f g2 Remodeling into f l Same dimension h i ×w i ×c i

[0072] f g3 =F re (f g2 size=h i ×w i ×c i )

[0073] Among them, F re This indicates a reshaping operation.

[0074] Ultimately, f g3 with f l Perform connection operation

[0075] fout = F concat (f l , f g3 )

[0076] At this point, the global feature map completes the convolution, replication, reshaping and connection steps.

[0077] Step 5: Feature up-sampling

[0078] The global feature map is restored by layer-by-layer up-sampling, each up-sampling layer is connected with the fused feature having the same size, and the color deviation existing in the original image is corrected. The 4-layer up-sampling in the U-Net network is expanded to 8-layer up-sampling, which corresponds to the down-sampling layer in the feature extraction stage. Each up-sampling layer includes a ReLU layer, a bilinear up-sampling layer, a convolution layer (convolution kernel size is 4, step is 2) and a batch normalization layer, and the output size is 256x256xc g , c g is the number of channels.

[0079] Step 6: Image discrimination

[0080] The generated image is sent to the discriminator network PatchGAN for discrimination, the input is 256x256xc g , c g is the number of channels, and PatchGAN maps the input image into an NxN matrix, each point in the matrix represents the discrimination value for a small area of the image. Such a discrimination method can discriminate more details of the image, and when all areas have good details, the overall image will be discriminated as real.

[0081] Step 7: Model training and testing

[0082] In the model training stage, the loss function of WGAN-GP is introduced to stabilize the training, and the formula is as follows:

[0083]

[0084] Where x is the degraded underwater image, gt is the real underwater image with good details, is the uniform sampling between the generated image G(x) and the real image gt, and λ represents the weight factor.

[0085] In addition, compared with L2 loss, traditional L1 loss can make the generator produce less blur, therefore, the present application introduces L1 loss, and the formula is as follows:

[0086]

[0087] The application introduces a perception loss function to constrain the generated image at the deep feature level, prompting the generated image to have similar high-level semantic information as the real image. The perception loss model is trained based on the VGG-19 network, and weight distribution is allocated for feature matching of each module, as follows:

[0088]

[0089] wherein, is the jth layer of the convolutional layer, J represents the reference image, is the enhanced image.

[0090] In order to reduce the noise of the generated image and increase the smoothness of the image, the application introduces a traditional total variation loss function, as follows:

[0091]

[0092] wherein, is the horizontal gradient operator, represents the vertical gradient operator.

[0093] The overall objective function is as follows:

[0094]

[0095] Before training, all pictures participating in the training are scaled to a size of 256x256. Model training is performed on Intel(R) Xeon(R) E5-2630 v4 and NVIDIA GTX 1080, with an environment configuration of Pytorch1.5, and the weight of the loss function is set to λ = 10, λ1 = 10 -1 , λ2 = 10 -2 , λ3 = 10 -3 .The Adam optimizer is introduced to replace the traditional gradient descent optimization algorithm, with an initial learning rate of 1e -4 , β1 = 0.5, β2 = 0.99, a batch size of 16, and an iteration number of 50 for model training.

[0096] The comparison of the evaluation indexes of the UGAN data set is shown in Table 1.

[0097] Table 1 Comparison of UGAN data set evaluation indexes

[0098] PSNR↑ SSIM↑ UIQM↑ UCIQE↑ Fusion 18.2647 0.6437 2.7266 0.0625 IBLA 20.2019 0.6059 3.1725 0.0523 UDCP 18,6979 0.6171 3.5883 0.0415 ULAP 20.6336 0.6535 3.3515 0.0533 UGAN 23.3311 0.7497 2.8354 0.0392 FunieGAN 22.8422 0.7248 3.1934 0.0788 WaterNet 23.5637 0.7491 2.4786 0.0393 Style-Transfer 24.2179 0.7714 2.9364 0.0695 ]]> ​ UWCNN 17.2855 0.6332 2.3561 0.0452 MLFcGAN 25.1974 ]]> ​ 0.7982 ]]> ​ 4.1145 ]]> ​ 0.0533 MA-cGAN 26.1698 0.8281 5.0935 0.0638

[0099] The MA-cGAN has obvious advantages in the PSNR and SSIM indicators.

[0100] Due to the lack of paired pure images of real images, the application only selects a no-reference indicator to evaluate the quality of the enhanced results, and the comparison results are shown in Table 2.

[0101] Table 2 Comparison of evaluation indicators of real data set

[0102] UIQM↑ UICM↑ UISM↑ UIConM↑ UCIQE↑ Fusion 3.8687 ]]> ​ 3.2421 1.8985 0.0536 0.0465 IBLA 3.7646 3.6631 1.3435 ]]> ​ 0.0643 0.0476 UDCP 3.4876 3.1727 1.0694 0.0319 0.0297 ULAP 3.6588 <![CDATA[ 3.6379 ]]> 1.2364 0.0719 ]]> ​ 0.0488 UGAN 2.4739 2.5876 0.9506 0.0374 0.0314 FunieGAN 2.5422 3.1297 1.1004 0.0526 0.0469 WaterNet 2.7389 2.6592 1.0301 0.0584 0.0421 Style-Transfer 3.3795 3.0789 1.1373 0.0939 0.0513 UWCNN 2.5208 2.2156 1.1123 0.0417 0.0494 MLFcGAN 3.4831 3.0118 1.2366 0.0549 0.0536 ]]> ​ MA-cGAN 4.0794 3.3511 1.1626 0.0517 0.0562

[0103] The MA-cGAN performs better in the UIQM and UCIQE indicators, which indicates that the results obtained by using the method of the application have good color density and clarity. In terms of no-reference indicators, the traditional methods are better than the learning-based methods, and the enhanced results of these methods may have better saturated colors, but these images with high saturation may not be used for subsequent target detection and the like. The results also show that the method (MA-cGAN) of the application can be applied to various underwater environments.

[0104] The comparison results of the enhanced images in the UGAN data set are shown in Figure 7 The results show that the learning-based methods can achieve better results than the traditional methods. The results of the traditional methods mostly have over-saturation phenomenon, such as UDCP, IBLA and ULAP. For Fusion, its result has overexposure. Among the learning-based methods, the results of the GAN-based methods such as UGAN, FunieGAN and Style-Transfer will lose some texture information. At the same time, the results of the CNN-based methods including WaterNet and UWCNN lack detailed information. Unlike the above results, the effect of MLFcGAN seems more natural. Compared with the MLFcGAN, the result obtained by using the method of the application is further optimized in terms of color saturation.

[0105] Step 8: Perform underwater image processing by using the trained model.

[0106] In addition, the application also provides an underwater image enhancement system based on a conditional generative adversarial network, comprising:

[0107] A data set construction module is configured to obtain a set of underwater degraded images and corresponding pure images, and divide them into a training set and a test set;

[0108] An image processing module is configured to scale all the images to the same size.

[0109] a global and local feature extraction module for extracting global and local features of the image based on a codec structure;

[0110] a global and local feature fusion module for fusing the global feature map with the local features of each scale respectively;

[0111] a feature up-sampling module for restoring the image by layer-by-layer up-sampling of the global feature map, and connecting each up-sampling layer with the fused features of the corresponding scale;

[0112] an image discrimination module for inputting the generated image into a discriminator network to determine whether it comes from real data, and prompting the generator network to adjust;

[0113] a model training and testing module for inputting the image into the model for training and testing, and saving the tested model;

[0114] a model application module for processing the underwater image by using the tested model.

[0115] This embodiment is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An underwater image enhancement method based on a conditional generative adversarial network, comprising the following steps: Step 1: obtaining a paired image set of underwater degraded images and corresponding pure images, and dividing them into a training set and a test set; Step 2: scaling all images to the same size; Step 3: model construction, including: based on the encoder-decoder structure, extracting global and local features of the image; fusing the global features with the local features of each scale respectively; restoring the image by layer-by-layer upsampling of the global features, and connecting each upsampling layer with the fused features of the corresponding scale; inputting the generated image into the discriminator network to determine whether it comes from real data, prompting the generator network to adjust; Step 4: model training and testing, saving the tested model; Step 5: using the tested model to process actual underwater images; The encoder-decoder structure is an improved U-Net network, which includes 8 layers of down-sampling layers and extracts global and local features of the input image through layer-by-layer convolution; In the down-sampling process, based on the SENet and NAM module, the global average pooling module in SENet is replaced by the batch normalization scaling factor in the NAM module to construct an attention module for underwater image enhancement; before the results of layer-by-layer upsampling are connected with the down-sampled results with the same resolution, global and local feature fusion is performed, and the fusion process is: Step 4-1: Adjust the channel number c g of the global feature f g to the same channel number c l of the local feature map f i of the corresponding scale i by a convolution layer with kernel size 1x1 and stride 1, which is denoted as f g1 = F conv (f g , W) where F conv denotes a convolution operation, and W is a learnable weight. Step 4-2: f g1 is replicated h times i × w i , where h i and w i are the length and width of the local feature map f l of scale i, which is represented as f g2 = F copy (f g1 , num = h i × w i ) Step 4-3: f g2 is reshaped to the same dimensions h l x w i x c i i ​ f g3 = F re (f g2 , size = h i × w i × c i ) wherein F re represents a reshaping operation; Step 4-4: f g3 is connected with f l is connected f out = F concat (f l , f g3 ). 2.The method of claim 1, wherein, Each of the down-sampling layers is composed of a LeakyRelu layer, a two-dimensional convolution layer, and a batch normalization layer. 3.The method of claim 1, wherein, In the attention module, the input feature map is processed by a batch normalization layer and a 1x1 convolution, multiplied by a weight coefficient, then passed through a ReLU activation function, a 1x1 convolution layer, and a sigmoid activation function, and finally connected with the input feature map by layer skipping.

4. The method of claim 1, wherein the method is based on a conditional generative adversarial network. Based on the improved U-Net network for image restoration, it includes 8 layers of up-sampling and corresponds to the down-sampling layers.

5. The method of claim 4, wherein, Each of the up-sampling layers includes a ReLU layer, a bilinear up-sampling layer, a convolution layer, and a batch normalization layer. 6.The method of claim 1, wherein, The overall objective function of the model training loss is: Wherein, WGAN-GP, L1, L p , L TV are loss functions, λ1=10 -1 , λ2=10 -2 , λ3=10 -3 ; where x is the degraded underwater image, gtis the real underwater image with good details, For uniform sampling between the generated image G(x) and the real image gt, λ = 10.

7. An underwater image enhancement system based on a conditional generative adversarial network, comprising: a data set construction module for obtaining a paired image set of underwater degraded images and corresponding pure images, and dividing them into a training set and a test set; an image processing module for scaling all images to the same size; a model construction module, including: based on the encoder-decoder structure, extracting global and local features of the image; fusing the global features with the local features of each scale respectively; restoring the image by layer-by-layer upsampling of the global features, and connecting each upsampling layer with the fused features of the corresponding scale; inputting the generated image into the discriminator network to determine whether it comes from real data, prompting the generator network to adjust; a model training and testing module for inputting images into the model for training and testing, and saving the tested model; a model application module for processing actual underwater images using the tested model; The codec structure is an improved U-Net network, which includes 8 layers of down-sampling layers, and global and local feature extraction of an input image is performed through layer-by-layer convolution; In the down-sampling process, based on the SENet and the NAM module, the global average pooling module in the SENet is replaced by the batch normalization scaling factor in the NAM module to construct an attention module for underwater image enhancement; before the results of layer-by-layer up-sampling and the down-sampling results with the same resolution are connected, global and local feature fusion is performed, and the fusion process is: Step 4-1: The channel number c of the global feature f is adjusted to the same channel number c of the local feature map f at the corresponding scale i by a convolution layer with kernel size 1x1 and stride 1, which is denoted as g . g . l . i . Step 4-2: The local feature map f is upsampled to the corresponding scale i by a transposed convolution layer with kernel size 2x2 and stride 2, which is denoted as f g1 = F conv (f g , W) where F conv denotes a convolution operation, and W is a learnable weight. Step 4-2: f g1 is replicated h times i × w i , where h i and w i are the length and width of the local feature map f l of scale i, and the operation is denoted as f g2 = F copy (f g1 , num = h i × w i ) Step 4-3: f g2 is reshaped to the same dimensions h l x w i x c i i ​ f g3 = F re (f g2 , size = h i × w i × c i ) F = F + F re represents a reshaping operation; Step 4-4: f g3 is connected with f l is connected f out = F concat (f l , f g3 ).

Images