Underwater image enhancement method based on dual-path generative adversarial network
Through the brightness and image coloring network of the dual-path generative adversarial network, combined with multi-layer perceptual loss and gradient punishment, the problem of insufficient generalization ability of underwater image in the existing technology is solved, and high quality in extreme scenarios is achieved Image enhancement.
Patent Information
- Application Number
- CN202410690185.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-30
- Publication Date
- 2024-12-03
AI Technical Summary
The existing underwater image enhancement methods lack generalization capabilities when dealing with extreme scenes, and cannot consider image enhancement from multiple levels, resulting in limited improvement in visual quality.
Using a dual-path generative adversarial network method, the brightness and chromaticity of images are processed by the brightness enhancement network and the image coloring network respectively. Combined with the improved U-Net structure and PatchGAN discriminator, multi-layer perceptual loss and gradient punishment loss are used. Function to achieve multi-angle enhancement of images.
It improves the visual quality of underwater images, enhances the brightness and color of the image, reduces the amount of calculation, improves generalization ability, and can generate images that conform to human perception under insufficient lighting.
Smart Images

Figure CN119067897A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to an underwater image enhancement method based on a dual-path generative adversarial network. Background Art
[0002] Underwater image enhancement refers to optimizing the quality of underwater images through specific algorithms, making them clearer and more colorful, so as to facilitate subsequent analysis and recognition. The main challenge of underwater image enhancement lies in the special underwater environment, such as lighting conditions, water absorption and scattering effects, which will affect the quality of the image (contrast, clarity, saturation, brightness, etc.).
[0003] Underwater image enhancement methods can be roughly divided into three categories: image enhancement methods based on non-physical models, image restoration methods based on physical models, and image enhancement methods based on deep learning.
[0004] The non-physical model image enhancement method mainly improves the visual quality by adjusting the pixel values of the image without considering the physical process of underwater image degradation, such as image fusion, histogram equalization, etc. The advantage of this method is that it can quickly improve the image quality, but the disadvantage is that it may not completely solve the image degradation problem.
[0005] The image enhancement method based on physical models estimates the parameters in the physical model through prior knowledge, and infers the physical model to obtain a clear underwater image. For example, dark channel prior, minimum information prior, etc. The advantage of this method is that it can solve the image degradation problem more accurately, but the disadvantage is that it requires more computing resources and time.
[0006] Image enhancement methods based on deep learning enhance underwater degraded images by using deep learning techniques such as convolutional neural networks (CNNs) and generative adversarial networks (GANs). Currently, most underwater image enhancement methods based on deep learning belong to supervised learning. Supervised learning enhances new degraded images by learning the nonlinear relationship between degraded images and reference images. The effect of supervised learning depends on the design of data sets, loss functions, and neural network models. The advantage of this method is that it can greatly improve the clarity, contrast, and color balance of underwater images, reduce blur and distortion caused by underwater environments, and achieve end-to-end enhancement. The disadvantage is that existing methods often enhance images as a whole, without considering enhancement issues from multiple levels, resulting in weak generalization ability. When dealing with scenes with extreme conditions, the final enhancement effect is limited.
[0007] How to improve the visual quality of underwater images has become a technical problem that needs to be solved. Summary of the invention
[0008] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide an underwater image enhancement method based on a dual-path generative adversarial network.
[0009] The purpose of the present invention can be achieved by the following technical solutions:
[0010] According to one aspect of the present invention, a method for underwater image enhancement based on a dual-path generative adversarial network is provided, the method comprising the following steps:
[0011] Step S1, obtaining an underwater image dataset;
[0012] Step S2, converting the image in the underwater image data set in step S1 from the RGB color gamut to the CIELab color gamut, and dividing the image into a brightness image and a chromaticity image according to the channel;
[0013] Step S3, inputting the underwater image data set into the trained dual-path generative adversarial network to obtain an enhanced underwater image;
[0014] The dual-path generative adversarial network includes a parallel brightness enhancement network and an image colorization network, which are used for brightness enhancement and image colorization respectively.
[0015] Preferably, the brightness enhancement network includes a brightness map generator and a brightness map discriminator; the image colorization network includes a chromaticity map generator and a chromaticity map discriminator.
[0016] More preferably, the generator is an improved U-Net structure, including an encoder part and a decoder part.
[0017] More preferably, the encoder part has an n-layer structure, each layer comprising a feature extraction module and a downsampling module; the feature extraction module includes multiple hole convolution layers, a channel attention submodule is added before each hole convolution layer, and an instance normalization layer and a LeakyReLU activation layer are connected in sequence afterwards.
[0018] More preferably, the decoder part is an n-layer structure corresponding to the encoder, and each layer includes an upsampling module; the upsampling module includes a channel attention submodule, a transposed convolution layer, an instance normalization layer and a ReLU activation layer connected in sequence;
[0019] The same layers in the encoder and decoder are spliced using skip connections.
[0020] More preferably, the channel attention submodule is a channel attention submodule of a convolutional block attention mechanism.
[0021] More preferably, the training of the dual-path generative adversarial network uses three loss functions, namely, a chromaticity map generator loss function, a luminance map generator loss function and a discriminator loss function;
[0022] The formula of the discriminator loss function is as follows:
[0023] L(D)=D(x)-D(y)+λ1GP(x,y)
[0024] The formula of the brightness map generator loss function is as follows:
[0025] L(G L )=-λ2D(x)+λ3L SmoothL1 (x,y)+λ4L ssim (x,y)
[0026] The formula of the chromaticity map generator loss function is as follows:
[0027] L(G C )=-λ5D(x)+λ6L SmoothL1 (x,y)+λ7L per (x,y)
[0028] Among them, D is the discriminator, G L is the brightness map generator, G C is the chromaticity map generator, x is the enhanced image, y is the reference image, GP is the gradient penalty, L ssim is the structural similarity loss, L SmoothL1 is the SmoothL1 loss, L per is the multi-layer perceptual loss, and λ1~λ7 are hyperparameters.
[0029] More preferably, the discriminator is based on PatchGAN as a basic framework; the discriminator divides the image into multiple blocks and determines whether each block is true or false.
[0030] Preferably, the acquiring of the underwater image dataset comprises acquiring a paired underwater image dataset including an original image and a reference image, and an unpaired underwater image dataset including only the original image.
[0031] More preferably, the process of inputting the underwater image dataset into the trained dual-path generative adversarial network to obtain the enhanced underwater image comprises:
[0032] The original image brightness map, the original image boundary map and the inverse color of the brightness map of the underwater image dataset are Figure 1The brightness map generator is input, and the enhanced image brightness map is output after the image is reconstructed. Then the brightness map discriminator determines whether the enhanced image brightness map generated by the brightness map generator is real by learning the feature distribution of the reference image brightness map, and feeds back to the brightness map generator.
[0033] The original image brightness map and the original image chromaticity map are input into the chromaticity map generator, and the chromaticity map generator colors the original image brightness map to obtain the enhanced image chromaticity map. Then the chromaticity map discriminator discriminates whether the enhanced image chromaticity map generated by the chromaticity map generator is true according to the reference image chromaticity map, and feeds back to the chromaticity map generator;
[0034] Finally, the outputs of the two generators are fused to obtain the enhanced underwater image;
[0035] The inverse color map of the original image boundary map and the brightness map is used as an attention prior to guide the brightness map generator to perform local brightness enhancement.
[0036] Compared with the prior art, the present invention has the following beneficial effects:
[0037] 1) The present invention uses a dual-path generative adversarial network to perform brightness enhancement and colorization processing on underwater images respectively, enhances the images from multiple angles, and finally obtains enhanced underwater images through fusion and color gamut conversion processing, thereby improving the visual quality of underwater images.
[0038] 2) The encoder of the generator of the present invention uses continuous dilated convolution composed of dilated convolution layers to obtain a larger receptive field, better extract contextual information, reduce the amount of computation, promote multi-scale feature fusion, and combine with channel attention to improve the performance of the generator network; the discriminator uses the PatchGAN framework to better capture local textures and details, so that the dual-path generative adversarial network of the generator and the discriminator can reduce the amount of computation while improving the visual quality of underwater images.
[0039] 3) The same layers in the encoder and decoder of the present invention are spliced using skip connections, and feature maps of different levels are fused through skip connections, which promotes feature fusion and feature reuse, and retains certain details and textures.
[0040] 4) The generator of the present invention uses multi-layer perceptual loss to better constrain the network, making the brightness and color enhanced images more consistent with human perception of the massive amount of images; the discriminator loss function adds gradient penalty on the basis of WGAN (Wasserstein generative adversarial network) loss, making the network more stable during training and avoiding the problems of mode collapse and gradient disappearance. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1It is a schematic diagram of the process flow of the underwater image enhancement method based on the dual-path generative adversarial network in the present invention;
[0042] Figure 2 It is a schematic diagram of the structure of the dual-path generative adversarial network in the present invention;
[0043] Figure 3 It is a schematic diagram of the structure of the generator in the brightness enhancement network of the present invention;
[0044] Figure 4 It is a schematic diagram of the structure of the generator in the image colorization network of the present invention;
[0045] In the figure, Conv is a convolution layer, Dilation Conv is a hole convolution layer, IN is an instance normalization layer, LeakyReLU and ReLU are activation layers, TransposeConv is a transposed convolution, Channel Attention is a channel attention, and MaxPool is a maximum pooling layer. is the tensor element-wise multiplication, is tensor concatenation, B is the inverse color map of the original image brightness map, E is the boundary map of the original image, L is the original image brightness map, and C is the original image chromaticity map. DETAILED DESCRIPTION
[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present invention.
[0047] The purpose of the present invention is to improve the visual quality of underwater images. The underwater image enhancement method based on deep learning improves the generalization ability of processing underwater images of various scenes, improves the visual quality of underwater images from two aspects of brightness and chromaticity, and makes underwater images look more realistic. Even in the case of severe lack of light and relative darkness, this method can obtain an underwater image that conforms to human perception.
[0048] This embodiment relates to an underwater image enhancement method based on a dual-path generative adversarial network. Figure 1 , including the following technical steps:
[0049] Step S1, obtaining an underwater image dataset;
[0050] Step S2, converting the color gamut of the image in the underwater image dataset in step S1;
[0051] Convert the image from RGB color space to CIELab color space;
[0052] Step S3, constructing a dual-path generative adversarial network;
[0053] Step S4, training a dual-path generative adversarial network using the underwater image dataset of step S1;
[0054] Step S5, input the underwater image dataset into the trained dual-path generative adversarial network to obtain enhanced underwater images, and use evaluation indicators to evaluate the enhanced images.
[0055] Step S1 is specifically as follows: obtain a paired underwater image dataset containing original images and reference images and an unpaired underwater image dataset containing only original images, and divide the dataset into a training set and a test set. The underwater image dataset should include underwater images of different water quality, different lighting, different depths and other environmental conditions, and the images should contain a variety of underwater objects, including reefs, aquatic organisms, shipwrecks, etc. Mainstream paired underwater image datasets include EUVP, LSUI, UIEB, etc., and unpaired underwater image datasets include OceanDark, RUIE, etc.
[0056] In step S2, the original image and the reference image in the underwater image dataset are converted from the RGB color gamut to the CIELab color gamut. The image is converted into the CIELab color gamut using the cvtColor function provided by the OpenCV function library, and then the image is divided into a luminance map and a chromaticity map according to the L and ab channels using the split function and the merge function. By separating the luminance and chromaticity, the mutual influence between multiple channels can be avoided.
[0057] In step S3, the dual-path generative adversarial network is constructed by establishing two pairs of generators (brightness image generator and chromaticity image generator) and discriminators (brightness image discriminator and chromaticity image discriminator) to perform brightness enhancement and image colorization tasks respectively. The structure of the generator is an improved U-Net, and the structure of the discriminator is PatchGAN.
[0058] The main problems of underwater images are dimness and color distortion. Therefore, underwater image enhancement can also be divided into two tasks: brightness enhancement task and image colorization task. Brightness enhancement solves the problem of dimness, and image colorization solves the problem of color distortion. Especially in relatively dark scenes, the brightness is seriously insufficient and the color of the object is unknown. Through the present invention, while improving the brightness, the object can be given a suitable color through image colorization, achieving an enhancement effect that conforms to human perception. At the same time, the use of a generative adversarial network can make the images generated by the network clearer and more realistic.
[0059] like Figure 2As shown in the figure, a dual-path generative adversarial network is designed, which includes a brightness enhancement network and an image colorization network. The brightness enhancement network includes a brightness map generator and a brightness map discriminator, and the image colorization network includes a chromaticity map generator and a chromaticity map discriminator.
[0060] Figure 2 The upper half of the network is the brightness enhancement network. The brightness map of the original image is used as input, and the brightness map generator reconstructs the image output enhanced brightness map. The brightness map discriminator determines whether the enhanced image brightness map generated by the brightness map generator is real by learning the feature distribution of the reference image brightness map, and feeds it back to the generator.
[0061] The boundary map of the original image and the inverse color map of the brightness map are used as auxiliary information. The boundary map of the original image and the inverse color map of the brightness map can be used as attention priors to guide the brightness map generator to perform local brightness enhancement, that is, the degree of brightness enhancement varies according to the different brightness levels of different areas in the image. The boundary map of the original image is obtained by using the Canny operator on the original image, and the inverse color map of the brightness map is obtained by negating the brightness map of the original image. The calculation formula for brightness enhancement is as follows:
[0062] Output=L×B×(1+E) (1)
[0063] Where L is the brightness map of the original image, B is the inverse color map of the brightness map, and E is the boundary map of the original image.
[0064] Figure 2 The lower half of the network is the image colorization network. Image colorization is to fill the grayscale image with appropriate colors. The original image brightness map and the original image chromaticity map are used as input, and the chromaticity map generator colors the original image brightness map. The chromaticity map discriminator determines whether the enhanced image chromaticity map generated by the chromaticity map generator is real based on the reference image chromaticity map, and feeds it back to the generator.
[0065] Image colorization can be a classification task. The color gamut space of the output result is first quantized into 313 colors, and then the color value of each pixel in the image is predicted to realize image colorization. Because the original image chromaticity map can provide certain color information, the feature map of the original image chromaticity map is used as auxiliary information. By calculating the similarity matrix of the original image chromaticity map, the chromaticity map generator is helped to color the image. The similarity matrix calculation formula is as follows:
[0066] A=sigmoid(conv(F)) (2)
[0067] Where A is the similarity matrix and F is the feature map of the original image chromaticity map.
[0068] like Figure 3As shown in the figure, the brightness map generator for brightness enhancement is based on the U-Net framework. The generator is an encoder-decoder structure. The encoder is a contraction process that extracts features of images of different sizes by downsampling, while the decoder is an expansion process that gradually restores the image to its original size by upsampling.
[0069] The encoder has a 4-layer structure, each layer contains two modules, namely feature extraction and downsampling. Feature extraction is mainly composed of three dilated convolutional layers, the convolution kernel size of the three dilated convolutional layers is 3, the stride is 1, the padding is 1, 2, 3, and the dilation is 1, 2, 3. A channel attention module is added before each dilated convolutional layer, followed by an instance normalization layer and a LeakyReLU activation layer. Downsampling includes a maximum pooling layer with a kernel size of 2 and a stride of 2. Downsampling reduces the size of the image by half.
[0070] The decoder has a 4-layer structure, each of which contains an upsampling module. The upsampling includes a channel attention module, a transposed convolution layer with a convolution kernel of 4, a stride of 2, and a padding of 1, an instance normalization layer, and a ReLU activation layer. The upsampling doubles the size of the image.
[0071] The same layers in the encoder and decoder are spliced using skip connections. The feature maps at different levels are fused through skip connections to promote feature fusion and feature reuse, retaining certain details and textures.
[0072] At the input and output ends, a dilated convolution layer with a kernel size of 3, a stride of 1, a padding of 3, and a dilation of 3 is added. The module before the decoder is used to obtain more contextual information, and the module after the encoder is used to eliminate the checkerboard artifacts in the enhanced image.
[0073] Dilated convolution can obtain a larger receptive field than ordinary convolution when the convolution kernel size is the same, reducing the amount of calculation and promoting multi-scale feature fusion.
[0074] The channel attention mechanism uses the channel attention module in CBAM (Convolutional Block Attention Module). When the channel changes, the network can focus on important channels and improve the performance of the network.
[0075] like Figure 4 As shown in FIG. 1 , the structure of the chromaticity map generator for image colorization is basically similar to that of the brightness map generator for brightness enhancement. The difference is that the chromaticity map features are input into the third convolutional layer in the encoder to perform chromaticity information fusion to achieve auxiliary colorization.
[0076] The discriminator network uses PatchGAN as the basic framework. PatchGAN is a fully convolutional network that divides the image into multiple blocks and distinguishes the authenticity of each block. Compared with the discriminator of ordinary GAN, PatchGAN can better capture local textures and details.
[0077] Step S4 includes: designing a loss function and training a dual-path generative adversarial network using a training set of an underwater image dataset.
[0078] In a generative adversarial network, there is an adversarial game between the generator and the discriminator. When training a generative adversarial network, three loss functions need to be designed, namely the chromaticity image generator loss function, the luminance image generator loss function, and the discriminator loss function. The generator loss function mainly measures whether the enhanced image generated by the generator can deceive the discriminator and make the discriminator think that these images come from the reference image. The discriminator loss function mainly measures whether the discriminator can accurately distinguish between the image generated by the generator and the corresponding reference image.
[0079] The discriminator loss function (Equation 3), the brightness map generator loss function (Equation 4), and the chromaticity map generator loss function (Equation 5) are as follows:
[0080] L(D)=D(x)-D(y)+λ1GP(x,y) (3)
[0081] L(G L )=-λ2D(x)+λ3L smoothL1 (x,y)+λ4L ssim (x,y) (4)
[0082] L(G C )=-λ5D(x)+λ6L SmoothL1 (x,y)+λ7L per (x,y) (5)
[0083] Among them, D is the discriminator, G L is the brightness map generator, G C is the chromaticity map generator, x is the enhanced image, y is the reference image, GP is the gradient penalty, L ssim is the structural similarity loss, L SmoothL1 is the SmoothL1 loss, L per is the multi-layer perceptual loss, and λ1~λ7 are hyperparameters.
[0084] Both the chromaticity image discriminator and the brightness image discriminator use the discriminator loss function (Formula 3), which is the WGAN-GP (Wasserstein Generative Adversarial Network with Gradient Penalty) loss. This loss adds a gradient penalty to the WGAN (Wasserstein Generative Adversarial Network) loss, making the model more stable during training and avoiding the problems of mode collapse and gradient disappearance.
[0085] The brightness map generator loss function (Equation 4) is a combined loss function, including adversarial loss, SmoothL1 loss and structural similarity (SSIM) loss.
[0086] SmoothL1 loss is more robust than L2 loss and improves the problem of non-smooth zero points in L1 loss.
[0087] Structural similarity loss L ssim Function, which measures the difference in structural similarity between two knot images from the changes in image brightness, contrast and structure.
[0088] The chromaticity map generator loss function (Equation 5) is also a combined loss function, including adversarial loss, SmoothL1 loss and multi-layer perceptual loss.
[0089] Multi-layer perceptual loss L per The loss function is obtained by taking features of one or more layers from the pre-trained VGG19 and calculating the MSE (mean square error) loss. Compared with the MSE loss, the multi-layer perceptual loss pays more attention to the perceptual quality of the image, so that the image conforms to the human eye's perception of image quality. In order to take into account both the integrity and local details of the image, the features of the 2nd and 30th layers in VGG19 are selected to calculate the loss.
[0090] During training, both the generator and the discriminator use the Adam optimizer, the learning rate is set to 1e-4, and betas is set to (0.5, 0.999).
[0091] Step 5: Input the test set in the lab color domain underwater image dataset into the dual-path generative adversarial network to obtain the enhanced image.
[0092] (1) The degraded underwater image is converted into Lab color domain and divided into brightness map and chromaticity map according to the channel.
[0093] (2) The chromaticity map and the brightness map are input into the corresponding generators, and the outputs of the two generators are merged to obtain the enhanced image.
[0094] (3) Convert the enhanced image back into the RGB color space.
[0095] The enhanced image needs to be evaluated, and the present invention uses the following evaluation indicators, namely, peak signal-to-noise ratio (PSNR), structural similarity (SSIM) and underwater color image quality (UCIQE), underwater image quality measurement (UIQM) and underwater image fidelity (UIF).
[0096] Peak signal-to-noise ratio (PSNR) is an objective evaluation index for evaluating image quality. It evaluates the quality of an image by comparing the difference between the original image and the distorted image.
[0097] The Structural Similarity Index (SSIM) is a measure of the similarity between two images, which takes into account changes in image brightness, contrast, and structure. The larger the value, the more similar the two images are in structure and the better the quality.
[0098] Underwater color image quality (UCIQE) comprehensively considers the color measurement index (UICM), the clarity measurement index (UISM) and the contrast measurement index (UIConM), and finally linearly combines the three values to obtain UCIQE. The larger the value, the more the hue, saturation and contrast of the image are in line with human visual preferences, that is, the higher the image quality.
[0099] The underwater image quality measurement (UIQM) is a linear combination of the standardized chroma, saturation and contrast indicators to obtain the UIQM value. The larger the value, the better the image performance in terms of color balance, clarity and contrast, that is, the higher the image quality.
[0100] Underwater image fidelity (UIF) comprehensively evaluates the naturalness, sharpness, and structure index of an image. These indexes are combined with a saliency-based spatial pooling strategy to obtain the final UIF metric. The larger the UIF value, the better the underwater image enhancement effect, the higher the image quality, and the more in line with the requirements of human visual perception.
[0101] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. An underwater image enhancement method based on a dual-path generative adversarial network, characterized in that: The method comprises the following steps: Step S1, obtaining an underwater image dataset; Step S2, converting the image in the underwater image data set in step S1 from the RGB color gamut to the CIELab color gamut, and dividing the image into a brightness image and a chromaticity image according to the channel; Step S3, inputting the underwater image data set into the trained dual-path generative adversarial network to obtain an enhanced underwater image; The dual-path generative adversarial network includes a parallel brightness enhancement network and an image colorization network, which are used for brightness enhancement and image colorization respectively.
2. The underwater image enhancement method based on a dual-path generative adversarial network according to claim 1, characterized in that: The brightness enhancement network includes a brightness map generator and a brightness map discriminator; the image colorization network includes a chromaticity map generator and a chromaticity map discriminator.
3. The underwater image enhancement method based on a dual-path generative adversarial network according to claim 2, characterized in that: The generator is an improved U-Net structure, including an encoder part and a decoder part.
4. The underwater image enhancement method based on a dual-path generative adversarial network according to claim 3, characterized in that: The encoder part has an n-layer structure, each layer includes a feature extraction module and a downsampling module; the feature extraction module includes multiple hole convolution layers, each hole convolution layer is preceded by a channel attention submodule, and is followed by an instance normalization layer and a LeakyReLU activation layer.
5. The underwater image enhancement method based on a dual-path generative adversarial network according to claim 3, characterized in that: The decoder part is an n-layer structure corresponding to the encoder, and each layer includes an upsampling module; the upsampling module includes a channel attention submodule, a transposed convolution layer, an instance normalization layer, and a ReLU activation layer connected in sequence; The same layers in the encoder and decoder are spliced using skip connections.
6. The underwater image enhancement method based on a dual-path generative adversarial network according to claim 4, characterized in that: The channel attention submodule is a channel attention submodule of the convolutional block attention mechanism.
7. The underwater image enhancement method based on a dual-path generative adversarial network according to claim 2, characterized in that: The training of the dual-path generative adversarial network uses three loss functions, namely, a chromaticity map generator loss function, a luminance map generator loss function, and a discriminator loss function; The formula of the discriminator loss function is as follows: L(D)=D(x)-D(y)+λ1GP(x,y) The formula of the brightness map generator loss function is as follows: L(G L )=-λ2D(x)+λ3L SmoothL1 (x,y)+λ4L ssim (x,y) The formula of the chromaticity map generator loss function is as follows: L(G C )=-λ5D(x)+λ6L SmoothL1 (x,y)+λ7L per (x,y) Among them, D is the discriminator, G L is the brightness map generator, G C is the chromaticity map generator, x is the enhanced image, y is the reference image, GP is the gradient penalty, L ssim is the structural similarity loss, L SmoothL1 is the SmoothL1 loss, L per is the multi-layer perceptual loss, and λ1~λ7 are hyperparameters.
8. The underwater image enhancement method based on a dual-path generative adversarial network according to claim 2, characterized in that: The discriminator is based on PatchGAN as a basic framework; the discriminator divides the image into multiple blocks and determines whether each block is true or false.
9. The underwater image enhancement method based on a dual-path generative adversarial network according to claim 1, characterized in that: The acquiring of the underwater image data set includes acquiring a paired underwater image data set including an original image and a reference image, and an unpaired underwater image data set including only the original image.
10. The underwater image enhancement method based on a dual-path generative adversarial network according to claim 9, characterized in that: The process of inputting the underwater image dataset into the trained dual-path generative adversarial network to obtain the enhanced underwater image includes: The original image brightness map, the original image boundary map and the inverse color map of the brightness map of the underwater image dataset are input into the brightness map generator together, and the enhanced image brightness map is output after the image is reconstructed. Then the brightness map discriminator judges whether the enhanced image brightness map generated by the brightness map generator is true by learning the feature distribution of the reference image brightness map, and feeds back to the brightness map generator; The original image brightness map and the original image chromaticity map are input into the chromaticity map generator, and the chromaticity map generator colors the original image brightness map to obtain the enhanced image chromaticity map. Then the chromaticity map discriminator discriminates whether the enhanced image chromaticity map generated by the chromaticity map generator is true according to the reference image chromaticity map, and feeds back to the chromaticity map generator; Finally, the outputs of the two generators are fused to obtain the enhanced underwater image; The inverse color map of the original image boundary map and the brightness map is used as an attention prior to guide the brightness map generator to perform local brightness enhancement.
Citation Information
Cited By
Target identification method and system, electronic equipment and storage medium
CN119785007A
Book inside page stain detection and repair suggestion system, method and device and storage medium
CN122530027A
