Underwater image enhancement method based on physical guidance generative adversarial network

Abstract

The invention belongs to the technical field of computer vision and image enhancement, and particularly relates to an underwater image enhancement method based on a physical guidance generative adversarial network, which performs enhancement processing on a degraded image by constructing the physical guidance generative adversarial network, and comprises the following steps: S1, constructing a physical parameter estimation module based on an underwater degradation physical model, performing inversion on the underwater degradation physical model to obtain a physical enhanced underwater image Jc '(x), and extracting features through convolution and pooling operation; s2, performing multiple times of down-sampling on the underwater image by constructing double encoders to obtain multi-level features of the underwater image; and S3, by constructing a cross-branch attention module, carrying out dynamic interaction on the multi-level features of the double encoders. According to the method, a traditional physical model is combined with deep learning, the interpretability of the model is improved, the feature utilization rate of the model is remarkably enhanced through the cross-branch attention module by utilizing physical guidance constraint, and the detail information of the generated enhanced image is enriched.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image enhancement technology, specifically to an underwater image enhancement method based on a physics-guided generative adversarial network. Background Technology

[0002] Underwater image enhancement refers to optimizing the quality of underwater images using specific algorithms to make them clearer and more vibrant, thus facilitating subsequent analysis and identification. The main challenge of underwater image enhancement lies in the unique underwater environment, where factors such as lighting conditions and water absorption and scattering effects can all affect image quality (contrast, sharpness, saturation, brightness, etc.).

[0003] Underwater image enhancement methods can be broadly classified into three categories: non-physical model image enhancement methods, physical model-based image restoration methods, and deep learning-based image enhancement methods.

[0004] Non-physical model-based image enhancement methods primarily improve visual quality by adjusting pixel values ​​without considering the physical processes of underwater image degradation. Examples include image fusion and histogram equalization. The advantage of this approach is its ability to quickly improve image quality, but its disadvantage is that it may not completely resolve image degradation issues.

[0005] Physical model-based image enhancement methods estimate parameters in a physical model using prior knowledge and then reverse-engineer the model to obtain a clear underwater image. Examples include dark channel priors and minimum information priors. The advantage of this method is that it can more accurately address image degradation issues, but the disadvantage is that it requires more computational resources and time.

[0006] Deep learning-based image enhancement methods utilize deep learning techniques such as Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs) to enhance degraded underwater images. Currently, most deep learning-based underwater image enhancement methods fall under supervised learning. Supervised learning enhances new degraded images by learning the non-linear relationship between the degraded image and a reference image. The effectiveness of supervised learning depends on the dataset, loss function, and the design of the neural network model. The advantages of this approach are its ability to significantly improve the sharpness, contrast, and color balance of underwater images, reduce blurring and distortion caused by the underwater environment, and achieve end-to-end enhancement. The disadvantages are that existing methods often enhance the image holistically without considering the enhancement problem at multiple levels, resulting in poor generalization ability. In extreme scenarios, the final enhancement effect is limited.

[0007] Improving the visual quality of underwater images and addressing issues such as color distortion, loss of detail, and noise interference in complex underwater scenes have become key technical challenges. Summary of the Invention

[0008] (a) Technical problems to be solved

[0009] To address the shortcomings of existing technologies, this invention provides an underwater image enhancement method based on a physics-guided generative adversarial network, which solves the problems mentioned in the background section.

[0010] (II) Technical Solution

[0011] To achieve the above objectives, the present invention specifically adopts the following technical solution:

[0012] An underwater image enhancement method based on a physics-guided generative adversarial network (GAN) enhances degraded images by constructing a GAN, comprising the following steps:

[0013] S1. Based on the underwater degradation physical model, a physical parameter estimation module is constructed. Physically enhanced underwater images J are obtained by inverting the underwater degradation physical model. c '(x), and extract features through convolution and pooling operations;

[0014] S2. By constructing a dual encoder, the underwater image is downsampled multiple times to obtain multi-level features of the underwater image;

[0015] S3. By constructing a cross-branch attention module, the multi-level features of the dual encoders are dynamically interacted to obtain attention-weighted image features;

[0016] S4. Through the upsampling module and the skip connection module, the multi-level features are fused and reconstructed with the corresponding image features to generate the enhanced underwater image J.

[0017] S5. Construct a dual discriminator network. By comparing the image differences between the enhanced underwater image and the corresponding high-quality underwater image in the dataset, determine whether the quality of the generated image can fool the discriminator. Discriminator 1 uses the patch discriminator, and discriminator 2 uses the ResNet34 network.

[0018] S6. Construct a multivariate loss function and use the loss function to train the underwater image enhancement model.

[0019] Furthermore, in S1: based on the underwater degradation physical model, a physical parameter estimation module is constructed and the model is inverted to obtain the physically enhanced underwater image J. c Then, feature extraction is performed on '(x), specifically including:

[0020] S11. Physical modeling of underwater degraded images can be expressed as:

[0021] I c (x)=J c (x)tc (x)+B c (1-t c (x))

[0022] Where c∈{r,g,b} are color channels; I c (x) represents the observed image; J c (x) represents the restoration of a clear image; B c Background scattered light; t c (x) represents the transmittance;

[0023] S12. According to the underwater degradation physical model, the transmittance can be expressed as:

[0024] t c (x)=e -βd

[0025] Where e is an exponential function, β is a wavelength-dependent attenuation coefficient, and d is the scene depth, and a physical parameter estimation module is constructed based on this.

[0026] S13. Inversion of the underwater model does not consider the influence of background scattered light, resulting in a physically enhanced underwater image J. c '(x) can be represented as:

[0027]

[0028] S14. Physically enhance the underwater image J from S13 using 7×7 convolution and max pooling operations. c '(x) performs feature extraction.

[0029] Furthermore, the physical parameter estimation module includes an attenuation coefficient estimator, a scene depth estimator, and a transmittance estimator, specifically including:

[0030] S121. The implementation process of the attenuation coefficient estimator is as follows: each color channel of the original underwater image is input into a convolutional block to generate three attenuation coefficients, the formula of which can be expressed as:

[0031] β c =linear(relu(linear(conv.pr(I c (x)))))

[0032] Where I c (x) is the observed image, c∈{r,g,b} represents the color channels, conv.pr contains two convolutional layers with a kernel size of 3×3, followed by a pooling layer and a ReLU activation function, and linear represents a linear layer. Then, these three channels are concatenated as the final attenuation coefficient:

[0033] β = cat(β) r ,β g ,β b )

[0034] Where cat represents concatenation along the channel dimension;

[0035] S122. The specific implementation process of the scene depth estimator is to directly estimate the scene depth d through RBD blocks, which can be expressed as:

[0036] d=σ(conv(conv.br(RBD(conv.br(I c (x))))))

[0037] Where conv.br represents a convolutional layer with a kernel size of 3×3, followed by a normalization layer and a ReLU activation function, σ represents the sigmoid function, conv represents a convolutional layer, RBD represents an RBD block, and I c (x) is the observed image;

[0038] S123. From the formula for transmittance in S12, it can be seen that transmittance has an exponential relationship with the product of scene depth and attenuation coefficient. Therefore, the transmittance estimator can be expressed as:

[0039] t=σ(conv(conv.br(d·β)))

[0040] Where conv.br represents a convolutional layer with a kernel size of 3×3, followed by a normalization layer and a ReLU activation function, σ represents the sigmoid function, conv represents the convolutional layer, and (d·β) represents the input.

[0041] Furthermore, in step S2, multi-level features of the underwater image are obtained through dual encoders. The implementation of the dual encoders specifically includes:

[0042] The implementation process of encoder 1 in S21 is as follows: multi-level feature extraction operations are performed on the feature map obtained in S14 through depthwise separable convolutional blocks and spectral attention module SA.

[0043] The implementation process of S22 and encoder 2 is as follows: the observed image I is processed through the ResNet network architecture. c (x) Performs multi-level feature extraction operations, and in the last layer, retains more global features and detailed information of the feature map through multi-scale residuals and CBAM attention module.

[0044] Furthermore, the spectral attention module and depthwise separable convolutional block of each layer of encoder 1 in S21 are implemented through the following steps:

[0045] S211. Global average pooling is used to convert the global spatial information of each channel of the feature map obtained in S14 into channel information.

[0046] S212. Attention weights are obtained through 1×1 convolution and activation functions ReLU and sigmoid.

[0047] S213. Multiply the input with the attention weights element by element through residual connections to fuse feature information from different channels and obtain the enhanced features.

[0048] S214. Further enhance the feature map using depthwise separable convolution and pointwise convolution;

[0049] S215. By downsampling the feature map through max pooling, the dimensionality of the image is reduced, the number of image channels is increased, and deeper features are extracted.

[0050] Furthermore, the feature extraction operation of encoder 2 in S22 is implemented through the following steps:

[0051] S221. The observed image I described in S11 is processed by a 7×7 convolution operation and a max pooling operation. c (x) performs feature extraction;

[0052] S222. Using the common [3, 4, 6, 3] hierarchical structure of the ResNet network architecture, fill the identity mapping residual block and the downsampled convolution mapping residual block to further extract the feature map;

[0053] S223. Further enhancement processing is performed on the feature map extracted in S222 using multi-scale residual blocks and CBAM attention module.

[0054] Furthermore, in step S3, the cross-branch attention module dynamically interacts with the multi-level features of the dual encoders and uses the features of encoder 1 to guide and enhance the features of encoder 2. The specific implementation steps are as follows:

[0055] S31. Associate the features of the dual encoders as input to construct a Query-Key-Value triple; use the sliding window attention mechanism to guide the weight adjustment of the feature branch of encoder 2 by using the feature branch of encoder 1 as the query.

[0056] S32. Enhance the response of important channels by using the channel attention module, and use residual connections to retain the original feature information while avoiding gradient vanishing.

[0057] Furthermore, in step S31, the Query-Key-Value triple can be represented as:

[0058] Q = WQ *B1

[0059] K = W K *B2

[0060] V = W V *B2

[0061] Among them W Q W K W V It is a 1×1 convolutional block, B1 is the input of branch one, and B2 is the input of branch two;

[0062] The processing method of the sliding window attention mechanism can be expressed as follows:

[0063] W(x)=Unfold(x,k=w,p=w / 2)

[0064]

[0065] Where w is the window size, set to 7, W(·) center This is the query vector that takes only the center position of the window; d k The scaling factor for the key is set to Channel / 8, where T represents transpose.

[0066] Furthermore, in S4, the fusion and reconstruction of multi-level features through the upsampling module and the skip connection module are specifically achieved through the following steps:

[0067] S41. By using skip connections, the output of the corresponding feature extraction stage is fused with the output of the upsampling feature reconstruction stage step by step.

[0068] S42. Upsample the final output features of the feature extraction stage described in S223, and input the output to the next-level upsampling module;

[0069] S43. The dimension of the feature map is restored by transposing the convolution and the skip connection module, and finally the generator-enhanced underwater image J is generated.

[0070] Furthermore, in S6, the underwater image enhancement model is trained using a multivariate loss function, including global-local adversarial loss, global similarity loss L1, and perceptual loss.

[0071] The formula for the global-local adversarial loss function is as follows:

[0072] Discriminator 1 Local Adversarial Loss Function L patch It can be represented as:

[0073]

[0074] in, This represents a minimax game, where G represents the generator and D1 represents discriminator 1. Let X represent the mathematical expectation, Y represent the source domain (i.e., the low-quality underwater image to be observed), Z represent the target domain (i.e., the high-quality underwater image), and Z represent random noise.

[0075] Discriminator 2 Global Adversarial Loss Function L resnet34 It can be represented as:

[0076]

[0077] in, Let G represent a minimax game, D2 represent the generator, X represent the source domain (low-quality underwater image to be observed), Y represent the target domain (high-quality underwater image), and Z represent random noise. and Let ||·||² represent the mathematical expectation, ||·||² represent the L2 norm, and λ be the gradient penalty coefficient. For random interpolated samples, Indicates input The gradient;

[0078] The global similarity loss function L1(G) can be expressed as:

[0079]

[0080] Where G represents the generator, X is the source domain (low-quality underwater image to be observed), Y is the target domain (high-quality underwater image), and Z is random noise. Let ||·||1 represent the mathematical expectation, and let L1 norm be ||·||1.

[0081] The perceptual loss function L con It can be represented as:

[0082]

[0083] Where G represents the generator, X is the source domain (low-quality underwater image to be observed), Y is the target domain (high-quality underwater image), and Z is random noise. Let Φ represent the expected value. Φ() represents the pre-trained feature extraction network, and ||·||2 represents the L2 norm.

[0084] (III) Beneficial Effects

[0085] Compared with existing technologies, this invention provides an underwater image enhancement method based on a physics-guided generative adversarial network, which has the following beneficial effects:

[0086] This invention improves the interpretability of the model by combining traditional physical models with deep learning, and significantly enhances the feature utilization of the model by using physical guidance constraints through cross-branch attention modules, thus enriching the detailed information of the generated enhanced images.

[0087] The image encoder of the generator in this invention is designed with a multi-scale residual + CBAM module in the last layer. By combining multi-scale contextual information with a dynamic weighting mechanism, it enhances the preservation of key features while reducing information loss, thereby improving the robustness of the model to complex scenes.

[0088] This invention constructs a dual discriminator structure in the discriminator part of the network, which comprehensively evaluates the generated image from both local and global dimensions. This ensures that the enhancement process does not distort or lose the main objects and scene structures in the image, and also better captures high-frequency textures and details, avoiding the blurring problem caused by global discrimination.

[0089] This invention constructs a multivariate loss function to provide more comprehensive constraints and feedback on the generated images of the generator and the discrimination performance of the discriminator, thereby improving the robustness of network training, avoiding the gradient vanishing problem during training, and allowing the generator to retain more texture information of the original image during the generation process. Attached Figure Description

[0090] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0091] Figure 2 This is a schematic diagram of the dual-stream generator in this invention;

[0092] Figure 3 This is a diagram of the basic module PB of the physical encoder 1 in this invention;

[0093] Figure 4 This is a schematic diagram of the overall structure of the encoder 2 image encoder in this invention;

[0094] Figure 5 This is a CBA diagram of the cross-branch attention module in the generator section of this invention;

[0095] Figure 6 This is a schematic diagram of the generator decoder structure in this invention. Detailed Implementation

[0096] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0097] Example

[0098] like Figure 1-6 As shown in the figure, an embodiment of the present invention proposes an underwater image enhancement method based on a physics-guided generative adversarial network. The method includes constructing the physics-guided generative adversarial network by designing a physical parameter estimation module using a hybrid approach of physical models and deep learning, and designing a cross-branch attention mechanism to achieve dynamic feature interaction between the physical feature extraction module and the image feature extraction module. Through the collaborative optimization of multi-scale physical constraints and data features, the feature utilization rate of the model is significantly enhanced, making the generated images more resistant to visual interference caused by the underwater environment. For the discriminator, a dual-criteria discriminator is used to collaboratively evaluate the images generated by the generator, which not only improves the quality and efficiency of the next generation of images but also makes the generated images closer to high-quality images.

[0099] An underwater image enhancement method based on a physics-guided generative adversarial network (GAN) enhances degraded images by constructing a GAN, comprising the following steps:

[0100] S1. Based on the underwater degradation physical model, a physical parameter estimation module is constructed. Physically enhanced underwater images J are obtained by inverting the underwater degradation physical model. c '(x), and extract features through convolution and pooling operations;

[0101] Based on the underwater degradation physical model, a physical parameter estimation module is constructed and the model is inverted to obtain the physically enhanced underwater image J. c Then, feature extraction is performed on '(x), specifically including:

[0102] S11. Physical modeling of underwater degraded images can be expressed as:

[0103] I c (x)=J c (x)t c (x)+B c (1-t c (x))

[0104] Where c∈{r,g,b} are color channels; I c (x) represents the observed image; J c (x) represents the restoration of a clear image; B c Background scattered light; t c (x) represents the transmittance;

[0105] S12. According to the underwater degradation physical model, the transmittance can be expressed as:

[0106] t c (x)=e-βd

[0107] Where β is the wavelength-dependent attenuation coefficient, d is the scene depth, and a physical parameter estimation module is constructed based on this, and e is an exponential function;

[0108] The physical parameter estimation module includes an attenuation coefficient estimator, a scene depth estimator, and a transmittance estimator, specifically comprising:

[0109] S121. The implementation process of the attenuation coefficient estimator is as follows: each color channel of the original underwater image is input into a convolutional block to generate three attenuation coefficients, the formula of which can be expressed as:

[0110] β c =linear(relu(linear(conv.pr(I c (x)))))

[0111] Where I c (x) is the observed image, c∈{r,g,b} represents the color channels, conv.pr contains two convolutional layers with a kernel size of 3×3, followed by a pooling layer and a ReLU activation function, and linear represents a linear layer. Then, these three channels are concatenated as the final attenuation coefficient:

[0112] β = cat(β) r ,β g ,β b )

[0113] Where cat represents concatenation along the channel dimension;

[0114] S122. The specific implementation process of the scene depth estimator is to directly estimate the scene depth d through RBD blocks, which can be expressed as:

[0115] d=σ(conv(conv.br(RBD(conv.br(I c (x))))))

[0116] Where conv.br represents a convolutional layer with a kernel size of 3×3, followed by a normalization layer and a ReLU activation function, σ represents the sigmoid function, conv represents a convolutional layer, RBD represents an RBD block, and I c (x) is the observed image;

[0117] S123. From the formula for transmittance in S12, it can be seen that transmittance has an exponential relationship with the product of scene depth and attenuation coefficient. Therefore, the transmittance estimator can be expressed as:

[0118] t=σ(conv(conv.br(d·β)))

[0119] Where conv.br represents a convolutional layer with a kernel size of 3×3, followed by a normalization layer and a ReLU activation function, σ represents the sigmoid function, conv represents the convolutional layer, and (d·β) represents the input.

[0120] S13. Inversion of the underwater model does not consider the influence of background scattered light, resulting in a physically enhanced underwater image J. c '(x) can be represented as:

[0121]

[0122] S14. Physically enhance the underwater image J from S13 using 7×7 convolution and max pooling operations. c '(x) performs feature extraction.

[0123] S2. By constructing a dual encoder to downsample the underwater image multiple times, multi-level features of the underwater image are obtained. The implementation of the dual encoder specifically includes:

[0124] The implementation process of encoder 1 in S21 is as follows: multi-level feature extraction operations are performed on the feature map obtained in S14 through depthwise separable convolutional blocks and spectral attention module SA.

[0125] The spectral attention module and depthwise separable convolutional block of each layer of encoder 1 are implemented through the following steps:

[0126] S211. Global average pooling is used to convert the global spatial information of each channel of the feature map obtained in S14 into channel information.

[0127] S212. Attention weights are obtained through 1×1 convolution and activation functions ReLU and sigmoid.

[0128] S213. Multiply the input with the attention weights element by element through residual connections to fuse feature information from different channels and obtain the enhanced features.

[0129] S214. Further enhance the feature map using depthwise separable convolution and pointwise convolution;

[0130] S215. By downsampling the feature map through max pooling, the dimensionality of the image is reduced, the number of image channels is increased, and deeper features are extracted.

[0131] The implementation process of S22 and encoder 2 is as follows: the observed image I is processed through the ResNet network architecture. c (x) Performs multi-level feature extraction operations, and in the last layer, retains more global features and detailed information of the feature map through multi-scale residuals and CBAM attention module.

[0132] The feature extraction operation of encoder 2 is achieved through the following steps:

[0133] S221. The observed image I described in S11 is processed by a 7×7 convolution operation and a max pooling operation. c (x) performs feature extraction;

[0134] S222. Using the common [3, 4, 6, 3] hierarchical structure of the ResNet network architecture, fill the identity mapping residual block and the downsampled convolution mapping residual block to further extract the feature map;

[0135] S223. Further enhancement processing is performed on the feature map extracted in S222 using multi-scale residual blocks and CBAM attention module.

[0136] S3. By constructing a cross-branch attention module, the multi-level features of the dual encoders are dynamically interacted to obtain attention-weighted image features;

[0137] The cross-branch attention module dynamically interacts with the multi-level features of the dual encoders and uses the features of encoder 1 to guide and enhance the features of encoder 2. The specific implementation steps are as follows:

[0138] S31. Associate the features of the dual encoders as input to construct a Query-Key-Value triple; use the sliding window attention mechanism to guide the weight adjustment of the feature branch of encoder 2 by using the feature branch of encoder 1 as the query.

[0139] The Query-Key-Value triplet can be represented as:

[0140] Q = W Q *B1

[0141] K = W K *B2

[0142] V = W V *B2

[0143] Among them W Q W K W V It is a 1×1 convolution block, B1 is the input of branch one, and B2 is the input of branch two.

[0144] The sliding window attention mechanism can be represented as follows:

[0145] W(x)=Unfold(x,k=w,p=w / 2)

[0146]

[0147] Where w is the window size, set to 7, W(·) center This is the query vector that takes only the center position of the window; d k The scaling factor for the key is set to Channel / 8, where T represents transpose.

[0148] S32. Enhance the response of important channels by using the channel attention module, and use residual connections to retain the original feature information while avoiding gradient vanishing.

[0149] S4. Through the upsampling module and the skip connection module, the multi-level features are fused and reconstructed with the corresponding image features to generate the enhanced underwater image J.

[0150] The multi-level feature fusion and reconstruction through the upsampling module and the skip connection module are specifically achieved through the following steps:

[0151] S41. By using skip connections, the output of the corresponding feature extraction stage is fused with the output of the upsampling feature reconstruction stage step by step.

[0152] S42. Upsample the final output features of the feature extraction stage described in S223, and input the output to the next-level upsampling module;

[0153] S43. The dimension of the feature map is restored by transposing the convolution and the skip connection module, and finally the generator-enhanced underwater image J is generated.

[0154] like Figure 2 As shown, the dual-stream generator is based on the U-net architecture. The generator is an encoder-decoder structure. The encoder performs downsampling for multi-level feature extraction, while the decoder performs upsampling to reconstruct the image and restore it to its initial resolution. A cross-branch attention module is added between the two encoders to guide and enhance encoder 2 from encoder 1. A skip connection Res Path is added between encoder 2 and the decoder to retain more original image information and enrich the details of the image reconstructed by the decoder.

[0155] Encoder 1 is a physical encoder, and the first layer is a physical parameter estimation module that estimates the observed image I. c (x) is inverted into a physically enhanced underwater image J through a physical model. c '(x) uses the same convolution and pooling operations as the first layer of ResNet for feature extraction to ensure the same feature dimension as the corresponding layer of encoder 2. Layers 2 to 5 are each spectral attention modules and depthwise separable modules, such as...' Figure 3 As shown, it includes a convolutional layer (CONV), a normalization layer (BN), an activation function layer (ReLU), and a pooling layer (Pool).

[0156] Encoder 2 is an image encoder, such as Figure 4 As shown, using the ResNet architecture, the first convolutional layer has a kernel size of 7, followed by a max pooling layer. Layers 2 to 5 use a combination of identity mapping residual blocks and downsampling mapping residual blocks in a hierarchical structure of [3,4,6,3]. The last layer, SMC, is a multi-scale residual with CBAM attention mechanism. The feature maps are passed through parallel branches of identity mapping residual blocks with kernel sizes of 1, 3, and 5, respectively. The extracted multi-scale features are then concatenated using the torch.cat function. Next, 1×1 convolutions are used to reduce the dimensionality of the channels to restore the number of input channels. Finally, the output is passed through the CBAM module and superimposed on the input to retain more global features and detailed information.

[0157] A cross-branch attention module (CBA) was designed between encoder 1 and encoder 2, such as... Figure 5 As shown, after each downsampling, the feature map of encoder 1 is dynamically interacted with the corresponding feature map of encoder 2 through a cross-branch attention mechanism. Physical information is used to generate guiding constraints for image feature extraction. After aligning and stitching the two features, the physical features are used to supplement and enhance the image features through the attention mechanism, and the enhanced image features are used as the input for the next layer of downsampling.

[0158] The decoder has a 5-layer structure, such as Figure 6 As shown, each layer contains a transposed convolutional layer, a normalization layer (BN), and an activation function layer (ReLU). Each layer upsamples the image, gradually restoring it to its initial resolution, thus obtaining the enhanced image J generated by the generator.

[0159] S5. Construct a dual discriminator network. By comparing the image differences between the enhanced underwater image and the corresponding high-quality underwater image in the dataset, determine whether the quality of the generated image can fool the discriminator. Discriminator 1 uses the patch discriminator, and discriminator 2 uses the ResNet34 network.

[0160] The difference between Discriminator 1 and traditional discriminators is that it makes local discrimination rather than global judgment. It can better capture high-frequency textures and details and avoid the blurring problem caused by global discrimination. It is very suitable for image enhancement, a scenario that requires improving the quality of details.

[0161] Discriminator 2 is chosen from a global perspective. In order to balance the strength relationship between the generator and the discriminator and avoid one being too strong and the other too weak to engage in a game, thus affecting the stability of training and the robustness of the model, ResNet-34 is selected as the discriminator 2 to correspond with the structure of the generator.

[0162] S6. Construct a multivariate loss function and use the loss function to train the underwater image enhancement model;

[0163] The underwater image enhancement model is trained using a multivariate loss function, including global-local adversarial loss, global similarity loss L1, and perceptual loss.

[0164] The formula for the global-local adversarial loss function is as follows:

[0165] Discriminator 1 Local Adversarial Loss Function L patch It can be represented as:

[0166]

[0167] in, Let G represent a minimax game, D1 represent the generator, X represent the source domain (low-quality underwater image to be observed), Y represent the target domain (high-quality underwater image), and Z represent random noise. It represents the mathematical expectation.

[0168] Discriminator 2 Global Adversarial Loss Function L resnet34 It can be represented as:

[0169]

[0170] in, Let G represent a minimax game, D2 represent the generator, X represent the source domain (low-quality underwater image to be observed), Y represent the target domain (high-quality underwater image), and Z represent random noise. and Let ||·||² represent the mathematical expectation, ||·||² represent the L2 norm, and λ be the gradient penalty coefficient. For random interpolated samples, Indicates input The gradient.

[0171] The global similarity loss function L1(G) can be expressed as:

[0172]

[0173] Where G represents the generator, X is the source domain (low-quality underwater image to be observed), Y is the target domain (high-quality underwater image), and Z is random noise. Let ||·||1 represent the mathematical expectation, and let L1 norm be ||·||1.

[0174] The perceptual loss function L con It can be represented as:

[0175]

[0176] Where G represents the generator, X is the source domain (low-quality underwater image to be observed), Y is the target domain (high-quality underwater image), and Z is random noise. Let Φ represent the expected value. Φ() represents the pre-trained feature extraction network, and ||·||2 represents the L2 norm.

[0177] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An underwater image enhancement method based on a physics-guided generative adversarial network, characterized in that, a physics-guided generative adversarial network is constructed to enhance degraded images, and the method is further characterized in that, Includes the following steps: S1. Based on the underwater degradation physical model, a physical parameter estimation module is constructed. Physically enhanced underwater images J are obtained by inverting the underwater degradation physical model. c '(x), and extract features through convolution and pooling operations; S2. By constructing a dual encoder, the underwater image is downsampled multiple times to obtain multi-level features of the underwater image; S3. By constructing a cross-branch attention module, the multi-level features of the dual encoders are dynamically interacted to obtain attention-weighted image features; S4. Through the upsampling module and the skip connection module, the multi-level features are fused and reconstructed with the corresponding image features to generate the enhanced underwater image J. S5. Construct a dual discriminator network. By comparing the image differences between the enhanced underwater image and the corresponding high-quality underwater image in the dataset, determine whether the quality of the generated image can fool the discriminator. Discriminator 1 uses the patch discriminator, and discriminator 2 uses the ResNet34 network. S6. Construct a multivariate loss function and use the loss function to train the underwater image enhancement model.

2. The underwater image enhancement method based on a physics-guided generative adversarial network according to claim 1, characterized in that: In S1: Based on the underwater degradation physical model, a physical parameter estimation module is constructed and the model is inverted to obtain the physically enhanced underwater image J. c Then, feature extraction is performed on '(x), specifically including: S11. Physical modeling of underwater degraded images can be expressed as: I c (x)=J c (x)t c (x)+B c (1-t c (x)) Where c∈{r,g,b} are color channels; I c (x) represents the observed image; J c (x) represents the restoration of a clear image; B c Background scattered light; t c (x) represents the transmittance; S12. According to the underwater degradation physical model, the transmittance can be expressed as: t c (x)=e -βd Where e is an exponential function, β is a wavelength-dependent attenuation coefficient, and d is the scene depth, and a physical parameter estimation module is constructed based on this. S13. Inversion of the underwater model does not consider the influence of background scattered light, resulting in a physically enhanced underwater image J. c '(x) can be represented as: S14. Physically enhance the underwater image J from S13 using 7×7 convolution and max pooling operations. c '(x) performs feature extraction.

3. The underwater image enhancement method based on a physics-guided generative adversarial network according to claim 2, characterized in that: The physical parameter estimation module includes an attenuation coefficient estimator, a scene depth estimator, and a transmittance estimator, specifically including: S121. The implementation process of the attenuation coefficient estimator is as follows: each color channel of the original underwater image is input into a convolutional block to generate three attenuation coefficients, the formula of which can be expressed as: β c =linear(relu(linear(conv.p.r(I c (x))))) Among them I c (x) is the observed image, c∈{r,g,b} represents the color channels, conv.pr contains two convolutional layers with a kernel size of 3×3, followed by a pooling layer and a ReLU activation function, and linear represents a linear layer. Then, these three channels are concatenated as the final attenuation coefficient: β=cat(β r ,b g ,b b ) Where cat represents concatenation along the channel dimension; S122. The specific implementation process of the scene depth estimator is to directly estimate the scene depth d through RBD blocks, which can be expressed as: d=σ(conv(conv.b.r(RBD(conv.b.r(I c (x)))))) Where conv.br represents a convolutional layer with a kernel size of 3×3, followed by a normalization layer and a ReLU activation function, σ represents the sigmoid function, conv represents a convolutional layer, RBD represents an RBD block, and I c (x) is the observed image; S123. From the formula for transmittance in S12, it can be seen that transmittance has an exponential relationship with the product of scene depth and attenuation coefficient. Therefore, the transmittance estimator can be expressed as: t=σ(conv(conv.br(d·β))) Where conv.br represents a convolutional layer with a kernel size of 3×3, followed by a normalization layer and a ReLU activation function, σ represents the sigmoid function, conv represents the convolutional layer, and (d·β) represents the input.

4. The underwater image enhancement method based on a physics-guided generative adversarial network according to claim 1, characterized in that: In step S2, multi-level features of the underwater image are obtained through a dual encoder. The implementation of the dual encoder specifically includes: The implementation process of encoder 1 in S21 is as follows: multi-level feature extraction operations are performed on the feature map obtained in S14 through depthwise separable convolutional blocks and spectral attention module SA. The implementation process of S22 and encoder 2 is as follows: the observed image I is processed through the ResNet network architecture. c (x) Performs multi-level feature extraction operations, and in the last layer, retains more global features and detailed information of the feature map through multi-scale residuals and CBAM attention module.

5. The underwater image enhancement method based on a physics-guided generative adversarial network according to claim 4, characterized in that: The spectral attention module and depthwise separable convolutional block of each layer of encoder 1 in S21 are implemented through the following steps: S211. Global average pooling is used to convert the global spatial information of each channel of the feature map obtained in S14 into channel information. S212. Attention weights are obtained through 1×1 convolution and activation functions ReLU and sigmoid. S213. Multiply the input with the attention weights element by element through residual connections to fuse feature information from different channels and obtain the enhanced features. S214. Further enhance the feature map using depthwise separable convolution and pointwise convolution; S215. By downsampling the feature map through max pooling, the dimensionality of the image is reduced, the number of image channels is increased, and deeper features are extracted.

6. The underwater image enhancement method based on a physics-guided generative adversarial network according to claim 4, characterized in that: The feature extraction operation of encoder 2 in S22 is achieved through the following steps: S221. The observed image I described in S11 is processed by a 7×7 convolution operation and a max pooling operation. c (x) performs feature extraction; S222. Using the common [3, 4, 6, 3] hierarchical structure of the ResNet network architecture, fill the identity mapping residual block and the downsampled convolution mapping residual block to further extract the feature map; S223. Further enhancement processing is performed on the feature map extracted in S222 using multi-scale residual blocks and CBAM attention module.

7. The underwater image enhancement method based on a physics-guided generative adversarial network according to claim 1, characterized in that: In step S3, the cross-branch attention module dynamically interacts the multi-level features of the dual encoders and uses the features of encoder 1 to guide and enhance the features of encoder 2. The specific implementation steps are as follows: S31. Associate the features of the dual encoders as input to construct a Query-Key-Value triple; use the sliding window attention mechanism to guide the weight adjustment of the feature branch of encoder 2 by using the feature branch of encoder 1 as the query. S32. Enhance the response of important channels by using the channel attention module, and use residual connections to retain the original feature information while avoiding gradient vanishing.

8. The underwater image enhancement method based on a physics-guided generative adversarial network according to claim 7, characterized in that: In step S31, the Query-Key-Value triple can be represented as: Q=W Q *B1 K=W K *B2 V=W V *B2 Among them W Q W K W V It is a 1×1 convolutional block, B1 is the input of branch one, and B2 is the input of branch two; The processing method of the sliding window attention mechanism can be expressed as follows: W(x)=Unfold(x,k=w,p=w / 2) Where w is the window size, set to 7, W(·) center This is a query vector that only takes the center position of the window; d k The scaling factor for the key is set to Channel / 8, where T represents transpose.

9. The underwater image enhancement method based on a physics-guided generative adversarial network according to claim 1, characterized in that: In step S4, the fusion and reconstruction of multi-level features through the upsampling module and the skip connection module are specifically achieved through the following steps: S41. By using skip connections, the output of the corresponding feature extraction stage is fused with the output of the upsampling feature reconstruction stage step by step. S42. Upsample the final output features of the feature extraction stage described in S223, and input the output to the next-level upsampling module; S43. The dimension of the feature map is restored by transposing the convolution and the skip connection module, and finally the generator-enhanced underwater image J is generated.

10. The underwater image enhancement method based on a physics-guided generative adversarial network according to claim 1, characterized in that: In step S6, the underwater image enhancement model is trained using a multivariate loss function, including global-local adversarial loss, global similarity loss L1, and perceptual loss. The formula for the global-local adversarial loss function is as follows: Discriminator 1 Local Adversarial Loss Function L patch It can be represented as: in, This represents a minimax game, where G represents the generator and D1 represents discriminator 1. Let X represent the mathematical expectation, Y represent the source domain (i.e., the low-quality underwater image to be observed), Z represent the target domain (i.e., the high-quality underwater image), and Z represent random noise. Discriminator 2 Global Adversarial Loss Function L resnet34 It can be represented as: in, Let G represent a minimax game, D2 represent the generator, X represent the source domain (low-quality underwater image to be observed), Y represent the target domain (high-quality underwater image), and Z represent random noise. and Let ||·||² represent the mathematical expectation, ||·||² represent the L2 norm, and λ be the gradient penalty coefficient. For random interpolated samples, Indicates input The gradient; The global similarity loss function L1(G) can be expressed as: Where G represents the generator, X is the source domain (low-quality underwater image to be observed), Y is the target domain (high-quality underwater image), and Z is random noise. Let ||·||1 represent the mathematical expectation, and let L1 norm be ||·||1. The perceptual loss function L con It can be represented as: Where G represents the generator, X is the source domain (low-quality underwater image to be observed), Y is the target domain (high-quality underwater image), and Z is random noise. Let Φ represent the expected value. Φ() represents the pre-trained feature extraction network, and ||·||2 represents the L2 norm.

Images