Underwater image cross-domain enhancement method based on unsupervised UCycleGAN

Abstract

The invention discloses an underwater image cross-domain enhancement method based on an unsupervised UCycleGAN. The underwater image cross-domain enhancement method comprises the following steps: constructing an improved U-shaped generator; inputting the degraded underwater image into a U-shaped generator, and outputting a forged clear image; sending the real clear image and the forged clear image into a discriminator together, and calculating an adversarial loss through a discrimination probability value; calculating to obtain perception loss; carrying out weighted summation on the adversarial loss, the perception loss, the cyclic consistency loss and the identity loss according to weights to form a final composite loss function; and inputting the underwater image into the trained U-shaped generator to obtain an enhanced clear image. According to the invention, the U-shaped generator is designed, so that the extraction capability of the generator on underwater image features and the processing capability on details can be enhanced. According to the invention, the U-shaped generator is designed, so that the extraction capability of the generator on underwater image features and the processing capability on details can be enhanced.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to a cross-domain enhancement method for underwater images based on unsupervised UCYcleGAN. Background Technology

[0002] Underwater images often suffer from low contrast, color distortion, and blurriness due to scattering and absorption by water, severely impacting applications such as underwater target recognition and underwater robot navigation. While some underwater image enhancement methods exist, they generally suffer from the following problems: First, they require large amounts of labeled data for supervised learning, but labeling underwater images is costly and difficult to obtain; second, existing enhancement methods often fail to adapt well to different underwater environments and image types, exhibiting limited generalization ability; and third, they lack effective cross-domain conversion capabilities, making it difficult to transfer image styles from one underwater environment to another, thus limiting the application of underwater images in various scenarios.

[0003] While existing CycleGANs support unsupervised transformations, their standard generator structure is simple and lacks prior knowledge of underwater physics, resulting in limited detail recovery capabilities. Furthermore, the lack of constraints on underwater physics degradation models during cross-domain transformations easily leads to artifacts and detail distortion in the enhanced images. Additionally, a single loss function is insufficient to simultaneously optimize color correction, contrast enhancement, and texture restoration. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a simple algorithm with strong cross-domain transformation capabilities for underwater image cross-domain enhancement based on unsupervised UCYcleGAN.

[0005] The technical solution of this invention to solve the above-mentioned technical problems is: an underwater image cross-domain enhancement method based on unsupervised UCYcleGAN, comprising the following steps:

[0006] Step 1: Constructing the improved U-shaped generator Pyra-Unet: Pyra-Unet includes encoder downsampling, decoder upsampling, bottleneck layer, and multi-scale spatial pyramid attention module MSPA embedded in skip connections;

[0007] Step 2: Input the degraded underwater image into Pyra-Unet, and Pyra-Unet outputs a fake, clear image;

[0008] Step 3: Input a real clear image from the target domain clear image dataset and a fake clear image output by Pyra-Unet into the discriminator. The discriminator outputs a discrimination probability value, and the adversarial loss is calculated from the discrimination probability value.

[0009] Step 4: Input the real clear image and the fake clear image into the pre-trained deep convolutional neural network VGG respectively, extract the corresponding high-dimensional feature maps respectively, and calculate the perceptual loss between the two high-dimensional feature maps.

[0010] Step 5: Weight the adversarial loss, perception loss, cycle consistency loss and identity loss to form the final composite loss function. Update the parameters of Pyra-Unet and the discriminator through backpropagation so that Pyra-Unet minimizes the total loss and the discriminator maximizes its discrimination ability.

[0011] Step 6: Repeat steps 1 to 5 until Pyra-Unet and the discriminator reach Nash equilibrium, thus obtaining the trained Pyra-Unet. Input the underwater image into the trained Pyra-Unet to obtain the enhanced and clear image.

[0012] The above-mentioned underwater image cross-domain enhancement method based on unsupervised UCYcleGAN includes a three-layer downsampling encoder in step one. The first downsampling layer includes a first convolutional layer, a first batch normalization layer, a first ReLU layer, and a first residual block connected in sequence. BN represents batch normalization processing, and ReLU represents the ReLU activation function. The second downsampling layer has the same structure as the first downsampling layer. The third downsampling layer includes a second convolutional layer, a second batch normalization layer, and a second ReLU layer connected in sequence.

[0013] The above-mentioned underwater image cross-domain enhancement method based on unsupervised UCYcleGAN, in step one, the bottleneck layer includes a dilated convolution group, a convolution stacking unit, a dilated convolution operation unit, and a feature concatenation unit. The dilated convolution group employs four levels of cascaded convolution layers. The convolution stacking unit includes dilated convolution, batch normalization, and an activation function. The formula for dilated convolution operation in the dilated convolution operation unit is as follows:

[0014]

[0015] in, Indicates location, Indicates the input feature map, Indicates position The output after dilated convolution operation, For expansion rate, This represents the sequence number of the convolutional layer in the dilated convolution group. Indicates the first The weights of each convolutional layer.

[0016] In the above-mentioned underwater image cross-domain enhancement method based on unsupervised UCYcleGAN, in step one, the decoder upsampling is a three-layer upsampling, the first layer upsampling corresponds to the feature map of the third layer downsampling, and the first layer upsampling includes the first upsampling, the first skip connection, and the first convolution;

[0017] In the first upsampling, the downsampling is performed after four dilated convolutions;

[0018] In the first hop connection, the features from the hop connection after the third MSPA layer are concatenated with the output of the third MSPA layer, and multi-scale features are weighted and fused through pyramid pooling and channel attention.

[0019] The second upsampling layer corresponds to the feature map of the second downsampling layer. The second upsampling layer includes the second upsampling, the second skip connection, and the second convolution.

[0020] In the second hop connection, the features from the hop connection after the second MSPA layer are concatenated with the output of the second MSPA layer.

[0021] The third upsampling layer corresponds to the feature map of the first downsampling layer. The third upsampling layer includes the third upsampling, the third skip connection, and the third convolution.

[0022] In the third hop connection, the features from the hop connection after the first MSPA layer are concatenated with the output of the first MSPA layer.

[0023] The above-mentioned underwater image cross-domain enhancement method based on unsupervised UCYcleGAN, in step one, the multi-scale spatial pyramid attention module MSPA includes a hierarchical phantom convolution module HPC and a spatial pyramid recalibration module SPR;

[0024] The hierarchical phantom convolutional module (HPC) enhances spatial information through multi-scale feature extraction, performs channel dimension segmentation on the input feature map, and generates... Each subset is processed through a hierarchical residual convolutional chain to fuse features from different receptive fields;

[0025] The hierarchical phantom convolutional module (HPC) includes splitting units, convolutional hierarchy chains, and splicing units.

[0026] The splitting unit uniformly splits the input feature map along the channel dimension into There are subsets, and each subset has channels. ;

[0027] The convolutional hierarchy uses convolutional groups with hierarchical residual connections. The first group directly processes a subset; in subsequent groups, the current subset is added to the output of the previous group and then convolved.

[0028] The splicing unit splices all subset outputs along the channel dimension to form multi-scale enhanced features.

[0029] In the above-mentioned underwater image cross-domain enhancement method based on unsupervised UCYcleGAN, the spatial pyramid recalibration module SPR in step one adopts a two-component collaborative mechanism, including a spatial pyramid aggregation block and a channel interaction block.

[0030] The spatial pyramid aggregation blocks employ dual-path pooling, namely global path pooling and local path pooling; after dual-path pooling, adaptive fusion and weighted summation are performed to obtain the feature map. ;

[0031] Channel Interaction Block Received Feature Map The initial channel attention weight vector is computed through a lightweight two-layer 1×1 convolutional bottleneck structure; the first layer 1×1 convolution will... Number of channels Compress to Then ReLU activation is applied, introducing non-linearity; the second 1×1 convolution reduces the number of channels from... Restore to .

[0032] In the above-mentioned underwater image cross-domain enhancement method based on unsupervised UCYcleGAN, step three involves adversarial loss. The calculation formula is:

[0033]

[0034] in, Expressing expectations, Indicates the discriminator, Represents a generator. For the input image, This represents the probability value output by the discriminator after evaluating the image generated by the generator.

[0035] In the above-mentioned underwater image cross-domain enhancement method based on unsupervised UCYcleGAN, step four involves perceptual loss. The calculation formula is:

[0036]

[0037] in, Indicates correspondence The weighting coefficients, Indicates correspondence The weighting coefficients, Indicates correspondence The weighting coefficients, Indicates peak signal-to-noise ratio. Represents a quality index. Represents a correlation indicator.

[0038] In the above-mentioned underwater image cross-domain enhancement method based on unsupervised UCYcleGAN, step five involves cycle consistency loss. The calculation formula is:

[0039]

[0040] in, Represents the feature extraction function. express Norm;

[0041] Loss of identity The calculation formula is:

[0042]

[0043] in, Represents the target domain image.

[0044] In the above-mentioned underwater image cross-domain enhancement method based on unsupervised UCYcleGAN, step five involves a composite loss function. The calculation formula is:

[0045]

[0046] in, To counteract the weight of loss, Weights for perceiving loss, The weights for the cycle consistency loss, Weighting for identity loss.

[0047] The beneficial effects of this invention are as follows:

[0048] 1. This invention designs a U-shaped generator, Pyra-Unet: Pyra-Unet includes a multi-scale spatial pyramid attention module (MSPA) embedded in the encoder downsampling, decoder upsampling, bottleneck layer, and skip connections to enhance the generator's ability to extract underwater image features and process details. The U-shaped structure can effectively utilize the multi-scale features of the image. The enhancement mechanism based on this is to insert MSPA into the skip connection path after downsampling at each layer of the encoder, replacing the traditional feature concatenation. MSPA includes a hierarchical phantom convolution module (HPC) and a spatial pyramid recalibration module (SPR), which can enhance the generator's robustness to underwater image color distortion and scattering noise, and strengthen the attention weights of key features (such as edges and textures). The bottleneck layer is designed with four layers of dilated convolution stacked (with the dilation rate increasing stepwise, such as 1 / 2 / 4 / 8), and each convolution kernel is 3×3. This can further improve the generator's ability to expand the receptive field without reducing the resolution, capture long-range contextual dependencies in underwater images (such as the distribution of suspended particles and the light attenuation pattern), and suppress local artifacts. At the same time, the residual block design further ensures the stable propagation of gradients and the feature reuse capability of the network.

[0049] 2. This invention utilizes VGG for feature evaluation to achieve unsupervised cross-domain transformation of underwater images. The features obtained through VGG pre-training can effectively evaluate the differences in image features under different underwater environments, thereby achieving image style transformation from one underwater environment to another in an unsupervised manner, greatly improving the applicability and generalization ability of underwater images in different scenarios.

[0050] 3. This invention designs a composite loss function, including adversarial loss, perceptual loss, cycle consistency loss, and identity loss. The adversarial loss is used to ensure the authenticity of the generated image; the perceptual loss is used to optimize the peak signal-to-noise ratio, quality index, and relevance of the image, thereby improving the visual quality of the image; the cycle consistency loss is used to ensure the consistency between the generated image and the input image at the feature level; and the identity loss is used to maintain the invariance of the image content. Through the comprehensive optimization of these four losses, high-quality enhancement of underwater images can be achieved, while maintaining the semantic information and detailed features of the image during the enhancement process. Attached Figure Description

[0051] Figure 1 This is the overall flowchart of the present invention.

[0052] Figure 2 This is a flowchart of the workflow of the U-shaped generator Pyra-Unet of the present invention.

[0053] Figure 3 This is a graph showing the peak signal-to-noise ratio (PSNR) of the method of this invention compared with other methods on a dataset.

[0054] Figure 4 This is a graph showing the structural similarity (SSIM) index between the method of this invention and other methods on the dataset. Detailed Implementation

[0055] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0056] like Figure 1 As shown, the underwater image cross-domain enhancement method based on unsupervised UCYcleGAN includes the following steps:

[0057] Step 1: Constructing the improved U-shaped generator Pyra-Unet: Pyra-Unet includes encoder downsampling, decoder upsampling, a bottleneck layer, and a multi-scale spatial pyramid attention module (MSPA) embedded in skip connections. U represents the constructed U-shaped generator Pyra-Unet, Cycle represents the cyclic consistency mechanism, and GAN refers to the generative adversarial network framework. Therefore, UCycleGAN accurately summarizes the core architecture and technical features of this invention, namely, a cyclic consistency generative adversarial network based on a U-shaped generator.

[0058] like Figure 2 As shown, Figure 2 This paper visually illustrates the core workflow of the improved U-shaped generator Pyra-Unet, particularly the internal information exchange between the generator G and the discriminator D, and the iterative reconstruction process. The following is a detailed explanation based on the graph:

[0059] A: Represents the input image for domain A (e.g., a blurred underwater image domain).

[0060] B: Represents the input image of domain B (e.g., a clear reference image domain).

[0061] G AtoB : Represents a generator that transforms an image from domain A to domain B, and its function is to convert an image from domain A into an image in the style of domain B.

[0062] G BtoA : Represents a generator that transforms an image from domain B to domain A, and its function is to convert an image from domain B into an image in the style of domain A.

[0063] D A : Represents the discriminator for domain A, whose function is to determine whether the input image is a true image of domain A or G. BtoA A forged image of domain A.

[0064] D B : Represents the B-domain discriminator, whose function is to determine whether the input image is a true B-domain image or a G-domain image. AtoB Forged domain B image.

[0065] : Represents a forged domain A style image.

[0066] : Represents a forged domain B style image.

[0067] Decision probability [0, 1]: represents the discriminator (D A Or D B The output of ) is a value between 0 and 1, representing the probability that the discriminator considers the input image to be a real image (1 means it is very certain that it is a real image, and 0 means it is very certain that it is a generated image).

[0068] A clear bidirectional generative and adversarial framework ensures the effectiveness of unsupervised learning. It constrains the network to learn content-fidelity and reversible inter-domain mappings through cycle consistency loss, fundamentally preventing mode collapse and ensuring that key information of the original image is preserved during the transformation process.

[0069] The encoder downsampling consists of three layers. The first downsampling layer includes a first convolutional layer, a first batch normalization layer, a first ReLU layer, and a first residual block connected in sequence. BN represents batch normalization, and ReLU represents the ReLU activation function.

[0070] The first convolutional layer receives the original input tensor and performs initial feature projection using a 7×7 large kernel convolution (stride 1, padding 3). The padding mode is reflection padding to avoid boundary artifacts. The input is an underwater image (e.g., 512×512×3), and the output is 512×512×64. This allows for a large receptive field to capture low-frequency degradation features (e.g., fog-like blur).

[0071] The first batch normalization layer: performs batch normalization on the 64-channel feature maps to suppress illumination inhomogeneities. It then performs affine transformations on the convolutional output, scaling and shifting it.

[0072] First ReLU layer: Introduces nonlinearity, preserves positive phase characteristics, and ensures that the signal variance is conserved after ReLU activation.

[0073] First residual block: Establishes an identity mapping between input and output:

[0074]

[0075] The design of the first residual block can alleviate gradient vanishing in deep networks and preserve high-frequency detail features. For the input feature map, and These represent the first and second weights, respectively; BN indicates batch normalization; and ReLU indicates the ReLU activation function. Represents residual mapping, This represents the final output of the residual block; for The results are batch normalized by subtracting the batch mean and dividing by the batch standard deviation to standardize the activation values ​​of the intermediate layers. Learnable scaling and translation parameters are also typically introduced, which helps stabilize the training process, accelerate convergence, and reduce sensitivity to initialization. The ReLU function sets all negative values ​​to zero and retains positive values, introducing non-linearity into the network and enabling it to learn more complex patterns. The first convolutional stride in the first residual block is 1, and the stride of the next layer is also 1, with zero-padding used to maintain the feature map size. Subsequent skip connections are fed into the first MSPA layer (optimizing high-frequency details for decoder fusion).

[0076] The second-layer downsampling has the same structure as the first-layer downsampling.

[0077] The third downsampling layer consists of a second convolutional layer, a second batch normalization (BN) layer, and a second ReLU layer connected in sequence.

[0078] Second convolutional layer: 3×3 convolution (stride 1, padding 1), channels increased to 256, output 256×256×256.

[0079] Second BN layer: accelerates cross-domain data convergence.

[0080] The second ReLU layer activates high-level semantic features. The sampling path is: input to a four-level dilated convolutional layer to simulate underwater light scattering and capture multi-scale global context. The skip connections after this are input to the third MSPA layer.

[0081] The bottleneck layer includes dilated convolutional groups, stacked convolutional units, dilated convolutional units, and feature concatenation units. The dilated convolutional groups employ four cascaded convolutional layers with dilation rates of 1, 2, 4, and 8. Layer output receptive field size for: ,in For the first The dilation rate of each convolutional layer To indicate the first The kernel size of each convolutional layer This represents the sequence number of the convolutional layer in the dilated convolution group. For the first Layer output receptive field size.

[0082] The convolution stacked unit contains dilated convolution, batch normalization, and activation functions.

[0083] The formula for dilated convolution in the dilated convolution operation unit is:

[0084]

[0085] in, Indicates location, Indicates the input feature map, Indicates position The output after dilated convolution operation, For expansion rate, This represents the sequence number of the convolutional layer in the dilated convolution group. Indicates the first The weights of each convolutional layer.

[0086] The decoder recovers a sharp image through feature reconstruction and cross-level fusion. During decoder upsampling, MSPA is applied to skip connection paths, replacing traditional feature stitching.

[0087] During the upsampling process at each layer, the decoder integrates complementary information from the encoder through skip connections and has made a key improvement by using MSPA instead of the traditional direct feature concatenation. The specific operation is as follows:

[0088] For the deeper decoder (the first layer) The output feature map of the current layer (the first layer) is upsampled to make its spatial size the same as that of the current layer (the second layer). The deep decoder features are obtained by matching the encoder feature maps to be fused (layer 1). Simultaneously, the feature maps symmetrically corresponding to the current decoding layer in the encoder are extracted and input into MSPA for processing. MSPA captures global context dependencies at different ranges through multi-scale spatial pyramid pooling, effectively enhancing the feature response capability for small targets, reconstructing skip features, suppressing noise transmitted between the encoder and decoder, and highlighting effective signals valuable for segmentation and reconstruction. The upsampled deep decoder features and the MSPA-enhanced encoder features are concatenated along the channel dimension. A 1×1 convolution is applied to the concatenated final feature map to achieve effective information fusion and channel number adjustment along the channel dimension, ultimately outputting the feature map of the current decoding layer.

[0089] The decoder upsampling consists of three upsampling layers. The first upsampling layer corresponds to the feature map of the third downsampling layer. The first upsampling layer includes the first upsampling, the first skip connection, and the first convolution.

[0090] In the first upsampling, downsampling is performed after four dilated convolutions; the output of the dilated convolution is then transposed and convolved (kernel 3×3, stride 2), doubling the resolution to 256×256×128.

[0091] In the first hop connection, the features from the hop connection after the third MSPA layer are concatenated with the output of the third MSPA layer, and multi-scale features are weighted and fused through pyramid pooling and channel attention.

[0092] The first convolution operation involves residual blocks and transposed convolution. The residual blocks are placed before the transposed convolution to avoid amplifying high-frequency noise; the residual blocks are used to refine the fused features (output 256×256×128); the transposed convolution parameters are: kernel size 4×4, stride 2, padding 1; the transposed convolution is used for secondary upsampling to 512×512×64 (preparing for the next layer of fusion).

[0093] The second upsampling layer corresponds to the feature map of the second downsampling layer. The second upsampling layer includes the second upsampling, the second skip connection, and the second convolution.

[0094] Second upsampling: transposed convolution (kernel 3×3, stride 2), output 512×512×64.

[0095] In the second hop connection, the features from the hop connection after the second MSPA layer are concatenated with the output of the second MSPA layer.

[0096] The second convolution also operates on a residual block and a transposed convolution.

[0097] The third upsampling layer corresponds to the feature map of the first downsampling layer. The third upsampling layer includes the third upsampling, the third skip connection, and the third convolution.

[0098] In the third hop connection, the features from the hop connection after the first MSPA layer are concatenated with the output of the first MSPA layer to fuse the high-frequency textures of the original image (such as coral details).

[0099] The third convolution: operates on a convolutional layer plus a residual block, adding a 3×3 convolution beforehand: aligning the skip features through channels, and finally following a 1×1 convolution to output the result, mapping the convolution channels to 3.

[0100] The Multi-Scale Spatial Pyramid Attention Module (MSPA) includes the Hierarchical Phantom Convolution Module (HPC) and the Spatial Pyramid Recalibration Module (SPR).

[0101] The hierarchical phantom convolutional module (HPC) enhances spatial information through multi-scale feature extraction, performs channel dimension segmentation on the input feature map, and generates... Each subset is processed through a hierarchical residual convolutional chain. The structure generates multi-scale receptive fields (small receptive fields capture texture, large receptive fields capture global structure) through a combined explosion effect, adapting to underwater blur / uneven lighting problems. The input channels are split into... Each subset is connected by hierarchical residuals to fuse features from different receptive fields;

[0102] The hierarchical phantom convolutional module (HPC) includes splitting units, convolutional hierarchy chains, and splicing units.

[0103] The splitting unit uniformly splits the input feature map along the channel dimension into There are subsets, and each subset has channels. ;

[0104] The convolutional hierarchy uses a set of convolutional groups with hierarchical residual connections. The first group directly processes a subset; in subsequent groups, the current subset is added to the output of the previous group and then convolved. This can produce a combinatorial explosion effect, where the output contains a combination of different receptive fields.

[0105] The splicing unit concatenates all subsets of the output along the channel dimension to form multi-scale enhanced features, achieving multi-scale coverage. The output includes... A combination of receptive fields is used to address underwater multi-scale degradation issues (small receptive fields enhance underwater particulate details, while large receptive fields improve object recognition in dimly lit backgrounds).

[0106] The Space Pyramid Recalibration Module (SPR) employs a two-component collaborative mechanism, including a Space Pyramid aggregation block and a channel interaction block.

[0107] The spatial pyramid aggregation block employs dual-path pooling, namely global path pooling and local path pooling. , These represent the height and width of the input feature map, respectively.

[0108] Global path pooling: 1×1 average pooling → compressed to vector;

[0109] Local path pooling: 2×2 average pooling → output Feature map;

[0110] After dual-path pooling, adaptive fusion and weighted summation are performed to obtain the feature map. ;

[0111] Channel Interaction Block Received Feature Map The initial channel attention weight vector is computed through a lightweight two-layer 1×1 convolutional bottleneck structure; the first layer 1×1 convolution will... Number of channels Compress to Then ReLU activation is applied, introducing non-linearity; the second 1×1 convolution reduces the number of channels from... Restore to .

[0112] Step 2: Input the degraded underwater image into Pyra-Unet, and Pyra-Unet will output a fake, clear image.

[0113] Step 3: Input a real clear image from the target domain clear image dataset and a fake clear image output by Pyra-Unet into the discriminator. The discriminator outputs a discrimination probability value, and the adversarial loss is calculated from the discrimination probability value.

[0114] The adversarial loss from the discriminator is the fundamental driving force of GANs, achieved through a game between the generator G and the discriminator D. The generator's goal is to generate realistic, clear images so that the discriminator cannot distinguish its output from real, clear images. The discriminator's goal is to identify real images (output probability → 1) and generated images (output probability → 0) as accurately as possible.

[0115] Combat losses The calculation formula is:

[0116]

[0117] in, Expressing expectations, Indicates the discriminator, Represents a generator. For the input image, This represents the probability value output by the discriminator after evaluating the image generated by the generator.

[0118] Step 4: Input the real clear image and the fake clear image into the pre-trained deep convolutional neural network VGG respectively, extract the corresponding high-dimensional feature maps, and calculate the perceptual loss between the two high-dimensional feature maps.

[0119] Using VGG as a feature extractor: The generated image and the real, sharp image are input into VGG. High-dimensional feature maps are extracted in deep convolutional layers of VGG (such as ReLU3_3). The image is aligned in the semantic feature space, rather than the pixel space, focusing on improving semantic quality and guiding the generator to learn high-level visual features of the sharp image (such as object edges, texture structure, and semantic content), significantly improving detail clarity and naturalness. It also suppresses artifacts: compared to pixel-level loss, it is robust to image translation and minor shape changes, reducing local distortion.

[0120] Perceived loss The calculation formula is:

[0121]

[0122] in, Indicates correspondence The weighting coefficients, Indicates correspondence The weighting coefficients, Indicates correspondence The weighting coefficients, Indicates peak signal-to-noise ratio. Represents a quality index. Represents a correlation indicator.

[0123] Step 5: Weight the adversarial loss, perception loss, cycle consistency loss and identity loss according to their weights and sum them to form the final composite loss function. Update the parameters of Pyra-Unet and the discriminator through backpropagation so that Pyra-Unet minimizes the total loss and the discriminator maximizes its discrimination ability.

[0124] Cycle consistency loss ensures that when the generated clear image is converted back to an underwater image using another generator (or the inverse process of the same generator), it approximates the original input underwater image. Introducing an inverse generator allows simultaneous learning of the mappings from Underwater to Clear (degraded underwater image → clear image) and from Clear to Underwater (clear image → degraded underwater image), expanding application scenarios and constructing a closed loop. It also maintains content consistency: forcing the input image and the cyclically reconstructed image to be consistent in content structure (e.g., object position and shape remain unchanged), preventing the generator from arbitrarily altering semantic information. This not only stabilizes unsupervised training but also solves the problems of pattern output homogenization and training divergence in unsupervised GANs, improving model robustness.

[0125] Cycle consistency loss The calculation formula is:

[0126]

[0127] in, Represents the feature extraction function. express Norm.

[0128] Because generators may unnecessarily shift the color distribution or brightness range of input images (such as over-enhancing already sharp images), an identity loss is introduced to help stabilize training. This ensures that the output should remain essentially unchanged when sharp images are input into the generator. It also accelerates convergence by providing the generator with an intuitive constraint to maintain its original state, avoiding drastic fluctuations in the early stages of training.

[0129] Loss of identity The calculation formula is:

[0130]

[0131] in, Represents the target domain image.

[0132] Composite loss function The calculation formula is:

[0133]

[0134] in, To counteract the weight of loss, Weights for perceiving loss, The weights for the cycle consistency loss, Weighting for identity loss.

[0135] While relying solely on adversarial loss can drive generated images to approximate the target domain in terms of overall distribution, it is highly susceptible to pattern collapse, leading to homogenized outputs and structural blurring or artifacts due to a lack of detail constraints. Perceptual loss alone can improve local quality by forcing semantic feature alignment through pre-trained VGG, but it loses its ability to shape the global realism of the image due to the adversarial game without the discriminator, and it cannot independently resolve domain mapping ambiguities in unsupervised scenarios. If only cyclic consistency loss is used, although it can constrain the basic consistency of content structure in cross-domain transformation, it is difficult to ensure the visual realism and detail sharpness of the generated results, and it is prone to color distortion or flat images. On the other hand, using identity loss in isolation can only stabilize the tonal and brightness relationships of input and output to a limited extent, and it is completely powerless to drive effective inter-domain transformation. By organically integrating adversarial loss (ensuring global realism), perceptual loss (injecting semantic-level quality supervision), cyclic consistency loss (maintaining content structure stability), and identity loss (regulating color and style shifts) into a composite loss function, the method of this invention can collaboratively overcome the core difficulties of underwater image enhancement under unsupervised conditions. It generates highly realistic and clear images in adversarial games, accurately restores details and textures lost due to underwater degradation through VGG feature alignment, protects key information from distortion by using cyclic constraints, and suppresses abnormal color shifts by using identity loss. Ultimately, it achieves a breakthrough enhancement effect that surpasses traditional single-loss or simple combination methods, achieving unified optimization in realism, clarity, content fidelity, and color naturalness.

[0136] Step 6: Repeat steps 1 to 5 until Pyra-Unet and the discriminator reach Nash equilibrium, thus obtaining the trained Pyra-Unet. Input the underwater image into the trained Pyra-Unet to obtain the enhanced and clear image.

[0137] To verify the applicability of the present invention, the indicators were tested on the UIEBD and LSUI underwater image datasets, and the results were compared with existing methods and techniques.

[0138] like Figure 3 , Figure 4 As shown, on the UIEBD dataset:

[0139] UDCP (Underwater Dark Passage Prior): PSNR is 11.5, SSIM is 0.61;

[0140] CycleGAN (Recurrent Adversarial Network): PSNR is 17.8, SSIM is 0.79;

[0141] The method of this invention (Ours) achieves a PSNR (Peak Signal-to-Noise Ratio) of 19.8 and an SSIM (Structural Similarity) of 0.84, indicating that the present invention has significant improvements over UDCP and CycleGAN in terms of image fidelity and structural similarity.

[0142] On the LSUI dataset:

[0143] UWGAN (Underwater Generative Adversarial Network): PSNR is 20.5, SSIM is 0.82;

[0144] MMLE (Multi-Module Fusion Modeling Based on Machine Learning for Underwater Image Enhancement): PSNR is 16.5, SSIM is 0.67;

[0145] U-shape (U-type transformer integration method): PSNR is 24.0, SSIM is 0.85;

[0146] FunIE-GAN (Fast Underwater Augmentation-Generative Adversarial Network): PSNR is 22.8, SSIM is 0.83;

[0147] The method of this invention achieves a PSNR of 21.5 and an SSIM of 0.86, which is superior to UWGAN and MMLE and close to the level of Ushape and FunIE-GAN, demonstrating the competitiveness of this invention under unsupervised conditions.

Claims

1. An underwater image cross-domain enhancement method based on unsupervised UCYcleGAN, characterized in that, Includes the following steps: Step 1: Constructing the improved U-shaped generator Pyra-Unet: Pyra-Unet includes encoder downsampling, decoder upsampling, bottleneck layer, and multi-scale spatial pyramid attention module MSPA embedded in skip connections; Step 2: Input the degraded underwater image into Pyra-Unet, and Pyra-Unet outputs a fake, clear image; Step 3: Input a real clear image from the target domain clear image dataset and a fake clear image output by Pyra-Unet into the discriminator. The discriminator outputs a discrimination probability value, and the adversarial loss is calculated from the discrimination probability value. Step 4: Input the real clear image and the fake clear image into the pre-trained deep convolutional neural network VGG respectively, extract the corresponding high-dimensional feature maps respectively, and calculate the perceptual loss between the two high-dimensional feature maps. Step 5: Weight the adversarial loss, perception loss, cycle consistency loss and identity loss to form the final composite loss function. Update the parameters of Pyra-Unet and the discriminator through backpropagation so that Pyra-Unet minimizes the total loss and the discriminator maximizes its discrimination ability. Step 6: Repeat steps 1 to 5 until Pyra-Unet and the discriminator reach Nash equilibrium, thus obtaining the trained Pyra-Unet. Input the underwater image into the trained Pyra-Unet to obtain the enhanced and clear image.

2. The underwater image cross-domain enhancement method based on unsupervised UCYcleGAN according to claim 1, characterized in that, In step one, the encoder downsampling is a three-layer downsampling. The first downsampling layer includes a first convolutional layer, a first batch normalization layer, a first ReLU layer, and a first residual block connected in sequence. BN represents batch normalization processing, and ReLU represents the ReLU activation function. The second downsampling layer has the same structure as the first downsampling layer. The third downsampling layer includes a second convolutional layer, a second batch normalization layer, and a second ReLU layer connected in sequence.

3. The underwater image cross-domain enhancement method based on unsupervised UCYcleGAN according to claim 2, characterized in that, In step one, the bottleneck layer includes dilated convolution groups, convolution stacking units, dilated convolution operation units, and feature concatenation units. The dilated convolution groups employ four levels of cascaded convolutional layers. The convolution stacking units include dilated convolution, batch normalization, and activation functions. The formula for dilated convolution operation in the dilated convolution operation unit is: ； in, Indicates location, Indicates the input feature map, Indicates position The output after dilated convolution operation For expansion rate, This represents the sequence number of the convolutional layer in the dilated convolution group. Indicates the first The weights of each convolutional layer.

4. The underwater image cross-domain enhancement method based on unsupervised UCYcleGAN according to claim 3, characterized in that, In step one, the decoder upsampling is a three-layer upsampling. The first layer upsampling corresponds to the feature map of the third layer downsampling. The first layer upsampling includes the first upsampling, the first skip connection, and the first convolution. In the first upsampling, the downsampling is performed after four dilated convolutions; In the first hop connection, the features from the hop connection after the third MSPA layer are concatenated with the output of the third MSPA layer, and multi-scale features are weighted and fused through pyramid pooling and channel attention. The second upsampling layer corresponds to the feature map of the second downsampling layer. The second upsampling layer includes the second upsampling, the second skip connection, and the second convolution. In the second hop connection, the features from the hop connection after the second MSPA layer are concatenated with the output of the second MSPA layer. The third upsampling layer corresponds to the feature map of the first downsampling layer. The third upsampling layer includes the third upsampling, the third skip connection, and the third convolution. In the third hop connection, the features from the hop connection after the first MSPA layer are concatenated with the output of the first MSPA layer.

5. The underwater image cross-domain enhancement method based on unsupervised UCYcleGAN according to claim 4, characterized in that, In step one, the multi-scale spatial pyramid attention module MSPA includes a hierarchical phantom convolution module HPC and a spatial pyramid recalibration module SPR. The hierarchical phantom convolutional module (HPC) enhances spatial information through multi-scale feature extraction, performs channel dimension segmentation on the input feature map, and generates... Each subset is processed through a hierarchical residual convolutional chain to fuse features from different receptive fields; The hierarchical phantom convolutional module (HPC) includes splitting units, convolutional hierarchy chains, and splicing units. The splitting unit uniformly splits the input feature map along the channel dimension into There are subsets, and each subset has channels. ; The convolutional hierarchy uses convolutional groups with hierarchical residual connections. The first group directly processes a subset; in subsequent groups, the current subset is added to the output of the previous group and then convolved. The splicing unit splices all subset outputs along the channel dimension to form multi-scale enhanced features.

6. The underwater image cross-domain enhancement method based on unsupervised UCYcleGAN according to claim 5, characterized in that, In step one, the Space Pyramid Recalibration Module (SPR) adopts a dual-component collaborative mechanism, including a space pyramid aggregation block and a channel interaction block. The spatial pyramid aggregation blocks employ dual-path pooling, namely global path pooling and local path pooling; after dual-path pooling, adaptive fusion and weighted summation are performed to obtain the feature map. ; Channel Interaction Block Received Feature Map The initial channel attention weight vector is computed through a lightweight two-layer 1×1 convolutional bottleneck structure; the first layer 1×1 convolution will... Number of channels Compress to Then ReLU is activated, introducing nonlinearity; The second 1×1 convolution reduces the number of channels from Restore to .

7. The underwater image cross-domain enhancement method based on unsupervised UCYcleGAN according to claim 6, characterized in that, In step three, combating losses The calculation formula is: ； in, Expressing expectations, Indicates the discriminator, Represents a generator. For the input image, This represents the probability value output by the discriminator after evaluating the image generated by the generator.

8. The underwater image cross-domain enhancement method based on unsupervised UCYcleGAN according to claim 7, characterized in that, In step four, loss is perceived. The calculation formula is: ； in, Indicates correspondence The weighting coefficients, Indicates correspondence The weighting coefficients, Indicates correspondence The weighting coefficients, Indicates peak signal-to-noise ratio. Represents a quality index. This represents a correlation indicator.

9. The underwater image cross-domain enhancement method based on unsupervised UCYcleGAN according to claim 8, characterized in that, In step five, the cycle consistency loss The calculation formula is: ； in, Represents the feature extraction function. express Norm; Loss of identity The calculation formula is: ； in, Represents the target domain image.

10. The underwater image cross-domain enhancement method based on unsupervised UCYcleGAN according to claim 9, characterized in that, In step five, the composite loss function The calculation formula is: ； in, To counteract the weight of loss, Weights for perceiving loss, The weights for the cycle consistency loss, Weighting for identity loss.

Images