An underwater image enhancement method, device, medium and product

Abstract

The application discloses an underwater image enhancement method and device, a medium and a product, and relates to the technical field of image enhancement. The method comprises the following steps: inputting a distorted underwater image into a picture coding network to obtain distorted image coding; inputting the distorted image coding into a style mapping network to obtain a plurality of global enhancement image codings and a plurality of local enhancement image codings; inputting the distorted image coding, the plurality of global enhancement image codings and the plurality of local enhancement image codings into a style matching network to obtain style matching coding; and inputting the distorted image coding, the style matching coding, the plurality of global enhancement image codings and the plurality of local enhancement image codings into a migration enhancement network to obtain a plurality of clear underwater images. The migration enhancement network can improve the rationality of underwater image enhancement.

Description

Technical Field

[0001] This invention relates to the field of image enhancement technology, and in particular to an underwater image enhancement method, apparatus, medium, and product. Background Technology

[0002] Clear underwater images play a crucial role in marine exploration, underwater scene analysis, and underwater rescue. However, due to the absorption and scattering of light by water and the influence of suspended particles, directly acquired underwater images suffer from severe nonlinear distortion, often resulting in a lack of red, predominantly blue-green hues, and low clarity and contrast. Furthermore, the scale and diversity of real-world underwater image datasets are far less than those for terrestrial images. This lack of high-quality underwater images and datasets severely impacts the performance of underwater vision-based tasks such as detection, classification, and segmentation. In recent years, deep learning-based underwater image enhancement methods have achieved excellent enhancement results. Models based on deep convolutional neural networks (CNNs) and generative adversarial networks (GANs) have demonstrated good performance in enhancing underwater image quality. Existing deep learning-based methods rely on large amounts of paired and unpaired datasets. However, the high cost and difficulty of acquiring large-scale underwater data, coupled with the limitations of existing underwater datasets such as small image size and limited underwater scene representation, makes it difficult for trained models to cope with different types of distortion and hinders their deployment and application, thus limiting the potential value of deep learning methods in practical applications. Summary of the Invention

[0003] The purpose of this invention is to provide an underwater image enhancement method, apparatus, medium, and product that can improve the rationality of underwater image enhancement.

[0004] To achieve the above objectives, the present invention provides the following solution:

[0005] An underwater image enhancement method, comprising:

[0006] The distorted underwater image is input into an image coding network to obtain the distorted image code;

[0007] The distorted image encoding is input into a style mapping network to obtain multiple global enhancement image encodings and multiple local enhancement image encodings; the number of global enhancement image encodings and the number of local enhancement image encodings are both equal to the preset number of styles;

[0008] The distorted image code, multiple global enhancement image codes, and multiple local enhancement image codes are input into the style matching network to obtain the style matching code;

[0009] Distorted image encoding, style pairing encoding, multiple global enhancement image encodings, and multiple local enhancement image encodings are input into the transfer enhancement network to obtain multiple clear underwater images; the multiple clear underwater images have different styles; the image encoding network, the style mapping network, the style pairing network, and the transfer enhancement network are jointly trained using historical distorted underwater images.

[0010] Optionally, the migration enhancement network includes: a first branch and a second branch connected in parallel;

[0011] The first branch includes a first convolutional layer, a first encoder, a second convolutional layer, a first decoder, a third convolutional layer, and a filter connected in sequence;

[0012] The second branch includes a style selection module, a fourth convolutional layer, a second encoder, a fifth convolutional layer, a second decoder, and a sixth convolutional layer connected in sequence;

[0013] The first input of the first convolutional layer is connected to the output of the third convolutional layer through a first splicing layer;

[0014] The second input of the first convolutional layer is connected to the input of the style selection module;

[0015] The first encoder and the first decoder are connected via a multiple singular value decomposition module;

[0016] The first input of the fourth convolutional layer is connected to the output of the sixth convolutional layer through a second splicing layer;

[0017] The first encoder and the second encoder are connected through multiple splicing layers;

[0018] The first decoder is connected to the second decoder.

[0019] Optionally, the first encoder includes a first CST module, a second CST module, and a third CST module connected in sequence; the second encoder includes a fourth CST module, a fifth CST module, and a sixth CST module connected in sequence.

[0020] The output terminals of the first CST module and the fourth CST module are connected through a third splicing module;

[0021] The output terminals of the second CST module and the fifth CST module are connected through a fourth splicing module;

[0022] The output terminals of the third CST module and the sixth CST module are connected through the fifth splicing module;

[0023] The outputs of the first CST module, the second CST module, and the third CST module are all connected to the first decoder via a multi-singular value decomposition module.

[0024] Optionally, the first decoder includes a sixth splicing module, a seventh CST module, a seventh splicing module, an eighth CST module, an eighth splicing module, and a ninth CST module connected in sequence;

[0025] The second decoder includes a ninth splicing module, a tenth CST module, a tenth splicing module, an eleventh CST module, an eleventh splicing module, and a twelfth CST module connected in sequence;

[0026] The sixth, seventh, and eighth stitching modules are all connected to the first encoder via a multi-singular value decomposition module;

[0027] The sixth splicing module is also connected to the ninth splicing module;

[0028] The seventh splicing module is also connected to the tenth splicing module;

[0029] The eighth splicing module is also connected to the eleventh splicing module.

[0030] Optionally, the first CST module, the second CST module, the third CST module, the fourth CST module, the fifth CST module, the sixth CST module, the seventh CST module, the eighth CST module, the ninth CST module, the tenth CST module, the eleventh CST module, and the twelfth CST module have the same structure;

[0031] The first CST module includes: a seventh convolutional layer, a fourth singular value decomposition module, an eighth convolutional layer, a ninth convolutional layer, a tenth convolutional layer, and an average pooling layer connected in sequence.

[0032] Optionally, the multi-singular value decomposition module includes a first singular value decomposition module, a second singular value decomposition module, and a third singular value decomposition module;

[0033] The first singular value decomposition module is connected to the output of the first CST module and the eighth splicing module, respectively;

[0034] The second singular value decomposition module is connected to the output of the second CST module and the seventh splicing module, respectively;

[0035] The third singular value decomposition module is connected to the output of the third CST module and the sixth splicing module, respectively.

[0036] Optionally, before inputting the distorted underwater image into the image coding network to obtain the distorted image code, the following steps are also included:

[0037] Historically distorted underwater images are input into an initial image coding network to obtain historically distorted image codes;

[0038] The historical distorted image encoding is input into the initial style mapping network to obtain multiple historical global enhanced image encodings and multiple historical local enhanced image encodings; the number of historical global enhanced image encodings and the number of historical local enhanced image encodings are both equal to the preset number of styles;

[0039] The historical distortion image code, multiple historical global enhancement image codes, and multiple historical local enhancement image codes are input into the initial style pairing network to obtain the historical style pairing code;

[0040] The historical distorted image encoding, historical style pairing encoding, multiple historical global enhancement image encoding, and multiple historical local enhancement image encoding are input into the initial transfer enhancement network to obtain multiple clear historical underwater images;

[0041] Adjust the network parameters of the initial image encoding network, initial style mapping network, initial style pairing network, and initial transfer enhancement network, update the historical distorted underwater images, and return to the step "input the historical distorted underwater images into the initial image encoding network to obtain the historical distorted image encoding" until the number of iterations reaches the preset iteration threshold, and obtain the image encoding network, style mapping network, style pairing network, and transfer enhancement network.

[0042] A computer device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the underwater image enhancement method described above.

[0043] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the underwater image enhancement method described above.

[0044] A computer program product includes a computer program that, when executed by a processor, implements the underwater image enhancement method described above.

[0045] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0046] This invention provides an underwater image enhancement method, apparatus, medium, and product. It solves the problem of poor transferability and inability to enhance complex scenes in existing underwater image enhancement methods without complex training. Furthermore, it expands the existing dataset while enhancing image quality, alleviating the difficulty of obtaining real underwater image data to some extent. The method uses image encoding processed by several semantic information-based encoding modules as input to a transfer enhancement module. This module employs a pre-trained encoder-decoder structure, allowing the network to learn various complex underwater scenes without needing to learn them all, thus improving the model's generalization ability. Multiple cascaded CST modules and multi-scale jump connections transfer the styles of various clear images to distorted underwater images. Thanks to the use of average pooling and unpooling in the encoder-decoder structure, this method effectively preserves and restores the texture details of degraded images. Moreover, guided by semantic information, this method can further enhance local regions and enrich their colors. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a schematic flowchart of an underwater image enhancement method provided in Embodiment 1 of the present invention;

[0049] Figure 2 This is an overall structural diagram of the TSUIE provided in Embodiment 1 of the present invention;

[0050] Figure 3 This is a schematic diagram of the migration enhancement module provided in Embodiment 1 of the present invention;

[0051] Figure 4 This is a schematic diagram of the CST module provided in Embodiment 1 of the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] The purpose of this invention is to provide an underwater image enhancement method, apparatus, medium, and product that can improve the rationality of underwater image enhancement.

[0054] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] Example 1

[0056] like Figure 1 As shown, an underwater image enhancement method in this embodiment includes:

[0057] Step 101: Input the distorted underwater image into the image coding network to obtain the distorted image code.

[0058] Step 102: Input the distorted image encoding into the style mapping network to obtain multiple global enhancement image encodings and multiple local enhancement image encodings. The number of global enhancement image encodings and the number of local enhancement image encodings are both equal to the preset number of styles.

[0059] Step 103: Input the distorted image code, multiple global enhancement image codes, and multiple local enhancement image codes into the style matching network to obtain the style matching code.

[0060] Step 104: Input the distorted image encoding, style pairing encoding, multiple global enhancement image encodings, and multiple local enhancement image encodings into the transfer enhancement network to obtain multiple clear underwater images. The multiple clear underwater images have different styles. The image encoding network, style mapping network, style pairing network, and transfer enhancement network are jointly trained using historical distorted underwater images.

[0061] like Figures 2 to 4 The migration-enhanced network consists of a first branch and a second branch connected in parallel.

[0062] The first branch includes a first convolutional layer, a first encoder, a second convolutional layer, a first decoder, a third convolutional layer, and a filter, which are connected in sequence.

[0063] The second branch consists of a style selection module, a fourth convolutional layer, a second encoder, a fifth convolutional layer, a second decoder, and a sixth convolutional layer, connected in sequence.

[0064] The first input of the first convolutional layer and the output of the third convolutional layer are connected through the first splicing layer.

[0065] The second input of the first convolutional layer is connected to the input of the style selection module.

[0066] The first encoder and the first decoder are connected via a multi-singular value decomposition module.

[0067] The first input of the fourth convolutional layer is connected to the output of the sixth convolutional layer through the second splicing layer.

[0068] The first encoder and the second encoder are connected through multiple splicing layers.

[0069] The first decoder is connected to the second decoder.

[0070] The first encoder includes a first CST module, a second CST module, and a third CST module connected in sequence. The second encoder includes a fourth CST module, a fifth CST module, and a sixth CST module connected in sequence.

[0071] The output terminals of the first CST module and the fourth CST module are connected through the third splicing module.

[0072] The outputs of the second CST module and the fifth CST module are connected through the fourth splicing module.

[0073] The outputs of the third CST module and the sixth CST module are connected through the fifth splicing module.

[0074] The outputs of the first CST module, the second CST module, and the third CST module are all connected to the first decoder through a multi-singular value decomposition module.

[0075] The first decoder includes a sixth splicing module, a seventh CST module, a seventh splicing module, an eighth CST module, an eighth splicing module, and a ninth CST module connected in sequence.

[0076] The second decoder includes the ninth splicing module, the tenth CST module, the eleventh CST module, the eleventh splicing module, and the twelfth CST module connected in sequence.

[0077] The sixth, seventh, and eighth stitching modules are all connected to the first encoder via a multi-singular value decomposition module.

[0078] The sixth splicing module is also connected to the ninth splicing module.

[0079] The seventh splicing module is also connected to the tenth splicing module.

[0080] The eighth splicing module is also connected to the eleventh splicing module.

[0081] The first, second, third, fourth, fifth, sixth, seventh, eighth, ninth, tenth, eleventh, and twelfth CST modules have the same structure.

[0082] The first CST module includes: the seventh convolutional layer, the fourth singular value decomposition module, the eighth convolutional layer, the ninth convolutional layer, the tenth convolutional layer, and the average pooling layer, which are connected in sequence.

[0083] The multi-singular value decomposition module includes a first singular value decomposition module, a second singular value decomposition module, and a third singular value decomposition module.

[0084] The first singular value decomposition module is connected to the output of the first CST module and the eighth splicing module, respectively.

[0085] The second singular value decomposition module is connected to the output of the second CST module and the seventh splicing module, respectively.

[0086] The third singular value decomposition module is connected to the output of the third CST module and the sixth splicing module, respectively.

[0087] Before step 101, the following is also included:

[0088] Step 105: Input the historical distorted underwater image into the initial image coding network to obtain the historical distorted image code.

[0089] Step 106: Input the historical distorted image codes into the initial style mapping network to obtain multiple historical global augmentation image codes and multiple historical local augmentation image codes. The number of historical global augmentation image codes and the number of historical local augmentation image codes are both equal to the preset number of styles.

[0090] Step 107: Input the historical distortion image code, multiple historical global enhancement image codes, and multiple historical local enhancement image codes into the initial style pairing network to obtain the historical style pairing code.

[0091] Step 108: Input the historical distorted image encoding, historical style pairing encoding, multiple historical global enhancement image encoding, and multiple historical local enhancement image encoding into the initial migration enhancement network to obtain multiple clear historical underwater images.

[0092] Step 109: Adjust the network parameters of the initial image encoding network, initial style mapping network, initial style pairing network, and initial transfer enhancement network; update the historical distorted underwater images; and return to step 105 until the number of iterations reaches the preset iteration threshold to obtain the image encoding network, style mapping network, style pairing network, and transfer enhancement network.

[0093] This invention uses transfer learning to match styles from clear images to distorted underwater images based on the semantic information of the images. This enhances the visual quality of underwater images while generating underwater images of various styles, thus expanding the underwater image dataset. This enhancement method is called TSUIE. The process involves: an image encoding network analyzing the semantic information of the image to encode it; a style mapping network analyzing the encoding of the input distorted image and the number of styles to be generated, randomly selecting clear image encodings to generate global and local enhanced image encodings; inputting the distorted image encoding and the enhanced image encodings selected by the style mapping network into a style pairing network to generate style-paired encodings based on the encoding information; and inputting the distorted image encoding, the mapped enhanced image encoding, and the style-paired encodings into a transfer enhancement network to generate clear images of various styles. This process includes the following components:

[0094] 1. Image Coding Network: The semantic information of the content image X and the style reference image Y can be obtained through manual masking annotation or by using a pre-trained underwater semantic annotation model. The image coding network converts the semantic information of the input image into category codes and their coordinate information reflecting multiple species. Due to the complexity of underwater scenes, an image often contains multiple categories, so this encoding includes the category codes for multiple categories within an image. The inclusion of semantic segmentation information allows for the centralization of category information, which is beneficial for the implementation of transfer enhancement.

[0095] This invention considers the coding of eight common underwater species categories:

[0096] a) Water background.

[0097] b) Diver.

[0098] c) Aquatic plants.

[0099] d) Shipwrecks and ruins.

[0100] e) Robots and instruments.

[0101] f) Coral reefs and other invertebrates.

[0102] g) Fish and other vertebrates.

[0103] h) Seabed and rocks.

[0104] The semantic information of the input image is represented by 3-bit binary RGB color encoding to represent these eight categories. The corresponding semantic category encodings after passing through the image encoding network are shown in Table 1.

[0105] Table 1. Semantic category encoding table after image encoding network.

[0106] Water background 000(black) 0 seabed and rocks 111(white) 1 Aquatic plants 010 (green) 2 diver 001 (blue) 3 Robots and Instruments 100 (red) 4 fish 110 (yellow) 5 Shipwrecks and ruins 011 (lightblue) 7 Coral reef 101 (purple) 8

[0107] 2. Style Mapping Network: This part consists of local mapping F L (*) and global mapping F g (*) Consists of two parts, given a target style number i. The global mapping part F g By randomly sampling y∈Y, the encoding S of i clear images is randomly mapped. g =i*F g (*). Due to the complexity and diversity of underwater imaging degradation, image transfer enhancement from a global perspective alone is difficult. Therefore, local style mapping is incorporated to guide local image transfer enhancement. The local style mapping network F... L By randomly sampling y∈Y, the encoding S of i clear images is mapped. L =i*F L (y).

[0108] 3. Style Matching Network: This part matches the content image encoding with the global target category style encoding S. L and local target category style coding S g The process involves matching a style category in a style image with a style category in a content image. These two categories can be the same or different. The matching pairs are then output, along with the selection criteria, which are as follows:

[0109]

[0110] Where f c (x) represents the number of features of a certain category x∈X in the content image, f s (x) represents the number of features for a certain category y∈Y in the style image, and α and β are hyperparameters. This network is a multi-branch output including global style pairing and local style pairing. The local pairing part contains multiple outputs. For example, if the style image has m categories, it can generate n = m! + m local style pairings. This allows multiple styles of enhanced images to be generated using only one style reference image.

[0111] 4. Transfer Enhancement Network: This network consists of two parts: global enhancement and local enhancement. The transfer enhancement network of this invention transforms the input image into enhanced images of various styles based on style pairing encoding. The network as a whole adopts an Encoder-Decoder structure. In the encoder and corresponding decoder modules, convolutional blocks and SVD are used to form a CST module to realize the transfer of the reference image style. SVD is used in the jump link part to realize transfer enhancement. The local enhancement part incorporates style pairing encoding generated by semantic information as supplementary information. After passing through the CST module, it generates multi-scale local feature information with semantic information enhancement. The global enhancement module runs at the original resolution of the input image. After passing through the CST module, it generates multi-scale global feature information. Since the encoder-decoder structure is prone to losing spatial detail information, the local feature information is fused with the multi-scale global feature information generated by the global enhancement module and input into the decoder through a jump link. The dual-branch complementary structure and jump link provide multi-scale perception for the transfer enhancement of underwater images. The existence of style pairing encoding allows multiple style-enhanced images to be generated using only one style reference image. Furthermore, the network can accept style reference images of any category as input for any content image. This eliminates the need to generate a corresponding ground truth image for each image requiring enhancement, and outputs multiple styles of enhanced images, fully demonstrating that the method of this invention not only enhances images but also expands the dataset. This invention outputs multiple intermediate results and adds a filter at the end of the network, using the Underwater Image Quality Metric (UIQM), Underwater Color Image Quality Evaluation (UCIQE), Structure Similarity Index Measure (SSIM), and Peak Signal-to-Noise Ratio (PSNR) as filtering indicators to remove poor-quality enhanced images.

[0112] The Content Image and Style Reference Image Feature Transfer (CST) module is the main component of the transfer enhancement network. Multiple CST modules are cascaded to form the encoder and decoder structure. It consists of convolution, Singular Value Decomposition (SVD), and average pooling. SVD is embedded in the ConvX_1 layer, and its structure is as follows: Figure 4As shown, this invention uses SVD to map content features to a feature space that matches style features. This style extraction method has strong theoretical support and interpretability. While max pooling in CNNs can effectively capture multi-scale features, it makes pixel-level spatial details unreconstructable during upsampling, failing to fully recover the structural information of the content image. To better preserve the local details of the input image, this invention uses average pooling instead of max pooling. Average pooling can integrate all information on the feature map to better preserve the original image information.

[0113] A pre-trained VGG encoder is used to construct the transfer learning network, and the decoder is trained on multiple underwater image datasets (UIEB, EUVP, LUSI). The l2 loss and the reconstruction loss (Equation 1) are minimized. The network is optimized using Adam with an initial learning rate set to 1e. -3 The linear decay was adjusted to zero. The experiment was conducted on a PC equipped with an NVIDIA 24GB RTX 3090 GPU and Ubuntu 20.04 operating system.

[0114]

[0115] Among them, I i I o These are the input image and the output image after decoding, respectively. E is the VGG encoder. After training, the decoder parameters are fixed.

[0116] Comparison Method: This invention compares TSUIE with eight recently proposed typical methods, including two physical model-based methods (Uw-HL, L2UWE) and three deep learning-based methods (Water-Net, Ucolor, Ushape, FUnIE-GAN, UWCNN). All of the deep learning-based methods were tested using the SUIM dataset.

[0117] Evaluation Metrics: For image quality comparison, this invention employs a variety of widely used metrics for comprehensive evaluation, including full-reference quality metrics such as Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM), and non-reference quality metrics such as Underwater Image Quality Measurement (UIQM) and Underwater Color Image Quality Evaluation (UCIQE). For image diversity, Learned Perceptual Image Patch Similarity (LPIPS, a parameter used to evaluate the perceptual similarity between two images) is used to assess the diversity of the enhanced images produced by this invention.

[0118] Traditional methods like UW-HL and L2UWE often suffer from color bias due to the introduction of excessive blue and red components. Water-Net introduces a white balance channel during enhancement, which is not always reliable for underwater images, potentially leading to color bias and lower brightness in the enhanced image due to backscattering. From a color correction perspective, Ucolor achieves better visual quality due to the introduction of a transmission medium, but ignores the inconsistent attenuation characteristics of different color channels in underwater images, failing to effectively improve low contrast and blurred details and exhibiting overall color cast. FUnIE's advantage lies in its fast, lightweight, and low-parameter model, but this limits its scalability on complex images, and the enhancement result is generally yellowish and reddish. UWCNN, trained on synthetic underwater image data, has poor overall generalization and suffers from overcompensation for severely attenuated images. U-shape typically fails to correct blue tones in images.

[0119] The TSUIE of this invention can stably correct severe color deviations without introducing other artifacts. TSUIE naturally improves global brightness and enhances contrast in low-light underwater scenes. It outperforms physically-based models without using scene depth or prior information, and its network structure is much simpler than existing learning-based models. Comparative results show that the method of this invention achieves a better balance between visual quality and color richness.

[0120] Table 2 shows the quantitative results of TSUIE and various comparison models on the SUIM dataset. The results indicate that TSUIE achieves better overall results, performing best in PSNR, SSIM, and UIQM metrics. Compared to the second-ranked method, the method of this invention improves PSNR by 24.1% (from 20.742 to 25.743). Regarding the UIQM metric, Ucolor scores 0.799 because it considers the features of different color spaces and integrates them into a unified structure; its enhancement effect naturally benefits from UIQM, resulting in a relatively high score for Ucolor on this metric. However, although its UCIQE score is high, Ucolor's user-perceived score is lower than TSUIE's. TSUIE, the method of this invention, achieves a better UCIQE score, further demonstrating the advantages of this method in maintaining and restoring image sharpness and contrast. Furthermore, the method of this invention also achieves the best results in SSIM and PSNR metrics, indicating that the enhanced images, while having different styles, still retain the content and structure of the original image. The user-perceived score also shows that TSUIE's enhancement effect is more in line with the user's visual experience.

[0121] Table 2. Quantitative results of TSUIE and various comparative models on the SUIM dataset.

[0122] L2UWE 16.212 0.791 0.710 2.238 UW-HL 17.077 0.713 0.655 1.945 FUnIE-NET 20.354 0.806 0.766 2.682 U-shape 20.962 0.771 0.763 2.181 Ucolor 20.742 0.847 0.799 2.915 UWCNN 18.158 0.649 0.717 1.585 Water-NET 19.298 0.873 0.734 2.510 TSUIE 25.743 0.882 0.781 3.875

[0123] To further verify that TSUIE can enhance visual quality while increasing style diversity, this invention was tested on the SUIM dataset. Specifically, this invention sets the number of images for the target style to i=4, i.e., four local style images and four global style images. This invention uses LPIPS to evaluate the diversity of the enhanced images, as shown in Table 3. TSUIE obtained the highest LPIPS score, indicating that the method of this invention can not only enhance the visual quality of distorted underwater images but also increase image diversity, thus expanding the existing dataset.

[0124] Table 3. Comparison of Style Diversity Among Different Models

[0125] FUnIE-NET 0.113 U-shape 0.109 Ucolor 0.129 UWCNN 0.121 Water-NET 0.136 TSUIE 0.217

[0126] To demonstrate the role of each module in TSUIE, this invention employs ablation experiments with a TSUIE model lacking a style mapping network, a style pairing network, and a local enhancement module. As shown in Table 4, the performance of TSUIE is affected to varying degrees when any one of the three modules is missing. The variety of enhanced images decreases significantly when the style mapping network is missing. When local style transfer enhancement is not used, all evaluation metrics for enhanced images decrease, demonstrating that semantic information guidance can improve enhancement performance. While all evaluation metrics decrease when the style pairing network is not used, the UIQM metric improves, indicating that the style pairing network, to some extent, sacrifices image sharpness but increases the variety of generated images. Overall, the performance of a model lacking any one module is inferior to that of a complete model.

[0127] Table 4 Ablation Experiment Results for Each Module

[0128]

[0129] Example 2

[0130] A computer device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement an underwater image enhancement method as described in Embodiment 1.

[0131] Example 3

[0132] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an underwater image enhancement method as described in Embodiment 1.

[0133] Example 4

[0134] A computer program product includes a computer program that, when executed by a processor, implements an underwater image enhancement method as described in Embodiment 1.

[0135] Example 5

[0136] A computer device, which may be a database, includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The database stores pending transactions. The I / O interfaces facilitate information exchange between the processor and external devices. The communication interface allows communication with external terminals via a network connection. When executed by the processor, the computer program implements an underwater image enhancement method as described in Embodiment 1.

[0137] It should be noted that the object information (including but not limited to object device information, object personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this invention are all information and data authorized by the object or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0138] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided by this invention may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided by this invention may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0139] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0140] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An underwater image enhancement method, characterized in that, include: The distorted underwater image is input into an image coding network to obtain the distorted image code; The distorted image code is input into a style mapping network to obtain multiple global enhancement image codes and multiple local enhancement image codes; The number of global enhanced image codes and the number of local enhanced image codes are both equal to the number of preset styles; The distorted image code, multiple global enhancement image codes, and multiple local enhancement image codes are input into the style matching network to obtain the style matching code; Distorted image encoding, style pairing encoding, multiple global enhancement image encodings, and multiple local enhancement image encodings are input into the transfer enhancement network to obtain multiple clear underwater images; The multiple clear underwater images have different styles; The image encoding network, the style mapping network, the style matching network, and the transfer enhancement network are obtained by joint training using historically distorted underwater images; The migration enhancement network includes: a first branch and a second branch connected in parallel; The first branch includes a first convolutional layer, a first encoder, a second convolutional layer, a first decoder, a third convolutional layer, and a filter connected in sequence; The second branch includes a style selection module, a fourth convolutional layer, a second encoder, a fifth convolutional layer, a second decoder, and a sixth convolutional layer connected in sequence; The first input of the first convolutional layer is connected to the output of the third convolutional layer through a first splicing layer; The second input of the first convolutional layer is connected to the input of the style selection module; The first encoder and the first decoder are connected via a multiple singular value decomposition module; The first input of the fourth convolutional layer is connected to the output of the sixth convolutional layer through a second splicing layer; The first encoder and the second encoder are connected through multiple splicing layers; The first decoder is connected to the second decoder.

2. The underwater image enhancement method according to claim 1, characterized in that, The first encoder includes a first CST module, a second CST module, and a third CST module connected in sequence; the second encoder includes a fourth CST module, a fifth CST module, and a sixth CST module connected in sequence. The output terminals of the first CST module and the fourth CST module are connected through a third splicing module; The output terminals of the second CST module and the fifth CST module are connected through a fourth splicing module; The output terminals of the third CST module and the sixth CST module are connected through the fifth splicing module; The outputs of the first CST module, the second CST module, and the third CST module are all connected to the first decoder via a multi-singular value decomposition module.

3. The underwater image enhancement method according to claim 2, characterized in that, The first decoder includes a sixth splicing module, a seventh CST module, a seventh splicing module, an eighth CST module, an eighth splicing module, and a ninth CST module connected in sequence; The second decoder includes a ninth splicing module, a tenth CST module, a tenth splicing module, an eleventh CST module, an eleventh splicing module, and a twelfth CST module connected in sequence; The sixth, seventh, and eighth stitching modules are all connected to the first encoder via a multi-singular value decomposition module; The sixth splicing module is also connected to the ninth splicing module; The seventh splicing module is also connected to the tenth splicing module; The eighth splicing module is also connected to the eleventh splicing module.

4. The underwater image enhancement method according to claim 3, characterized in that, The first CST module, the second CST module, the third CST module, the fourth CST module, the fifth CST module, the sixth CST module, the seventh CST module, the eighth CST module, the ninth CST module, the tenth CST module, the eleventh CST module, and the twelfth CST module have the same structure; The first CST module includes: a seventh convolutional layer, a fourth singular value decomposition module, an eighth convolutional layer, a ninth convolutional layer, a tenth convolutional layer, and an average pooling layer connected in sequence.

5. The underwater image enhancement method according to claim 3, characterized in that, The multi-singular value decomposition module includes a first singular value decomposition module, a second singular value decomposition module, and a third singular value decomposition module; The first singular value decomposition module is connected to the output of the first CST module and the eighth splicing module, respectively; The second singular value decomposition module is connected to the output of the second CST module and the seventh splicing module, respectively; The third singular value decomposition module is connected to the output of the third CST module and the sixth splicing module, respectively.

6. The underwater image enhancement method according to claim 1, characterized in that, Before inputting the distorted underwater image into the image coding network to obtain the distorted image code, the following steps are also included: Historically distorted underwater images are input into an initial image coding network to obtain historically distorted image codes; The historical distorted image encoding is input into the initial style mapping network to obtain multiple historical global enhanced image encodings and multiple historical local enhanced image encodings; the number of historical global enhanced image encodings and the number of historical local enhanced image encodings are both equal to the preset number of styles; The historical distortion image code, multiple historical global enhancement image codes, and multiple historical local enhancement image codes are input into the initial style pairing network to obtain the historical style pairing code; The historical distorted image encoding, historical style pairing encoding, multiple historical global enhancement image encoding, and multiple historical local enhancement image encoding are input into the initial transfer enhancement network to obtain multiple clear historical underwater images; Adjust the network parameters of the initial image encoding network, initial style mapping network, initial style pairing network, and initial transfer enhancement network, update the historical distorted underwater image, and return to the step "input the historical distorted underwater image into the initial image encoding network to obtain the historical distorted image encoding" until the number of iterations reaches the preset iteration threshold, and obtain the image encoding network, style mapping network, style pairing network, and transfer enhancement network.

7. A computer device, comprising: The memory and processor contain a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement an underwater image enhancement method according to any one of claims 1-6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements an underwater image enhancement method according to any one of claims 1-6.

9. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements an underwater image enhancement method according to any one of claims 1-6.

Images