A method and system for underwater image enhancement based on hierarchical decomposition and fusion

Abstract

This invention discloses an underwater image enhancement method and system based on hierarchical decomposition and fusion, comprising: constructing an HDNet model; training the HDNet model; and performing underwater image enhancement using the trained HDNet model. The HDNet model, according to the image processing procedure, sequentially includes: a first point convolutional layer for expanding the channels of the input image; a first channel-specific color correction module for splitting the channels of the input image and adjusting the color distribution of each channel individually; a first feature fusion module for fusing the colors of each channel; a pixel-specific detail adjustment module for representing pixels using various wave functions and modulating the signal; a second feature fusion module for further fusion; a second channel-specific color correction module for adjusting the color distribution of each channel to achieve a normal distribution; and a second point convolution for compressing channels to output the enhanced image.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically a method and system for underwater image enhancement based on hierarchical decomposition and fusion. Background Technology

[0002] The ocean is a rich ecological treasure trove, of great significance to sustainable development. In recent years, advancements in marine technology have ushered in a new era of ocean exploration. However, the unique characteristics of the underwater environment, especially light attenuation and scattering, severely degrade the quality of underwater images, resulting in images that are generally bluish-green, blurry, and lacking in contrast. Underwater images are a critical source of information for ocean exploration missions, and their quality directly affects the effectiveness and efficiency of these activities. Therefore, underwater image enhancement technology is crucial for ensuring the efficiency and effectiveness of ocean exploration activities.

[0003] Underwater image degradation is a unique nonlinear distortion phenomenon caused by multiple factors, and its complexity and irreversibility pose a significant challenge to the field of image processing. Although computer vision technology has made some progress in underwater image enhancement, existing methods still need further optimization and innovation to address the unique underwater image degradation problem. Traditional methods rely on the statistical properties and physical assumptions of images, and while they can correct color casts, they are only effective in specific underwater scenes. In recent years, deep learning technology has become increasingly popular in the field of image processing. However, in the field of underwater image enhancement, researchers often encounter obstacles when obtaining realistic underwater scenes and ideal reference images. Initial research methods utilized Generative Adversarial Networks (GANs) to generate datasets or implemented unpaired learning paradigms, such as WaterGAN and UWGAN. Although the models can achieve significant color enhancement effects on synthetic underwater datasets, empirical evidence shows that there are distributional differences between synthetic and real underwater datasets, thus limiting the model's generalization effect on real datasets. To address this, Li et al. established a real-world underwater image enhancement dataset (UIEB), whose reference images were generated by more than a dozen excellent algorithms and selected through comparison by volunteers. The authors experimentally demonstrated that the model trained on the proposed dataset exhibits good generalization ability. This effort provided a reliable benchmark for subsequent deep learning-based underwater image enhancement methods, leading to the development of many excellent algorithms. However, current algorithms treat degraded images as a whole, focusing on fitting the dataset with deep learning models without fully considering the main factors contributing to underwater image degradation: light attenuation and light scattering. This limits further improvements in the performance of deep learning networks. Regarding light attenuation, the degree of light wave attenuation varies with color, resulting in abnormal color distribution in the RGB channels of the image, often manifesting as blue-green hues. Light reflected from objects is scattered by suspended particles in the water as it propagates towards the camera. This scattering alters the intensity and direction of light received by the camera, reducing the contrast and sharpness of the captured image. Furthermore, real underwater data better reflects underwater imaging patterns and has good generalization ability, but manually selected reference images inevitably introduce some quality defects. While semi-supervised frameworks exist for image augmentation, these frameworks aim to utilize both labeled and unlabeled data, allowing deep learning models to be less reliant on paired datasets. Nevertheless, these models still depend on some supervised learning, and low-quality reference images can hinder their learning from high-quality images. Therefore, deep learning-based underwater image augmentation networks still require further improvement and innovation based on degradation factors, and the adverse effects of suboptimal reference images on the model's learning process need to be mitigated. Summary of the Invention

[0004] Purpose of the invention: To address the shortcomings of existing methods in image enhancement and to reduce the negative impact of suboptimal reference images on the model learning process, this invention proposes an underwater image enhancement method and system based on hierarchical decomposition and fusion.

[0005] Technical solution: An underwater image enhancement method based on hierarchical decomposition and fusion, comprising:

[0006] Construct the HDNet model;

[0007] Train the HDNet model;

[0008] Underwater image enhancement was performed using the trained HDNet model;

[0009] The constructed HDNet model, following the image processing procedure, includes the following steps:

[0010] The first convolutional layer is used to expand the channels of the input image;

[0011] The first sub-channel color correction module is used to split the input image into channels and adjust the color distribution of each channel individually;

[0012] The first feature fusion module is used to fuse the colors of each channel;

[0013] The pixel detail adjustment module is used to represent pixels and modulate signals using various wave functions;

[0014] The second feature fusion module is used for further fusion.

[0015] The second sub-channel color correction module is used to adjust the color distribution of each channel to make it normally distributed.

[0016] The second convolution is used to compress channels to output an enhanced image.

[0017] Furthermore, both the first sub-channel color correction module and the second sub-channel color correction module are constructed according to the following steps:

[0018] The feature map is split to obtain three color channel information F. R F G F B and image content information F CAM_base ;

[0019] F R F G F B Each dimension is expanded to a higher dimension through 1×1 convolution, and then the color distribution is adjusted using InstanceNorm normalization.

[0020] Then, the three color channel information is compressed back to the original spatial dimension through 1×1 convolution to obtain the enhanced three color channel information;

[0021] The enhanced three color channel information and image content information F CAM_base The two are added together to obtain the enhanced image content information.

[0022] Finally, the enhanced three color channel information and the enhanced image content information are merged to form a new feature.

[0023] Furthermore, the first sub-channel color correction module is represented as follows:

[0024] F′ color =Conv 1×1 (InstanceNorm(Conv 1×1 (F color ))), color∈{R,GB}

[0025] F C ' AM_base =F R '+F G '+F b '+F CAM_base

[0026] F CAM_out =concat([F R ',F G ',F b ',F′ CAM_base ])

[0027] The second sub-channel color correction module is similar to the first sub-channel color correction module.

[0028] Furthermore, in the pixel-level detail adjustment module, various wave functions are used to represent pixels and modulate signals, specifically including:

[0029] Based on the optical principles of camera image acquisition, the pixels of underwater images are treated as light signals affected by light scattering. The pixels are then adjusted according to the principle of Fourier decomposition of signals using sine and cosine functions, including:

[0030] The optical signal is decomposed into a combination of different signal wave characteristics, which are represented as follows:

[0031] F wave =Conv 1×1 (F PAM_in )·wave(Conv 1×1 (F PAM_in wave∈{sin,cos,tanh}

[0032] In the formula, F wave F represents the characteristics of the signal wave. PAM_in This indicates the features input to the pixel-level detail adjustment module;

[0033] Convolutional operations of different scales are applied to the signal wave features to obtain contextual information;

[0034] Each signal wave characteristic and its input to the pixel-level detail adjustment module F PAM_in Fusion, yielding the fused wave characteristics Represented as:

[0035] F PAM_base =Conv 1×1 (F PAM_in )

[0036] F w ' ave =Conv k×k (F wave )

[0037]

[0038] In the formula, k is the size of the convolution kernel;

[0039] The feature representation output by the pixel detail adjustment module is as follows:

[0040]

[0041] Furthermore, both the first feature fusion module and the second feature fusion module adopt a dense residual connection structure.

[0042] Furthermore, the first feature fusion module is constructed according to the following steps:

[0043] The fusion process is initiated by a 3×3 convolution, followed by the PReLU activation function;

[0044] Subsequently, a concatenation of three 3×3 convolutions is employed. In this concatenation, the result of each convolutional layer is combined with the results of all previous layers, and the combined features are then input into the next convolutional stage; the output f of each layer in the concatenated convolution... l Represented as:

[0045] f0 = PReLU(Conv) 3×3 (F Fusion_in ))

[0046] f l =PReLU(Conv 3×3 ([f l-1 ,f l-2,...,f0]))

[0047] In the formula, F Fusion_in This represents the features input to the first feature fusion module;

[0048] Establish residual connections between the input and output of cascaded convolutions;

[0049] The output of the first feature fusion module is represented as follows:

[0050] Fusion(f)=f3(f2(f1(f0)))+f0.

[0051] The construction of the second feature fusion module is similar to that of the first feature fusion module.

[0052] Furthermore, the HDNet model is trained by combining a multi-dimensional loss function with an auxiliary gradient guidance strategy based on stochastic CLAHE.

[0053] The multi-dimensional combined loss function includes: pixel content loss function, structural similarity loss function, and perceptual loss function.

[0054] Furthermore, the multi-dimensional combined loss function, combined with the auxiliary gradient guidance strategy based on stochastic CLAHE, is expressed as:

[0055]

[0056] In the formula, L pcont (·) represents the pixel content loss function, L ssim (·) represents the structural similarity loss function, L per (·) represents the perceptual loss function, I gt Indicates a reference image. I represents the image output by the HDNet model. m This represents the image output by the pixel-by-pixel detail adjustment module; α, β, γ, and η are the corresponding weights. This indicates the reference image enhanced using CLAHE. Let p represent the original, given reference image, p represent the probability of the reference image after CLAHE enhancement, and r represent a random floating-point number between 0 and 1 generated during training.

[0057] This invention proposes an underwater image enhancement system based on hierarchical decomposition and fusion, including a trained HDNet model, and uses the trained HDNet model for underwater image enhancement.

[0058] The HDNet model, in accordance with the image processing procedure, includes the following steps:

[0059] The first convolutional layer is used to expand the channels of the input image;

[0060] The first sub-channel color correction module is used to split the input image into channels and adjust the color distribution of each channel individually;

[0061] The first feature fusion module is used to fuse the colors of each channel;

[0062] The pixel detail adjustment module is used to represent pixels and modulate signals using various wave functions;

[0063] The second feature fusion module is used for further fusion.

[0064] The second sub-channel color correction module is used to adjust the color distribution of each channel to make it normally distributed.

[0065] The second convolution is used to compress channels to output an enhanced image.

[0066] Furthermore, the HDNet model is trained by combining a multi-dimensional loss function with an auxiliary gradient guidance strategy based on stochastic CLAHE. The multi-dimensional loss function includes: pixel content loss function, structural similarity loss function, and perceptual loss function.

[0067] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0068] (1) In view of the insufficient underwater image enhancement capabilities of existing methods, this invention proposes an underwater image enhancement network HDNet based on hierarchical decomposition and fusion. It corrects color deviation through macroscopic channel decomposition and improves detail degradation through microscopic pixel decomposition, fully and meticulously considering the two key factors of image degradation: light attenuation and scattering.

[0069] (2) In terms of pixel decomposition, the present invention innovatively regards the pixels of degraded underwater images as light signals affected by scattering; by utilizing signal representation and modulation, the method can effectively adjust the local features of pixels and enhance image details;

[0070] (3) When constructing the combined loss function, this invention enables the model to learn the enhancement effect of the reference image in multiple dimensions. In addition, it also incorporates an auxiliary gradient guidance strategy based on random CLAHE. This method can enhance the gradient information of the model regarding white balance and contrast during training, and can mitigate the adverse effects of low-quality data to a certain extent, enabling the model to learn better. Attached Figure Description

[0071] Figure 1 This is a schematic diagram illustrating the training of the HDNet model according to the present invention;

[0072] Figure 2 This is a schematic diagram of the structure of each module in the HDNet model proposed in this invention; wherein, Figure 2 (a) in the diagram is a schematic diagram of the sub-channel color correction module. Figure 2 (b) in the diagram is a schematic diagram of the sub-pixel detail adjustment module. Figure 2 (c) in the diagram is a schematic diagram of the feature fusion module structure;

[0073] Figure 3 The graph shows a comparison of the enhancement effects of each model on the V90, U45, and UCCS datasets. Detailed Implementation

[0074] The technical solution of the present invention will now be further described in conjunction with the accompanying drawings and embodiments.

[0075] like Figure 1 As shown in the figure, this embodiment proposes an underwater image enhancement method based on hierarchical decomposition and fusion, which mainly includes the following steps:

[0076] Step 1: Obtain a real underwater image dataset, including training and test sets: The real underwater image datasets used in this embodiment include the UIEB dataset, the U45 dataset, and the UCCS dataset. The UIEB dataset consists of 890 pairs of original real underwater images and their corresponding high-quality reference images. The U45 dataset contains 45 original underwater images demonstrating various degradation types, such as color aberration, insufficient contrast, and blur. The UCCS dataset contains 300 underwater images demonstrating various color aberrations. 800 image pairs are randomly selected from the UIEB dataset as the training set (T800), and the remaining 90 image pairs are used as the test set with reference images (V90). The U45 dataset and the UCCS dataset are used as the test set without reference images.

[0077] Step 2: Data preprocessing: First, the training set is randomly scaled, cropped, and flipped, and then uniformly sized to 256×256.

[0078] Step 3: Construct an underwater image enhancement model based on hierarchical decomposition and fusion, namely the HDNet model: The HDNet model designed in this embodiment enhances images at different levels, with each level employing a decomposition method to address specific problems of underwater image degradation. It follows a processing flow from macro to micro and back to macro, combining modules into a systematic enhancement model. At the macro level, the HDNet model decomposes channels to correct color deviations. At the micro level, the HDNet model treats pixels as light signals, decomposing and modulating them to mitigate image detail degradation. The two intermediate feature fusion operations are used to fuse the decomposed features and improve the model's enhancement effect. Specific design operations include:

[0079] Model structure design: For degraded underwater images D∈R H×W×3 The HDNet model's first layer uses 1×1 point convolutions to expand the input image's channels. Then, the second layer uses a macro-level channel-specific color adjustment module (CAM), which splits the channels and adjusts the color distribution of each channel individually. The third layer uses a fusion module for fusion. The image then enters the fourth layer's micro-level pixel-level detail adjustment module (PAM), which represents pixels using various wave functions and modulates the signal. Next, the fifth layer uses a fusion module for further fusion. Finally, the image enters the sixth layer's macro-level detail adjustment module... Branch A color adjustment module (CAM) is used to ensure that the enhanced colors are normally distributed, i.e., still within a normal distribution state. Finally, after compressing the channels through a 1x1 convolution in the seventh layer, the enhanced image is obtained. In addition, to accelerate the convergence of model parameters, feature maps F are extracted from the fourth layer, i.e., the intermediate layer. m ∈R H×W×C It is reduced to I by 1×1 point convolution. m ∈R H×W×3 To calculate the loss against the reference image. Figure 1 The left half of the diagram shows the overall structure of the HDNet model.

[0080] The entire process can be summarized as follows:

[0081] F m =PAM(Fusion(CAM(Conv) 1×1 (D))))

[0082] I m =Conv 1×1 (F m )

[0083]

[0084] The concept behind the Sub-Channel Color Adjustment (CAM) module is as follows: From a macroscopic perspective, underwater images often appear bluish-green due to the varying degrees of attenuation of different colors of light in water. The module effectively corrects this color cast by splitting the image into its RGB channels and adjusting the color distribution of each channel separately. Specifically, in an underwater environment, the attenuation characteristics of light vary with wavelength. Red light has the longest wavelength and attenuates most rapidly, while blue and green light have shorter wavelengths and attenuate relatively more slowly, leading to abnormal distributions of red, green, and blue. The Sub-Channel Color Adjustment (CAM) module can adjust the distribution of the three channels (blue, red, and green) in an image to restore them to normal. First, the expanded feature map F... in_expand It is broken down into 4 parts, FR F G F B Corresponding to 3 color channels, F CAM_base This provides image content information, serving as the basis for color adjustments. Then, F... R F G F B The color distribution is adjusted by expanding the space to a higher dimension using 1×1 convolutions and then normalizing it using InstanceNorm. Finally, 1×1 convolutions are applied to the F... R F G F B Compress it back to its original spatial dimension. Then combine the enhanced information from the three channels with the image content F. CAM_base The components are added together to improve the foundation of color adjustments. Finally, the four parts are merged into channels to create new features. Figure 2 Part (a) shows the specific structure of the Subchannel Color Adjustment (CAM) module. It can be expressed mathematically as follows:

[0085] F′ color =Conv 1×1 (InstanceNorm(Conv 1×1 (F color ))), color∈{R,GB}

[0086] F′ CAM_base =F R '+F G '+F b '+F CAM_base

[0087] F CAM_out =concat([F R ',F G ',F b ',F C ' AM_base ])

[0088] The Pixel Detail Adjustment (PAM) module is designed based on the principle that underwater image pixels are formed from light signals captured by a camera. At a microscopic level, these pixels can be considered as light signals affected by light scattering. The module effectively optimizes image details by characterizing degraded pixels, decomposing them into linear combinations of different signal waves, and modulating the signals. Specifically, pixels are the basic units that constitute image details; they are obtained by sampling, quantizing, and encoding the captured light signals by the camera. In the underwater environment, this process is affected by light scattering, leading to reduced image contrast and sharpness. Therefore, this embodiment represents underwater image pixels as light signals affected by scattering interference and improves image details through pixel signal characterization and signal modulation. Following the principle of Fourier decomposition of signals using sine and cosine functions, the PAM module decomposes the signal into combinations of various waveforms, including sine (sin), cosine (cosine), and hyperbolic tangent (tanh). The first two are periodic functions, while the added hyperbolic tangent (tanh) is a non-periodic function, enriching the representation of pixels. Wave function F wave The representation requires phase and amplitude, which can be obtained through a series of convolution operations. It can be expressed mathematically as:

[0089] F wave =Conv 1×1 (F PAM_in )·wave(Conv 1×1 (F PAM_in ))

[0090] wave∈{sin,cos,tanh}

[0091] In underwater image processing, pixel adjustments typically require consideration of the contextual information of the region in which the pixel resides. This approach helps the module more accurately optimize image details, thereby improving the clarity and visual quality of underwater images. Therefore, this embodiment uses convolution operations of different scales on the wavefunction to obtain contextual information. To preserve the structure and texture of the image, each wave feature F... wave It is also necessary to fuse the information F from the input features. PAM_in Its fused wave characteristics The mathematical formula can be expressed as:

[0092] F PAM_base =Conv 1×1 (F PAM_in )

[0093] F w ' ave =Conv k×k (F wave )

[0094]

[0095] Among them, for F wave , including F cos F sin F tanh k is the size of the convolution kernel. To obtain multi-scale contextual information, k is set to 5, 7, and 9 in this embodiment. Finally, the signal representation of the image pixels can be expressed mathematically as:

[0096]

[0097] Figure 2 Part (b) shows the specific structure of PAM.

[0098] Fusion Module Design: Although fusion operations are included in the Multi-Channel Color Adjustment (CAM) and Multi-Pixel Detail Adjustment (PAM) modules, the granularity of the fusion is still relatively coarse. Therefore, this embodiment uses a dense residual connection structure to fuse the decomposed features layer by layer in a more refined manner, to better fuse and enhance the effects of these two decomposition enhancements. Its dense connections can combine features from each previous layer layer by layer for a more delicate feature fusion. The Fusion module initiates the fusion process with a 3×3 convolution, followed by a PReLU activation function. Subsequently, a cascade of three 3×3 convolutions is employed. In this cascade, the result of each convolutional layer is combined with the results of all previous layers, and then the combined features are input into the next convolutional stage. The output f of each layer in the cascaded convolution is... l This can be represented by the following equation:

[0099] f0 = PReLU(Conv) 3×3 (F Fusion_in ))

[0100] f l =PReLU(Conv 3×3 ([f l-1 ,f l-2 ,...,f0]))

[0101] In the formula, F Fusion_in This represents the features input to the first feature fusion module;

[0102] To enhance the learning process, residual connections are established between the input and output of the cascaded convolutions. The fusion module can ultimately be represented as:

[0103] Fusion(f) = f3(f2(f1(f0))) + f0

[0104] Figure 2Part (c) shows the specific structure of Fusion.

[0105] Step 4: Loss Function Design: To enhance the learning effect of the reference image in a multi-dimensional way for the deep learning model, this invention uses the pixel content loss function L. pcont That is, pixel-based L1 loss, structural similarity loss function L ssim That is, SSIM loss, and the perceptual loss function L per These are the losses calculated based on the feature maps of the classification model. These three losses allow the model to comprehensively learn the pixel details, structural features, and overall content of the reference image. To accelerate the convergence of the parameters in the first half of the model, this embodiment calculates these losses at the end of the model, while also calculating the perceptual loss function in the middle layers. In this embodiment, the perceptual loss function uses the "relu1_2" and "relu2_2" layer features from VGG16 to calculate the mean squared error loss.

[0106] To mitigate the negative impact of suboptimal reference images on the model learning process, this embodiment incorporates a stochastic CLAHE-based auxiliary gradient guidance strategy in addition to the three training losses. The stochastic CLAHE-based auxiliary gradient guidance strategy randomly applies the Contrast Limiting Adaptive Histogram Equalization (CLAHE) algorithm to the reference image to enhance its contrast and balance illumination. This strategy strengthens the model's gradients in these aspects, thereby mitigating the adverse effects of suboptimal reference images. The complete loss function calculation formula is as follows:

[0107]

[0108] Where α, β, γ, and η are the weights of each loss; I gt This is a reference image. It is the model predicting the image, I m It is the image output by the fourth layer of the model, that is, the intermediate layer. The perceptual loss between the intermediate layer and the reference image is calculated to speed up the convergence of the parameters in the first half of the model. This is the reference image enhanced using CLAHE (contrast-limited histogram equalization). is the original, given reference image, p is the probability of using CLAHE to enhance the reference image, and r is a random floating-point number between 0 and 1 generated randomly during model training. Figure 1 The right half of the diagram illustrates the use of a stochastic CLAHE-based auxiliary gradient guidance strategy during model training.

[0109] Step 5: Evaluation Metric Selection: To assess the enhancement effect of the model, this invention uses PSNR, SSIM, and UCIQE as evaluation metrics. PSNR is a pixel-based evaluation metric with a reference image, with values ​​ranging from 0 to +∞. SSIM comprehensively evaluates the similarity of an image's brightness, structure, and contrast to a reference image, with values ​​ranging from 0 to 1. UCIQE is specifically designed for underwater images, quantitatively evaluating image quality through a linear combination of chroma, saturation, and contrast, with values ​​ranging from 0 to 1. Higher values ​​for these metrics generally indicate better enhancement.

[0110] Step 6: Model Training: Train the HDNet model using the T800 training set with uniform image size. Select appropriate hyperparameters based on factors such as hardware configuration, data size, and data characteristics. Specifically:

[0111] Experimental environment: The model experiment in this embodiment was conducted on a device running Ubuntu 20.04, which was equipped with an Intel i5-12400F CPU, 12GB of memory and an RTX3060 GPU. The software environment consisted of PyTorch 1.12.1, Python 3.7 and CUDA 11.3.

[0112] Parameter settings: Adam was selected as the optimizer, with an initial learning rate of 1e-3 and beta parameters of 0.9 and 0.999. The batch size was set to 16, training was conducted for 400 epochs, and the loss weights α, β, γ, and η were 1.0, 0.2, 0.5, and 0.2, respectively. The probability p in the auxiliary gradient guidance strategy based on stochastic CLAHE was 0.4.

[0113] Dataset Import and Model Training: Import the training set T800 (processed in step 2) into the HDNet model and train the model to fit the provided reference image. When the model parameters converge, save the model parameters with the best enhancement effect based on the PSNR metric of the model on the test set V90.

[0114] Step 7: Model Performance Evaluation: Enhance and visualize the original images from the V90, U45, and UCCS test sets using the trained HDNet model. For the V90 test set with reference images, evaluate using PSNR, SSIM, and UCIQE metrics. For the U45 and UCCS test sets without reference images, evaluate using the UCIQE metric.

[0115] This embodiment designs an underwater image enhancement model based on hierarchical decomposition and fusion. At the macroscopic level, it performs channel-level splitting and color distribution adjustment followed by detailed fusion; at the microscopic level, it performs pixel-level signal decomposition and modulation followed by detailed fusion. This alleviates color distortion caused by light attenuation and detail blurring caused by light scattering, respectively. A combination function is used to allow the model to learn the enhancement effect of the reference image from multiple dimensions. Furthermore, a stochastic CLAHE-based auxiliary gradient guidance strategy provides additional auxiliary gradients for model training, which can reduce the adverse effects of suboptimal reference images on the model learning process to some extent. This embodiment uses the UIEB, U45, and UCCS datasets to train and test the designed model. Quantitative comparison results with other methods are shown in Table 1, and visualization comparisons are shown below. Figure 3 As shown in Table 1 and Figure 3 It can be seen that the enhancement effect of the method in this embodiment is better than that of the comparative method.

[0116] Table 1

[0117] Network Model V90 / PSNR V90 / SSIM U45 / UCIQE UCCS / UCIQE Shallow-UWnet 18.062 0.796 0.518 0.483 UHD-SFNet 23.258 0.868 0.603 0.536 UIEC^2-Net 20.497 0.898 0.597 0.543 HDNet 24.158 0.923 0.610 0.564

Claims

1. An underwater image enhancement method based on hierarchical decomposition and fusion, characterized in that: include: Construct the HDNet model; Train the HDNet model; Underwater image enhancement was performed using the trained HDNet model; The constructed HDNet model, following the image processing procedure, includes the following steps: The first convolutional layer is used to expand the channels of the input image; The first sub-channel color correction module is used to split the input image into channels and adjust the color distribution of each channel individually; The first feature fusion module is used to fuse the colors of each channel; The pixel detail adjustment module is used to represent pixels and modulate signals using various wave functions; The second feature fusion module is used for further fusion. The second sub-channel color correction module is used to adjust the color distribution of each channel to make it normally distributed. The second convolution is used to compress channels to output an enhanced image.

2. The underwater image enhancement method based on hierarchical decomposition and fusion according to claim 1, characterized in that: The first sub-channel color correction module and the second sub-channel color correction module are both constructed according to the following steps: The feature map is split to obtain three color channel information F. R F G F B and image content information F CAM_base ; F R F G F B Each dimension is expanded to a higher dimension through 1×1 convolution, and then the color distribution is adjusted using InstanceNorm normalization. Then, the three color channel information is compressed back to the original spatial dimension through 1×1 convolution to obtain the enhanced three color channel information; The enhanced three color channel information and image content information F CAM_base The two are added together to obtain the enhanced image content information. Finally, the enhanced three color channel information and the enhanced image content information are merged to form a new feature.

3. The underwater image enhancement method based on hierarchical decomposition and fusion according to claim 2, characterized in that: The first sub-channel color correction module is represented as: F′ color =Conv 1×1 (InstanceNorm(Conv 1×1 (F color ))),color∈{R,G.B} F′ CAM_base =F′ R +F′ G '+F b '+F CAM_base F CAM_out =concat([F′ R ,F′ G ,F′ b ,F′ CAM_base ]) The second sub-channel color correction module is similar to the first sub-channel color correction module.

4. The underwater image enhancement method based on hierarchical decomposition and fusion according to claim 1, characterized in that: In the pixel-level detail adjustment module, various wave functions are used to represent pixels and modulate signals, specifically including: Based on the optical principles of camera image acquisition, the pixels of underwater images are treated as light signals affected by light scattering. The pixels are then adjusted according to the principle of Fourier decomposition of signals using sine and cosine functions, including: The optical signal is decomposed into a combination of different signal wave characteristics, which are represented as follows: F wave =Conv 1×1 (F PAM_in )·wave(Conv 1×1 (F PAM_in ))wave∈{sin,cos,tanh} In the formula, F wave F represents the characteristics of the signal wave. PAM_in This indicates the features input to the pixel-level detail adjustment module; Convolutional operations at different scales are applied to the signal wave features to obtain contextual information; Each signal wave characteristic and its input to the pixel-level detail adjustment module F PAM_in Fusion, yielding the fused wave characteristics Represented as: F PAM_base =Conv 1×1 (F PAM_in ) F w ' ave =Conv k×k (F wave ) F fusewave =F PAM_base +F w ' ave In the formula, k is the size of the convolution kernel; The feature representation output by the pixel-level detail adjustment module is as follows:

5. The underwater image enhancement method based on hierarchical decomposition and fusion according to claim 1, characterized in that: Both the first feature fusion module and the second feature fusion module adopt a dense residual connection structure.

6. The underwater image enhancement method based on hierarchical decomposition and fusion according to claim 1, characterized in that: The first feature fusion module is constructed according to the following steps: The fusion process is initiated by a 3×3 convolution, followed by the PReLU activation function; Subsequently, a concatenation of three 3×3 convolutions is employed. In this concatenation, the result of each convolutional layer is combined with the results of all previous layers, and the combined features are then input into the next convolutional stage; the output f of each layer in the concatenated convolution... l Represented as: f0=PReLU(Conv 3×3 (F Fusion_in )) f l =PReLU(Conv 3×3 ([f l-1 ,f l-2 ,...,f0])) In the formula, F Fusion_in This represents the features input to the first feature fusion module; Establish residual connections between the input and output of cascaded convolutions; The output of the first feature fusion module is represented as follows: Fusion(f) = f3(f2(f1(f0))) + f0 The construction of the second feature fusion module is similar to that of the first feature fusion module.

7. The underwater image enhancement method based on hierarchical decomposition and fusion according to claim 1, characterized in that: The HDNet model is trained by combining a multi-dimensional loss function with an auxiliary gradient guidance strategy based on stochastic CLAHE. The multi-dimensional combined loss function includes: pixel content loss function, structural similarity loss function, and perceptual loss function.

8. The underwater image enhancement method based on hierarchical decomposition and fusion according to claim 7, characterized in that: The multi-dimensional combined loss function, combined with an auxiliary gradient guidance strategy based on stochastic CLAHE, is expressed as: In the formula, L pcont (·) represents the pixel content loss function, L ssim (·) represents the structural similarity loss function, L per (·) represents the perceptual loss function, I gt Indicates a reference image. I represents the image output by the HDNet model. m This represents the image output by the pixel-by-pixel detail adjustment module; α, β, γ, and η are the corresponding weights. This indicates the reference image enhanced using CLAHE. Let p represent the original, given reference image, p represent the probability of the reference image after CLAHE enhancement, and r represent a random floating-point number between 0 and 1 generated during training.

9. An underwater image enhancement system based on hierarchical decomposition and fusion, characterized in that: This includes a trained HDNet model, which is used for underwater image enhancement. The HDNet model, in accordance with the image processing procedure, includes the following steps: The first convolutional layer is used to expand the channels of the input image; The first sub-channel color correction module is used to split the input image into channels and adjust the color distribution of each channel individually; The first feature fusion module is used to fuse the colors of each channel; The pixel detail adjustment module is used to represent pixels and modulate signals using various wave functions; The second feature fusion module is used for further fusion. The second sub-channel color correction module is used to adjust the color distribution of each channel to make it normally distributed. The second convolution is used to compress channels to output an enhanced image.

10. The underwater image enhancement system based on hierarchical decomposition and fusion according to claim 9, characterized in that: The HDNet model is trained by combining a multi-dimensional loss function with an auxiliary gradient guidance strategy based on stochastic CLAHE. The multi-dimensional loss function includes: pixel content loss function, structural similarity loss function, and perceptual loss function.

Images