Color correction and detail unlocking underwater image enhancement method based on frequency decoupling

Abstract

The invention discloses a color correction and detail unlocking underwater image enhancement method based on frequency decoupling. The method comprises the following steps: performing multi-frequency domain decomposition on an underwater image to obtain a low-frequency component and a high-frequency component; inputting the high-frequency component into a high-frequency detail unlocking branch, fusing frequency domain selection attention and channel attention mechanisms, performing fine-grained modeling on high-frequency texture features, strengthening real details, and suppressing high-frequency noise at the same time; inputting the low-frequency component into a low-frequency enhancement branch, carrying out global modeling and enhancement on the low-frequency feature by adopting a self-attention mechanism, and adjusting the overall brightness distribution, color saturation and structural information of the image; and carrying out fusion reconstruction on the output features of the low-frequency enhancement branch and the high-frequency detail unlocking branch to realize enhancement.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and relates to underwater imaging technology, and in particular to an underwater image enhancement method based on frequency decoupling color correction and detail unlocking. Background Technology

[0002] Underwater imaging technology plays a crucial role in marine exploration and research. However, the complex underwater environment brings serious image degradation problems, including color shift, low contrast, and haze effects, which greatly limit the effective conduct of underwater missions.

[0003] The degradation of underwater image quality primarily stems from the absorption and scattering of light during its propagation in water. When light enters water from the air, it undergoes absorption and scattering. Because red light has a longer wavelength, it is absorbed first, resulting in the characteristic blue-green hue of underwater images. Furthermore, scattering caused by suspended particles in the water, including forward and backscattering, further degrades image quality. To address these challenges, underwater image enhancement (UIE) technology has emerged.

[0004] In recent years, deep learning has made significant progress in the application of underwater image enhancement (UIE). Research in the field of underwater image enhancement has shown a vigorous development trend, with numerous methods emerging. These methods have made significant contributions to improving the quality of underwater images, making them more suitable for various application scenarios such as underwater detection, monitoring, and target recognition. These methods can be broadly classified into three categories: physical model-based methods, non-physical model-based methods, and deep learning-based methods. Physical model-based methods generally rely on accurate estimation of prior knowledge such as water parameters, lighting information, and even camera pose, and require multiple images or external equipment for acquisition, limiting their practicality. While non-physical model-based methods avoid dependence on complex imaging parameters and improve applicability and computational efficiency, most methods focus on image domain processing, making it difficult to accurately distinguish feature attenuation at different frequency levels. Among deep learning-based methods, Li et al. proposed UWCNN, which uses residual learning, dense connections, and SSIM loss to reconstruct clear underwater images on synthetic datasets. Peng et al. constructed a large-scale real underwater image dataset and proposed a U-shaped Transformer network to improve color restoration and image quality through the fusion of channel and spatial features. Wang et al.'s UIEC2-Net is the first deep network to integrate RGB and HSV color spaces, improving the subjective quality and objective metrics of underwater images through a multi-branch structure that jointly processes color, brightness, and saturation. Li et al.'s WaterGAN estimates and utilizes depth information to enhance underwater images. Cong et al.'s PUGAN combines a physical model of underwater imaging with a generative adversarial network, achieving good results. Qi et al.'s SGUIE-Net, a semantic attention-guided underwater image enhancement network, enhances local features of semantic regions with multi-scale perception through a semantic region enhancement module. Kang et al.'s Perception-Driven Decomposition and Fusion Framework SPDF fuses preprocessed images in a perception- and concept-independent image space, decomposing them into average intensity, contrast, and structural components, and fusing them to improve image quality. Zhou et al.'s Hybrid Contrast Learning Regularization method utilizes unpaired positive and negative samples to improve model generalization, combining an adaptive hybrid attention module and a detail restoration branch to effectively extract features and restore texture details. Qi et al. proposed a collaborative enhancement network that uses a relevant feature matching unit to enhance image quality. Li et al. proposed a human-perceived quality-driven framework that embeds a depth map before spatial attention blocks to adaptively recalibrate the quality-perceived feature map. Fu et al.'s PUIE-Net introduces uncertainty modeling into a probabilistic network for underwater image augmentation. This method is based on a conditional variational autoencoder and adaptive instance normalization, constructs an augmentation distribution, and derives the final result through a consensus mechanism.Qi et al. proposed a deep color correction multi-scale Retinex network, CCMSR-Net, which achieves color correction and visibility enhancement of underwater images through a color correction subnetwork and a multi-scale Retinex subnetwork, based on a hybrid convolutional-axial attention block. Tolie et al. proposed a deep Inception and channel attention module, which improves the quality of underwater images and corrects color bias through channel-level color restoration and color correction stages, based on the Inception and channel attention modules. Zhang et al. proposed a lightweight real-time single underwater image enhancement network, LiteEnhanceNet, which uses depthwise separable convolutions as its base module and achieves underwater image quality enhancement and real-time processing through one-time aggregation connections and squeeze excitation modules. Chen et al. proposed a semi-supervised multi-scale network, UWFormer, which achieves multi-frequency enhancement and visibility enhancement of underwater images through discrete wavelet transform and inverse transform, based on a nonlinear frequency-aware attention module and a multi-scale fusion feedforward network.

[0005] Despite their superior performance, these methods still suffer from the following problems: (1) Most methods only model in the spatial domain, neglecting the important role of the frequency components of underwater images in color and detail restoration. (2) It is difficult to balance global color restoration and local detail enhancement of underwater images, resulting in over-processing or loss of detail in some areas. Summary of the Invention

[0006] In order to overcome the shortcomings of the prior art, the present invention aims to provide a color correction and detail unlocking underwater image enhancement method based on frequency decoupling, so as to solve the problem that traditional frequency division processing methods only use shallow branches and lack explicit frequency feature decoupling modeling and targeted enhancement strategies, which makes it difficult to fully explore and utilize the potential information of different frequency components.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A color correction and detail unlocking underwater image enhancement method based on frequency decoupling includes the following steps: Step 1: Perform multi-frequency domain decomposition on the underwater image to obtain low-frequency and high-frequency components; Step 2: Input the high-frequency components into the high-frequency detail unlocking branch, fuse the frequency domain selection attention and channel attention mechanisms, perform fine-grained modeling of high-frequency texture features, enhance realistic details, and suppress high-frequency noise at the same time; Step 3: Input the low-frequency component into the low-frequency enhancement branch, and use a self-attention mechanism to globally model and enhance the low-frequency features, adjusting the overall brightness distribution, color saturation and structural information of the image; Step 4: The output features of the low-frequency enhancement branch and the high-frequency detail unlocking branch are fused and reconstructed to achieve enhancement.

[0008] In one embodiment, step 1 decomposes the underwater image into low-frequency components and multiple high-frequency components using discrete wavelet transform; step 2 merges the multiple high-frequency components into a multi-channel image as input to the high-frequency detail unlocking branch; and step 4 fuses and reconstructs the output features using inverse discrete wavelet transform.

[0009] In one embodiment, the high-frequency detail unlocking branch performs the following steps: Step 21: Extract features using the backbone network; Step 22: The extracted features are fed into the detail unlocking module, and the following steps are performed: First, a basic feature representation F is constructed, and then F is passed through the channel attention module to generate a channel attention map. Then F and Feature fusion is performed to obtain weighted features, and then the weighted features are used to generate a spatial attention map through a spatial attention module. Then combine the weighted features with Feature fusion is performed to obtain joint enhanced features. Finally The extracted features are fused with residual connections and enhanced by convolution and ReLU activation to obtain the output features of the high-frequency detail unlocking branch.

[0010] In one embodiment, the feature extraction using the backbone network is implemented as follows: Step 211: Perform preliminary feature extraction on the input high-frequency components through convolutional layers; Step 212: The initially extracted features are fed into a residual module composed of multiple fast Fourier convolutions to capture global texture patterns and minute structural details in the frequency domain. Step 213: The output of the residual module is compressed and fused through convolution and batch normalization layers, and then added to the initially extracted features through skip connections to obtain the extracted features.

[0011] In one embodiment, step 22 involves constructing a basic feature representation F from the extracted features using a set of convolutional layers, BatchNorm layers, and ReLU activation layers; feature fusion is achieved through element-wise multiplication; and enhanced decoding is achieved through an enhanced decoding structure consisting of three convolutional layers and ReLU activation.

[0012] In one embodiment, the low-frequency enhancement branch performs the following steps: Step 31: Map the low-frequency components to a low-dimensional feature space through image segmentation and linear embedding to obtain a low-level feature representation. ; Step 32, The data is fed into a four-layer encoding module. Each encoding layer contains a Transformer structure and spatial downsampling operations to gradually extract high-level semantic features. The first encoding layer uses pixel self-attention, the second and third encoding layers use channel self-attention, and the fourth encoding layer uses shift window self-attention. Each encoding layer introduces channel and spatial attention mechanisms. Step 33: In the decoding stage, the high-level semantic features are gradually restored to spatial details through a three-layer upsampling structure, and the color information and structural semantics in the encoding process are introduced into the decoding stream through skip connections. Color refinement and structural unification are performed in combination with the encoding layer structure. Step 34: The decoding result is fused with the low-frequency features of the backbone and the high-frequency residual information through the refining module to obtain the output features of the low-frequency enhancement branch.

[0013] In one embodiment, the four-layer encoding module comprises, in order: The first coding layer uses TransBlock, which is an improvement on the Swing Transformer. The second and third coding layers both use channel self-attention Transformer modules, and a lightweight convolutional attention module is introduced before the module input; The fourth encoding layer uses LatentBlock to model the entire image without using sliding windows, achieving global color consistency modeling with low computational complexity.

[0014] Compared with the prior art, the beneficial effects of the present invention are: (1) To address the problem that existing methods only model in the spatial domain and ignore the important role of frequency components of underwater images in color and detail restoration, we propose an underwater image enhancement network DDFCEnhanceNet based on frequency decoupling. It performs multi-frequency domain decomposition on the input image through discrete wavelet transform, separates global structural information and local detail information, and combines inverse wavelet transform to achieve feature reconstruction.

[0015] (2) To address the issues of color distortion, low contrast, and uneven illumination in underwater images, a low-frequency enhancement branch, CCFormer, was designed. Various self-attention mechanisms were used to globally model and enhance low-frequency features, thereby adjusting the overall brightness distribution, color saturation, and structural information of the image.

[0016] (3) To address the issues of blurring and loss of detail in underwater images, a high-frequency detail unlocking branch FFSCANet was designed. It integrates frequency domain selection attention and channel attention mechanisms to perform fine-grained modeling of high-frequency texture features, enhance realistic details, and effectively suppress high-frequency noise. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the overall architecture of the DFCEnhanceNet of the present invention.

[0018] Figure 2 This is a schematic diagram of the architecture of the Detail Recovery Module (DRM) of this invention.

[0019] Figure 3 This is the architecture of the CATBlock of this invention.

[0020] Figure 4 This is the architecture of CSWBlock in this invention.

[0021] Figure 5 This is a comparison of the visualization effects of L31 in the embodiments of the present invention.

[0022] Figure 6 This is a comparison of the visualization effects of Test-U90 in the embodiments of the present invention.

[0023] Figure 7 This is a comparison of the magnified visualization details of U45 in the embodiments of the present invention.

[0024] Figure 8 Visualization of the ablation results of the core component of this invention (from U45). (a) input (b) only CATBlock (c) only DRM (d) only SWABlock (e) CATBlock and SWABlock (f) DFCEnhanceNet. Detailed Implementation

[0025] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings and examples.

[0026] Underwater images are frequently affected by selective absorption and scattering of light, leading to problems such as blurring, color shift, detail degradation, and noise interference. Unlike other degradation scenarios such as fog or low light, underwater image degradation is more complex and involves more extensive frequency domain perturbations. The low-frequency components of underwater images mainly reflect the image's structure and color distribution, while the high-frequency components are rich in texture details and edge information.

[0027] To address this, this invention proposes a frequency-decoupled underwater image enhancement network, DFCEnhanceNet. After decomposing the image into low-frequency and high-frequency components, a low-frequency branch, CCFormer (Color Correction Transformer), modeling global color information, is designed for each frequency component. A high-frequency branch, FFSCANet (FFC Space and Channel Attention Net), combining Fast Fourier Residual Blocks and spatial channel attention, is also designed, focusing on texture detail restoration. Experiments show that the proposed method outperforms existing methods on multiple publicly available underwater image datasets, significantly improving color restoration, contrast, and detail preservation performance. Ablation experiments further validate the effectiveness of the frequency decoupling strategy and the design of each sub-module.

[0028] Combination such as Figure 1 The overall structure of DFCEnhanceNet shown in the figure, and the underwater image enhancement method of the present invention mainly includes the following steps: Step 1: Perform multi-frequency domain decomposition on the underwater image to obtain low-frequency and high-frequency components.

[0029] In this embodiment, the input underwater image can be effectively decomposed into a low-frequency component (LL) and multiple high-frequency components (LH, HL, and HH in this embodiment) using DWT, thereby achieving targeted processing of different frequency features.

[0030] Specifically, for the input underwater image X Discrete wavelet transform is performed to obtain three high-frequency images {LH, HL, HH}. and a low-frequency image LL .

[0031] Step 2: Input the high-frequency component into the high-frequency detail unlocking branch, fuse the frequency domain selection attention and channel attention mechanisms, perform fine-grained modeling of high-frequency texture features, enhance realistic details, and suppress high-frequency noise.

[0032] In this embodiment of the invention, the three high-frequency images LH, HL, and HH obtained in step 1 are merged into a 9-channel image High-Frequency and fed into the high-frequency detail unlocking branch FFSCANet.

[0033] Underwater images are often affected by light scattering and refraction during acquisition, leading to problems such as blurring and loss of detail. This degradation manifests primarily as attenuation and distortion of high-frequency components in the frequency domain. These high-frequency components typically carry minute but crucial details, significantly impacting image clarity and recognizability. To address this, this invention designs a high-frequency enhancement subnetwork—FFSCANet—based on frequency domain modeling. It integrates Fast Fourier Convolution (FCC) and a Detail Restoration Module (DRM) specifically designed to enhance high-frequency components such as texture details in underwater images.

[0034] The high-frequency detail unlocking branch of this invention performs the following main steps: Step 21, extracting features using the backbone network, specifically including: Step 211: Perform preliminary feature extraction on the high-frequency components of the input through a convolutional layer with a large receptive field.

[0035] Step 212: The initially extracted features are fed into a residual module composed of multiple fast Fourier convolutions in order to effectively capture global texture patterns and minute structural details in the frequency domain.

[0036] Step 213 involves compressing and fusing the output of the residual module through convolution and batch normalization layers, and then adding it to the initially extracted features via skip connections. This enhances the feature representation while preserving key original information, gradually unlocking rich details in the high-frequency components of the underwater image. This step yields the extracted features.

[0037] Step 22: After the backbone network features are extracted, the extracted features are sent to the detail unlocking module to further restore the texture details and edge structures in the high-frequency information.

[0038] like Figure 2 As shown, DRM performs the following steps: Using the extracted features, a basic feature representation F is first constructed through a set of convolutional layers, BatchNorm, and ReLU activation. To enhance the discriminative power of the features in the channel dimension, F is fed into a channel attention module to generate a channel attention map. This highlights key channels containing rich texture information and suppresses redundant or noisy channels; then, F is compared with... Element-wise feature fusion is performed to obtain weighted features. Then, to enhance the perception of local structures and edge regions, the weighted features are fed into the spatial attention module to generate a spatial attention map. The responses at key spatial locations are extracted to significantly enhance blurred texture regions. Then, the weighted features are combined with... Element-wise feature fusion yields joint enhanced features. This allows for precise location and enhancement of texture detail areas. Finally, to preserve low-level information and promote gradient propagation, thereby enhancing detail reproduction capabilities, [the following steps are taken]. The extracted features are fused with residual connections and enhanced decoding is performed through an enhanced decoding structure composed of three layers of convolution and ReLU activation to further improve nonlinear modeling capabilities, so as to achieve detailed reconstruction of minute textures in high-frequency information. After this, the output features of the high-frequency detail unlocking branch are finally obtained.

[0039] Step 3: Input the low-frequency components into the low-frequency enhancement branch, and use a self-attention mechanism to perform global modeling and enhancement of the low-frequency features, adjusting the overall brightness distribution, color saturation and structural information of the image.

[0040] In this embodiment of the invention, the high-frequency image LL obtained in step 1 is fed into the low-frequency enhancement branch CCFormer. LLs of different resolutions are subjected to feature extraction and then fed into the encoder and decoder in a top-down manner. Due to the prevalent degradation issues such as color distortion and low contrast in underwater images, especially those concentrated in low-frequency components, this invention designs the CCFormer module to better restore the contrast and color distribution in underwater images. This module employs a deep stacking structure of multi-resolution inputs and Transformers, using a coarse-to-fine approach to progressively improve the color consistency and contrast representation of the image in the low-frequency space. Compared to traditional Transformers, CCFormer not only integrates contextual information from different resolutions but also introduces pixel self-attention, shift window self-attention, and channel self-attention mechanisms, enabling multi-dimensional and multi-layered global and local interactions.

[0041] The whole process is as follows Figure 1 As shown in the green section, the low-frequency component is received at the input of CCFormer. The main steps are as follows: Step 31: Using an overlapping Patch Embedding module, image segmentation and linear embedding operations are employed to map the low-frequency image to a low-dimensional feature space, resulting in a low-level feature representation. Where C is the number of channels and H×W is the spatial dimension.

[0042] Step 32, feature The data is fed into a four-layer encoding module, where each encoding layer contains a Transformer structure and spatial downsampling operations to gradually extract high-level semantic features.

[0043] In the encoder design, to model the response differences of color channels locally and highlight key color features such as weak red channels in underwater images, the first encoding layer adopts TransBlock, an improvement based on Swing Transformer, which introduces pixel self-attention. The second and third encoding layers use CATBlock, which uses channel self-attention to further enhance the information interaction between channels and improve the discriminativeness of color correction by guiding the saliency of color channels. The fourth layer uses LatentBlock, employing shifted window self-attention to model the entire image without using sliding windows, achieving global color consistency modeling with low computational complexity and preventing regional unevenness or artifacts in color enhancement. Simultaneously, each encoding layer of this invention introduces channel and spatial attention mechanisms.

[0044] The following describes the specific structure and working process of each coding layer.

[0045] 1) CATBlock (Channel self-Attention Transformer Block).

[0046] Due to severe color channel imbalance and information degradation in underwater images, traditional spatial dimension-based self-attention mechanisms struggle to effectively model cross-channel feature dependencies, leading to inaccurate color restoration and insufficient structure perception. To address this, CATBlock was designed to model feature relationships from a channel perspective, enhancing global context awareness between channels. This results in more effective color correction and improved modeling capabilities for the overall image structure.

[0047] To improve the accuracy of the initial feature representation, a lightweight CBAM (Convolutional Block Attention Module) is introduced before the input of the channel self-attention Transformer module of CATBlock. CBAM enhances key feature regions and suppresses redundant information through the concatenation of channel attention and spatial attention.

[0048] Therefore, the execution process of CATBlock is as follows: Given input feature map The CBAM processing procedure is as follows: in and These represent the channel attention and spatial attention modules, respectively. This represents element-wise multiplication, and the output is the enhanced feature. .

[0049] Channel attention and spatial attention The calculation process is as follows: Here and This refers to the inputs for channel attention and spatial attention.

[0050] Based on this output, a multi-head channel self-attention (CSA) mechanism is used to model long-range dependencies in the channel dimension. For example... Figure 3 As shown, the query (Q), keyword (K), and value (V) projections are first generated in channel self-attention. To reduce computational complexity and enable the network to adapt to the size of the input image, convolutions are embedded in CAT. Then, Q, K, and V, the query, keyword, and value for channel self-attention, are generated in channel self-attention, as follows: Where conv is a 1×1 pointwise convolution operation, dconv is a 3×3 depthwise convolution operation, and channel self-attention is also included. It can be defined as follows: Where Q, K, V ∈ These are the query, key, and value for channel self-attention. After rearranging the attention results back to their original spatial dimensions, a 1×1 convolution is used for channel fusion to obtain the output features of the channel self-attention Transformer module. : in, This indicates that the attention results will be rearranged.

[0051] Finally, CATBlock is expressed as a typical Transformer structure, including residual connections, layer normalization, and feedforward network (FFN): Where LN represents layer normalization and FFN represents feedforward sublayer, which is used to further enhance the ability to represent nonlinear features.

[0052] 2) CSWBlock (Channel and Shifted Window self-Attention TransformerBlock).

[0053] While CATBlock effectively captures global contextual information from the channel dimension to improve color correction and structure perception capabilities, it neglects spatial non-uniformity modeling at the pixel level, thus losing the ability to model local information. To further enhance the model's ability to perceive and recover spatially degraded regions, this invention introduces CSWBlock.

[0054] Traditional Transformers typically compute self-attention globally, resulting in high computational complexity. However, by employing a method of computing self-attention within local windows, dividing the image into non-overlapping uniform windows, not only is computational complexity significantly reduced, but the model's scalability is also improved. Assuming each window contains M × M patches, the computational complexity of the global MSA module and windows based on h × w patch images is: The former has a computational complexity of hw squared, while the latter is linear when M is fixed. Window-based self-attention has lower computational complexity and higher scalability compared to global self-attention. Here, h and w represent the height and width of the original image, respectively, C represents the number of channels, and M represents the size of each image patch, with h and w for each patch being M.

[0055] The proposed CSWBlock structure is as follows: Figure 4 As shown, an alternating shift window partitioning strategy is adopted. While maintaining the linear growth of computational complexity with spatial resolution, pixel self-attention is calculated within local non-overlapping windows, and a cross-window information interaction mechanism is introduced to capture the local and global correlations of the image more efficiently.

[0056] CSWBlock, similar to CATBlock, first processes the input feature map... Features are obtained through preliminary channel spatial enhancement using spatial channel attention. .exist Figure 4 It can be observed that the calculation of Q, K, and V in CSWBlock is similar to that in CATBlock.

[0057] Pixel self-attention can be defined as follows: Where S represents the input feature map The resolution of the CSWBlock module can be defined as follows: Where LN is the layer norm, FFN is the feedforward network, RW is the regular window, WR is the window inversion, and SW is the shifted window. It is an intermediate feature used in calculation.

[0058] TransBlock is an adaptive fusion of CSWBlock and CATBlock, and its output is: in These are the weight parameters.

[0059] like Figure 4 As shown, pixel self-attention is computed within a rule window, transforming the rule window into a shift window in CSWBlock. Similar to Swing, masking operations are used in CSWBlock. Furthermore, since the feature resolution is significantly reduced at the bottleneck layer, pixel self-attention can be computed directly without windowing the input features.

[0060] Based on the above method, CSWBlock uses shifted window self-attention to implement pixel-to-pixel attention operations within a local window, which is a type of local pixel self-attention. Attention weights are calculated for each pixel within the local window, enabling pixel dependency modeling within a local scope.

[0061] If you do not perform a windowing operation on the input features, but instead directly compute pixel attention on the input features, then TransBlock will become LatentBlock.

[0062] Specifically, LatentBlock is an adaptive fusion of DPABlock and CATBlock. DPABlock embeds a LatentPixelAttention (LPA) module to perform self-attention operations in the latent space, where each pixel position is flattened into a one-pixel toke, forming a sequence of length H×W. Specifically, in DPABlock, the input feature map is first... Features are obtained through the LN layer. Then, extract the data through 1×1 convolution and depthwise separable convolution. ; Then, LPA computes pixel-level attention weights in the flattened spatial dimension: Contextual features are generated using a weighted summation method: Recovering spatial structure through projective convolution: The final output is as follows: FFN consists of two 1×1 convolutional layers and the nonlinear activation function ReLU.

[0063] The output of LatentBlock is These are the weight parameters.

[0064] Steps 33 and 31-32 constitute the encoding stage. This step is the decoding stage, where high-level semantic features are gradually restored to spatial details through a three-layer upsampling structure. Color information and structural semantics from the encoding process are introduced into the decoding stream via skip connections. Color refinement and structural unification are then performed using the encoding layer structure (i.e., CATBlock and TransBlock). The specific process can be described as follows: Step 331: The outputs of the fourth coding layer and the third coding layer are concatenated, and color refinement and structural unification are performed using the same structure as the third coding layer. Step 332: The output of the third coding layer and the output of step 331 are concatenated, and color refinement and structural unification are performed using the same structure as the second coding layer; Step 333: The output of the third coding layer and the output of step 332 are concatenated, and color refinement and structural unification are performed using the same structure as the first coding layer.

[0065] Step 34: Although the low-frequency component LL obtained after DWT decomposition loses significant high-frequency details, it still retains low-scale edge and structural information. To enhance the color stability and regional contrast of the reconstructed image, this invention designs a refining module at the end of the decoder. This module fuses the decoding results with the backbone low-frequency features and high-frequency residual information, further optimizing the continuity of color restoration and the consistency of contrast distribution, thus forming the output features of the low-frequency enhancement branch. in It is the decoder output. It is the low-frequency component obtained from step 1 decomposition.

[0066] Through steps 2 and 3, this invention performs fine modeling and enhancement of high-frequency and low-frequency information based on FFSCANet and CCFormer, respectively.

[0067] Step 4: The output features of the low-frequency enhancement branch and the high-frequency detail unlocking branch are fused and reconstructed to achieve enhancement.

[0068] In the experiments of this invention, two representative datasets, UIEB and LSUI, were selected for training and testing the proposed model. UIEB contains 890 pairs of real underwater images and their manually selected high-quality reference images; 800 pairs were randomly selected as the training set, and 90 pairs were used for testing (Test-U90). The LSUI dataset consists of 4279 pairs of real paired images; 3879 pairs were randomly selected for training, and 400 pairs were used for testing (L-400). Furthermore, to verify the model's robustness in scenarios without reference images, the U45 dataset, containing different types of degradation and covering typical underwater scenes such as blue, green, and fog, was used, comprising 45 images.

[0069] Comparison Method: This invention compares six existing mainstream UIE methods, including the probability estimation-based UIE method PUIE, the physical imaging model-generative adversarial network PUGAN, the network CCMSR which combines Retinex theory, the networks DICAM and LiteEnhanceNet which use depthwise separable convolutions, and the Transformer architecture based on frequency division processing, for a quantitative comparison.

[0070] Evaluation Metrics: This invention combines full-reference and no-reference evaluation metrics. Full-reference metrics include Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM), used to quantify the differences in overall quality and structural similarity between the enhanced image and the reference image, focusing on verifying the effectiveness of the frequency decoupling mechanism in global structure and local detail restoration. For the no-reference metric, UIQM is selected. 28] (Underwater Image Quality Measurement) and UCIQE (Underwater Color Image Quality Assessment) measure image quality from multiple dimensions such as contrast and color saturation. They introduce the ranking-based metric Uranker and the BRISQUE metric based on natural scene statistics to measure local image distortion from a structural statistical perspective, thus supplementing the evaluation of detail loss and texture destruction in underwater images.

[0071] The UIEB and LSUI datasets were used to train the proposed DFCEnhanceNet. During network training, all images were resized to a fixed size (256×256). The proposed DFCEnhanceNet was implemented using the PyTorch framework on an NVIDIA RTX 3090 Ti and trained for 100 epochs, taking approximately 13 and 20 hours respectively. DFCEnhanceNet used the ADAM optimizer (…). , Weight decay Training is performed using L2 loss and learning rate. .

[0072] Table 1 shows the specific performance comparison results on the L-400 test set, with the best values ​​underlined. DFCEnhanceNet achieved the best performance in PSNR and SSIM, demonstrating its significant advantages in image quality and structure restoration. It also leads in color and contrast metrics UCIQE and underwater image quality metric UIQM, exhibiting excellent overall performance. Overall, the method of this invention demonstrates stronger enhancement capabilities and higher image quality in comparative experiments with real reference images, outperforming current mainstream methods.

[0073] Table 1. Average evaluation metrics for all methods on the Test-L400 test set. L-400's visual counterpart Figure 5 As shown in the figures, CCMSR still exhibits green and blue color casts in images 1 and 2, failing to effectively restore true colors. DICAM does not adequately address the blue color cast in the new images 1 and 4, resulting in a grayish tone. LiteEnhanceNet still shows significant blue color cast in image 3. UWFormer performs well in eliminating color casts, but the enhanced images generally appear yellowish. PUGAN does not effectively improve color distortion in images 1 and 4. PUIE still appears grayish in images 1 and 4. In contrast, the DFCEnhanceNet proposed in this invention not only effectively corrects color deviations in the images but also significantly improves contrast and detail.

[0074] Table 2 shows the specific performance comparison results on the Test-U90 test set, with the best values ​​underlined. DFCEnhanceNet achieved the best results in PSNR, SSIM, and UCIQE, demonstrating significant advantages in image quality, structure reproduction, color, and contrast. Although it did not achieve the best in UIQM, DFCEnhanceNet still scored highly, showcasing good overall image quality.

[0075] Table 2. Average evaluation metrics for all methods on the Test-U90 test set. Test-U90's visual comparison Figure 6 As shown, DICAM and LiteEnhanceNet failed to effectively correct the blue color cast in images 1 and 2. CCMSR and PUIE did not sufficiently enhance the contrast of the images. PUGAN, while offering some improvement, suffered from overexposure and insufficient color reproduction in image 1. UWFormer performed well in color cast removal, but introduced red or yellow tones in different images. In contrast, the DFCEnhanceNet proposed in this invention showed more stable and superior performance in color cast correction, contrast enhancement, and detail preservation.

[0076] In addition to quantitative comparisons on a reference test set, the methods were also tested and compared on the publicly available no-reference dataset U45. The results are shown in Table 3. DFCEnhanceNet achieved the highest score in UCIQE, indicating its significant enhancement effect on color saturation and contrast. Meanwhile, DFCEnhanceNet also performed well in the UIQM metric, reflecting that its enhanced images have better overall quality in terms of brightness, contrast, and color fidelity. In the subjective perceived quality evaluation metric Uranker, CCMSR and PUGAN performed relatively well, but DFCEnhanceNet still ranked third with a score of 1.8777, demonstrating stable performance. On the BRISQUE metric, which measures image structural distortion, DFCEnhanceNet achieved the best value, indicating that its enhanced images most closely resemble the statistical distribution of natural images, possessing stronger structural fidelity and visual naturalness.

[0077] Table 3. Average evaluation metrics for all methods on the U45 test set. For example, the magnified visual contrast in U45 Figure 7As shown, in image1, which is predominantly bluish with low contrast and blurred details, neither DICAM nor PUIE effectively enhanced contrast. LiteEnhanceNet also exhibits a bluish tint. CCMSR, PUGAN, and UWFormer are inferior to DFCEnhanceNet in terms of color saturation and detail. In image2, with a strong green tint and severe color distortion, only CCMSR and PUGAN excel in removing the green tint. However, in detailed magnified images, DFCEnhanceNet demonstrates superior color correction capabilities, exhibiting excellent detail recovery and color correction, realistically reproducing the natural texture of objects such as rocks and fish.

[0078] To further demonstrate the effectiveness of different modules in DFCEnhanceNet, ablation experiments were conducted on test sets U-90 and U45. Key modules were removed in the ablation experiments to study their roles in the overall network. Specifically, the quantitative index values ​​for U-90 and U45 are listed in Table 4, with the best values ​​underlined. Visual comparison figures are shown below. Figure 8 Due to the introduction of channel self-attention, using only CATBlock alleviated the color shift problem to some extent, but significant color distortion still existed. With only DRM, it performed well in image structure restoration, but its ability in color correction was limited, and the overall image still had a green tint. Using only SWABlock improved detail enhancement and texture clarity, but the overall image color was unnatural, and the contrast in the background area remained low. Combining CATBlock and SWABlock significantly improved image contrast and alleviated the green color shift, but unnatural color artifacts appeared in local areas. The complete model DFCEnhanceNet performed best in detail restoration, color naturalness, and overall contrast, further demonstrating that the synergistic effect between modules is crucial for the final image enhancement effect.

[0079] Table 4 uses the average evaluation index values ​​of the proposed core components in turn. Table 4 lists the objective evaluations of each core component of the proposed model. Through continuous innovation and the introduction of better-performing modules, all five metrics used on the reference test set Tes-tU90 and the no-reference test set U45, except for Uranker, have consistently approached their optimal levels. The metric comparisons and visualizations demonstrate that the proposed model has achieved significant progress in feature and detail recovery, with each component playing a crucial role in reconstructing high-quality images.

[0080] In summary, this invention proposes a frequency-decoupled underwater image enhancement network—DFCEnhanceNet—aimed at addressing common problems in underwater images such as color distortion, low contrast, blurring, and loss of detail. After decomposing the image into low-frequency and high-frequency components, a low-frequency enhancement branch, CCFormer, and a high-frequency detail unlocking branch, FFSCANet, are designed to model and enhance color distribution and local texture details, comprehensively improving image quality. Experimental results show that the proposed method outperforms existing mainstream methods in color correction, contrast enhancement, and detail restoration on multiple publicly available underwater image datasets, demonstrating good enhancement effects and generalization ability. Ablation experiments further validate the effectiveness of each module design. Future work will consider incorporating multimodal information and lightweight design to further improve the model's adaptability to extreme degradation environments and its real-time processing performance.

Claims

1. A method for underwater image enhancement based on frequency decoupling color correction and detail unlocking, characterized in that, Includes the following steps: Step 1: Perform multi-frequency domain decomposition on the underwater image to obtain low-frequency and high-frequency components; Step 2: Input the high-frequency components into the high-frequency detail unlocking branch, fuse the frequency domain selection attention and channel attention mechanisms, perform fine-grained modeling of high-frequency texture features, enhance realistic details, and suppress high-frequency noise at the same time; Step 3: Input the low-frequency component into the low-frequency enhancement branch, and use a self-attention mechanism to globally model and enhance the low-frequency features, adjusting the overall brightness distribution, color saturation and structural information of the image; Step 4: The output features of the low-frequency enhancement branch and the high-frequency detail unlocking branch are fused and reconstructed to achieve enhancement.

2. The underwater image enhancement method based on frequency decoupling color correction and detail unlocking according to claim 1, characterized in that, Step 1 involves decomposing the underwater image into low-frequency components and multiple high-frequency components using discrete wavelet transform; Step 2 involves merging the multiple high-frequency components into a multi-channel image as the input to the high-frequency detail unlocking branch; and Step 4 involves fusing and reconstructing the output features using inverse discrete wavelet transform.

3. The underwater image enhancement method based on frequency decoupling color correction and detail unlocking according to claim 1 or 2, characterized in that, The high-frequency detail unlocking branch performs the following steps: Step 21: Extract features using the backbone network; Step 22: The extracted features are fed into the detail unlocking module, and the following steps are performed: First, a basic feature representation F is constructed, and then F is passed through the channel attention module to generate a channel attention map. Then F and Feature fusion is performed to obtain weighted features, and then the weighted features are used to generate a spatial attention map through a spatial attention module. Then combine the weighted features with Feature fusion is performed to obtain joint enhanced features. Finally The extracted features are fused with residual connections and enhanced by convolution and ReLU activation to obtain the output features of the high-frequency detail unlocking branch.

4. The underwater image enhancement method based on frequency decoupling color correction and detail unlocking according to claim 3, characterized in that, Step 21, which involves extracting features using the backbone network, is implemented as follows: Step 211: Perform preliminary feature extraction on the input high-frequency components through convolutional layers; Step 212: The initially extracted features are fed into a residual module composed of multiple fast Fourier convolutions to capture global texture patterns and minute structural details in the frequency domain. Step 213: The output of the residual module is compressed and fused through convolution and batch normalization layers, and then added to the initially extracted features through skip connections to obtain the extracted features.

5. The underwater image enhancement method based on frequency decoupling color correction and detail unlocking according to claim 1 or 2, characterized in that, The low-frequency enhancement branch performs the following steps: Step 31: Map the low-frequency components to a low-dimensional feature space through image segmentation and linear embedding to obtain a low-level feature representation. ; Step 32, The data is fed into a four-layer encoding module. Each encoding layer contains a Transformer structure and spatial downsampling operations to gradually extract high-level semantic features. The first coding layer uses pixel self-attention, the second and third coding layers use channel self-attention, and the fourth coding layer uses shift window self-attention. Each coding layer introduces channel and spatial attention mechanisms. Step 33: In the decoding stage, the high-level semantic features are gradually restored to spatial details through a three-layer upsampling structure, and the color information and structural semantics in the encoding process are introduced into the decoding stream through skip connections. Color refinement and structural unification are performed in combination with the encoding layer structure. Step 34: The decoding result is fused with the low-frequency features of the backbone and the high-frequency residual information through the refining module to obtain the output features of the low-frequency enhancement branch.

6. The underwater image enhancement method based on frequency decoupling color correction and detail unlocking according to claim 5, characterized in that, The four-layer encoding modules are as follows: The first coding layer uses TransBlock, which is an improvement on the Swing Transformer. The second and third coding layers both use channel self-attention Transformer modules, and a lightweight convolutional attention module is introduced before the module input; The fourth encoding layer uses LatentBlock to model the entire image without using sliding windows, achieving global color consistency modeling with low computational complexity.

7. The underwater image enhancement method based on frequency decoupling color correction and detail unlocking according to claim 6, characterized in that, Both the second and third coding layers are CATBlocks. The execution steps of a CATBlock are as follows: Given input feature map The processing procedure of the convolutional attention module is represented as follows: in and These represent the channel attention and spatial attention modules, respectively. This represents element-wise multiplication, and the output is the enhanced feature. ; The processing procedure of the channel self-attention Transformer module is represented as follows: in, The output features of the channel self-attention Transformer module, This indicates that the attention results will be rearranged. For channel self-attention queries, keywords, and values; Where conv is a 1×1 pointwise convolution operation and dconv is a 3×3 depthwise convolution operation; The formula for channel self-attention is as follows: in Number of channels; The output of CATBlock is ultimately expressed in a typical Transformer structure, as follows: Where LN represents layer normalization and FFN represents feedforward sublayer.

8. The underwater image enhancement method based on frequency decoupling color correction and detail unlocking according to claim 7, characterized in that, The TransBlock is an adaptive fusion of CSWBlock and CATBlock. The execution steps of CSWBlock are as follows: Given input feature map Enhanced features are obtained through convolutional attention modules. The output of CATBlock is represented as: Where RW is a regular window, WR is a flipped window, and SW is a shifted window. These are intermediate features used in the calculation; S represents the input feature map. resolution, The formula for representing pixel self-attention is as follows: The output of TransBlock is in These are the weight parameters.

9. The underwater image enhancement method based on frequency decoupling color correction and detail unlocking according to claim 7, characterized in that, The LatentBlock is an adaptive fusion of DPABlock and CATBlock. The execution steps of DPABlock are as follows: First, the input features are passed through an LN layer, and then extracted using a 1×1 convolution and a depthwise separable convolution. ; X is the output of the LN layer; Then, pixel-level attention weights are calculated in the flattened spatial dimension: Contextual features are generated using a weighted summation method: Recovering spatial structure through projective convolution: The final output is as follows: FFN consists of two 1×1 convolutional layers and the nonlinear activation function ReLU.

10. The underwater image enhancement method based on frequency decoupling color correction and detail unlocking according to claim 5, characterized in that, Step 33 is implemented as follows: Step 331: The outputs of the fourth coding layer and the third coding layer are concatenated, and color refinement and structural unification are performed using the same structure as the third coding layer. Step 332: The output of the third coding layer and the output of step 331 are concatenated, and color refinement and structural unification are performed using the same structure as the second coding layer; Step 333: The output of the third coding layer and the output of step 332 are concatenated, and color refinement and structural unification are performed using the same structure as the first coding layer.

Images